360Photography

Picturesqe is a software application that’s designed to speed up the process of selecting the best image from a sequence of frames and the most successful pictures from a day’s shooting or a project. Load your images into the program and it applies artificial intelligence and machine learning to pick the best shots based on its own criteria and common preferences in photography.

The idea is that the program analyzes the pictures as you import them, highlighting some and rejecting others, in an automated process that takes the strain out of determining which are the frames to use. All you have to do is review those Picturesqe selects and chose the best of three, for example, instead of the best of the 26 you shot in that sequence.

I have to say at this very early stage that I’m not very keen on a machine making artistic decisions for me. It’s enough to battle the wants and tastes of a pushy modern camera without then allowing a canned algorithm to replace the subtle process of appreciating line, form, focus, light, composition and atmospheric exposure. That is quite a lot to expect a computer program to do, no matter how ‘intelligent’ it is. Nevertheless, I endeavored to give Picturesque a shot.

What it does

The beginning of the process involves importing your images into Picturesqe, which sends thumbnails to its servers to create groups and then determine which pictures are the best of each group.

Picturesqe is a desktop application that runs on PC and Mac platforms, using support from the cloud to run its more power-intensive tasks. Once installed the program invites the user to import images either from an external source, such as a memory card, or from files already stored on the hard drive. As images are imported they are analyzed so that they can be formed into groups of similar-looking images and then arranged in order so that the best frames of each group are positioned at the top left of the screen.

The process of grouping and ordering is done via an algorithm that exists in the cloud – or Picturesqe’s servers – so thumbnail images are sent from your machine for inspection and the information gleaned returns to Earth to inform the arrangement of images in the Picturesqe user window.

The import process doesn’t actually copy files from the memory card onto your main desk drive but acts as a filtering system in between the two – a sort of staging post – so that you’ll eventually only have to save the best of your images. This way you’ll stop clogging your machine with pictures that will never be used and never seem to get deleted.

Does it work?

I imported the contents of a memory card into the software and it sorted the 460 image into groups according to content and what it determined would be the pictures I would want to choose the best of. Pictures it couldn’t find groups for, and lone images, are left floating freely.	While most grouping exercises go well there are exceptions, such as this group in which the images clearly do not belong together.

I found the results of an import and a dose of analysis to be a little mixed. At first I was impressed that Picturesqe was able to divide the contents of a memory card that contained street images, portraits and product shots into a number of mostly sensible groups. Images that feature the same color in about the same place get grouped easily, and those that contain the same objects in approximately the same composition are also bundled together with a decent degree of accuracy. Pictures that fall between stools are left ungrouped for us to leave floating on their own, to manually delete or to add to the group we think most appropriate.

Opening a group reveals what the program thinks of your pictures, as it orders them according to its perception of their merits and labels those it doesn’t think much of with a waste paper basket. Those with a sensitive nature should avoid this stage, as the program’s mathematical opinion will not necessarily reflect everyone’s perceptions of their artistic brilliance.

In some instances I was glad of Picturesqe’s help in selecting the best frames from each group, but in others I was left slightly mystified by the way my images had been treated – like entering a camera club competition. Pictures that I thought were the better of the group were often not placed ahead of others I thought less pleasing. On more than a few occasions pictures I had previously selected for printing were marked out for shredding by the algorithm in the cloud. I had to remind myself that it was a machine I was dealing with so I didn’t have to take things personally. I hope Picturesqe has good insurance to pay for user’s trauma therapy. 

I wouldn’t have grouped all of these images together as, while they are all from the same shoot, they represent the pool from which I’d hope to pull three quite different pictures. It doesn’t make sense, to me at least, to group uprights and landscape format images together.

The criteria for the grouping process is a little more open than I’d like as, for instance, it doesn’t seem to take into account the orientation of the image. When shooting a portrait, for example, I’ll shoot poses and locations in upright and landscape formats, and generally edit to offer the sitter one of each. To me then it doesn’t make sense to group uprights and landscape format images together as though you’d pick one or the other instead of one of each. On the whole though the grouping process does a pretty decent job of working out which pictures belong together, and will only rarely drop an odd frame into a group in which it clearly doesn’t belong.

Groups can be edited, of course, once the program has made them, and we can reject individual images from the group so that they float free in the main browser window. A process of drag-and-drop then allows us to manually create our own groups by bunching images together.

This is how the images look when they are first imported into the program – as they should.	Picturesqe though then drops the camera-dictated color, contrast and styles for a much more ‘Raw’ looking image. Removing image characteristics doesn’t make picking a winner easier.

The application works with Raw and JPEG files, and I was impressed it is able to display the latest Raw files from the Panasonic Lumix DMC-GX80/85 even before Adobe can. It does it though at the expense of any in-camera processing settings so we see the Raw ‘Raw’ data with distortions that are usually corrected before we get to view the picture. Anything you’ve shot in black and white using Raw will also be displayed in color, though for a short few seconds images are shown as we intended before Picturesqe renders them flat and with native colors. Thus we have to make our judgements without a good deal of the qualities we might often weigh-up when selecting the best of our images – such as color.

I found this quite irritating. JPEGs are rendered just as they should be though, but when fed simultaneously-shot Raw and JPEG files we only get to see the Raws but with the rendering of the JPEGs – though I found the relationship between what I’d shot and what was displayed a little inconsistent across camera brands. If you only shoot Raw though you get to look at slightly flat and unsharp images.

Judgement Day

It seems that the ‘best’ picture (top left) has been selected by the software because the subject is quite central in the frame. The picture the software likes least (bottom right) shows the subject against a distracting background.

It’s not completely clear on what grounds the program and analysis makes its judgements as it orders our images from good to bad. I spotted that it seems to like a subject that’s in the middle of the frame, and one that sits on a third, and that it can tell within a group when the subject is against a clear background and when it’s against one that’s a bit distracting.

Magnifying the eye of the sitter from this group of portraits makes it clear that the software doesn’t place sharpness at the top of its list of priorities. The sharpest picture (bottom center) is actually marked with a waste paper basket, indicating that that Picturesqe thinks I should dump it. The shot it recommends is a good deal less focused.

Things that humans might take into account though the software doesn’t seem to – such as exposure, over-powering flash, focus and whether there are distractions in the background. It can work out when eyes are closed though, and it knows the difference between someone facing the camera straight on and someone at an angle – but it always prefers face-on as though that’s the safe formula for a ‘good’ picture. It can’t tell an unflattering angle from a complementary one though, and doesn’t distinguish between a well-lit subject and lighting that makes a nose look big.

The zoom feature identifies the part of the subject that you want to inspect and just magnifies that bit. The clever bit is that it doesn’t just magnify the same area of the frame of each picture, but actually identifies the part of the subject – wherever that subject is in the frame.

A very nice feature is the synchronized zoom that is great for checking focus across a number of images at the same time. With a collection of portraits, for example, you can zoom into the eye of one frame and the eye area of all the other frames get magnified along with it. What is particularly clever is that the eyes don’t have to be at the same coordinates in the picture for the program to magnify them, as it can identify similar objects in the images across the series.

This feature worked brilliantly for portraits, allowing closed eyes and missed focus to be spotted easily and early on, and on occasions it worked even when the images weren’t all in the same orientation. The success rate is reduced for landscapes and more general scenes where the element you want to inspect isn’t as distinct or as obvious as an eye, but the application can concentrate on rocks, trees and buildings so long as the object you are checking has reasonably powerful edges and outlines.

When the images are less regular, such as in these street pictures, the software finds it more difficult to identify the subject.	Using the zoom feature didn’t allow me to compare the sharpness of the subjects in the street pictures, even though they have similar outlines and contrast in each frame.

Export

Once you have whittled your selections down to the best images in each group, or to the better groups in the collection, you are ready to export the pictures to their final destination. The editing process involves deleting the images that aren’t wanted – not from the card or from their home on your hard drive, but from the imported collection. No files are actually deleted.

To do this you’ll have to click on the trash can icon on the image. And that will have to be done on every unwanted image. So, if you have shot a sequence of 100 images and only need one you have to delete 99. You can attend to each frame individually and delete each one in turn, or you can select them all and use the delete key on the keyboard.

I’m a little unhappy about the solely subtractive nature of the process and feel I’d rather just select the best frame and export that without having to deal with the wreckage of the frames that didn’t work, but the truth is it’s more of a big deal in my head that in reality. Clearing away the duds doesn’t take that long.

Images can then be exported to a chosen folder on your hard drive or directly into Adobe Lightroom for editing. 

The research program

At the export stage the first option in the navigation is to send the images to Picturesqe’s research program. The company wants users to send images that have been rated by humans so that it can compare the ‘right’ ranking and grouping with the way the software performed. The idea is that the company can study the differences and similarities and develop algorithms that select and grade more like the user does.

The uploaded images, we are assured, will never be used for anything other than research, and the company takes only a 1500×1500 pixel thumbnail that is studied by a computer and not by a human. The idea is to build a database of how images are selected to allow the software to learn and get better at its job.

In this example the software has compared a Raw file with a JPEG I processed from that same file, and has concluded that the Raw file isn’t just weaker than the JPEG, but that it should be deleted. Had I used the software to select which Raw files I would save for processing in the first place, this one wouldn’t have ever got to the black and white JPEG stage.

Conclusion

So, Picturesqe isn’t perfect. Not by a long shot, in fact, but it gets enough right that shooters who work with long sequences of the same subject will find it useful. In more general work it is not much of a chore to compare three or four frames to select which is the better, but if your photography involves long bursts of action or multiple frames of the same thing, then what it does will be enough to make it useful and to save you time. As the engineers seem to have prioritized portraiture it makes sense that this is the area in which it works best, and for which I’d most recommend it, but motor sports and general action would suit it too.

As the selection process tends to judge on more formulaic principals you’ll have more success with images that are more about content than artistic ideals. The company doesn’t claim that Picturesqe can make judgements of taste, but that it uses math and obvious standards to rank images, and that is exactly what you get.

There is a good deal of potential to make this a very powerful program for factual photographic subjects and the learning element of Picturesqe promises great things. Right now it needs a bit more work to make it of a standard that the majority of photographers will find it useful, but it is an interesting idea and even as it is will be useful for a good many.

Fortunately we don’t have to take its advice completely, and when you work together with the program, combining its literal mind with your own taste and style, it can work very well if you shoot the right sort of subject.

What we like:

• A great concept
• Easy to use
• Very good for comparing magnified views 
• It does have some success
• Good for factual images

What we don’t like:

• Success rate just isn’t high enough yet
• It doesn’t sort by orientation
• Doesn’t seem to take focus into consideration

Interview Q&A with Picturesqe CEO Daniel Szollosi

We got some time with Daniel Szollosi, the CEO and founder of Picturesqe, and questioned him about the way the application works and some of the issues we picked up during the review process.

DPReview: The process to get to the pictures photographers want to save and work on means they have to delete those that they don’t. So, in a collection of 100 pictures to get to one you want to keep you need to delete 99. Is there a way to just export the one you want instead of going through the process of deleting the 99?

Daniel Szollosi: We realize it is not a perfect workflow concept, you are totally right. Originally our primary goal was to get rid of the digital trash – so image deletion was the main focus. Since then we have come up with a new workflow, which directly helps the selection of top quality photos. The new workflow is going to be released in the next version of Picturesqe.

DPR: Raw files don’t always look the way they were shot – when they are supposed to be black and white, for example. Does that only happen when there is no simultaneously shot JPEG imported with it?

DS: We know about this bug. It’s related to white balance, we have problems with setting the right value yet. 

DPR: Your program displays Raw files from cameras that are new and which Adobe can’t display yet. How does that happen?

DS: We’re using a 3rd party library for decoding Raw camera images. Another advantage of this library is that the user doesn’t need to install any kind of camera drivers. 

DPR: The program doesn’t always detect when images are out of focus – is it supposed to or is that something you are working on?

DS: We are definitely working on it! In the next version we expect a really big improvement regarding focus detection. We have developed a technology which seems to be better than the current state-of-the-art regarding local sharpness/blur evaluation. 

DPR: Picturesqe seems to prioritize images where the subject is in the centre of the frame, or on a third. Is that part of the analysis? What other factors are taken into account?

DS: We do take into account composition factors, like rule of thirds. The quality factors we take into account are:

• Location of visually attended area
• Exposure (globally and in the visually attended area)
• Focus (global blur, wrong focus localization)
• Lighting distribution
• Color harmony
• Composition
• Optical distortions
• Visual noise

DPR: In some cases the program recommends deleting a Raw file but promotes a JPEG processed from that file to the top of the stack. Why does that happen?

DS: Thanks for mentioning this, we have not thought about it. We resize the images to a smaller size when evaluating the quality and aesthetics and on this scale Raw information does not exist anymore. We can easily implement a filter which prioritize Raw images when compared to its JPEG descendant.

DPR: Does Picturesqe assess exposure and the content of the background?

DS: Semantically we do not interpret the content of the background, but visually we assess the background and the foreground separately. The quality attributes calculated from the foreground have more weights.

Articles: Digital Photography Review (dpreview.com)