MIT has developed a new machine learning technology that is able to retouch an image in real-time, presenting the photographer with the final product before they snap the photo. The system, which is being presented by both Google and MIT researchers at the digital graphics conference Siggraph in Los Angeles, was trained using thousands of both raw and retouched images that comprise a dataset created by MIT and Adobe, among others.
The automatic image retouching system is lightweight enough to be used on an ordinary smartphone, and it can be equipped with multiple different styles. While real-time image retouching isn’t a new idea, the ability to perform it on a smartphone is. Talking about this, researcher Jon Barron explains:
Using machine learning for computational photography is an exciting prospect but is limited by the severe computational and power constraints of mobile phones. This paper may provide us with a way to sidestep these issues and produce new, compelling, real-time photographic experiences without draining your battery or giving you a laggy viewfinder experience.
In order for the image retouching to work in real-time on low power devices like smartphones, the researchers developed a system that modifies a low-resolution version of the final image, then translates those edits to the full-resolution photo.
With this method arose a big problem, however. Namely, that the low-resolution image doesn’t contain enough data to adequately determine edits for the full-resolution version. Two different solutions were found to solve this problem, the first being that the system outputs ‘a set of simple formulae’ rather than the low-res image itself, and the second being a method for applying the formulae to the full-resolution photo. The team explains that technology in detail in the video below:
Articles: Digital Photography Review (dpreview.com)
