 |
Struggling with stylistic consistency, or wanting to transpose the style of your best picture onto the rest of your Instagram feed? Thanks to a group of scientists at Cornell University, you can now do just that with surprisingly accurate and realistic results.
The team created an algorithm for graphics card company NVIDIA that lifts the stylistic characteristics of one picture and drops them onto a completely different image with startling precision. The algorithm is called FastPhotoStyle, and it’s capable of transferring the coloration, drama and atmosphere of one picture and making an entirely different frame look as though it was taken at the same time even if the subject matter is totally unrelated.
According to the developers, the goal of photorealistic image style transfer is:
…to change the style of a photo to resemble that of another one. For a faithful stylization, the content in the output photo should remain the same, while the style of the output photo should resemble the one of the reference photo. Furthermore, the output photo should look like a real photo captured by a camera.
 |
There are programs already invented to do this, but the inventors of this algorithm claim that what already exists is slow, and doesn’t produce realistic results anyhow.
FastPhotoStyle is different, they say, because it uses a smoothing process after the initial whitening and Coloring Transfer step—or PhotoWCT step. This smoothing step tries to ensure that neighboring pixels receive similar styling and, by using what they call Matting Affinity, individual areas of the image can be subjected to slightly different treatment. This is what helps the algorithm produce such realistic looking results.
Another major difference is that this program reportedly operates as much as 60x faster than existing algorithms.
The code can be downloaded from NVIDIA’s GitHub for anyone to use under Creative Commons license (BY-NC-SA 4.0), and a user manual download is included on the page. If you’re brave, you can read the full technical paper as well.
Technical Paper Abstract:
A Closed-Form Solution to Photorealistic Image Stylization
Photorealistic image style transfer algorithms aim at stylizing a content photo using the style of a reference photo with the constraint that the stylized photo should remains photorealistic.
While several methods exist for this task, they tend to generate spatially inconsistent stylizations with noticeable artifacts. In addition, these methods are computationally expensive, requiring several minutes to stylize a VGA photo. In this paper, we present a novel algorithm to address the limitations.
The proposed algorithm consists of a stylization step and a smoothing step. While the stylization step transfers the style of the reference photo to the content photo, the smoothing step encourages spatially consistent stylizations. Unlike existing algorithms that require iterative optimization, both steps in our algorithm have closed-form solutions.
Experimental results show that the stylized photos generated by our algorithm are twice more preferred by human subjects in average. Moreover, our method runs 60 times faster than the state-of-the-art approach.
Articles: Digital Photography Review (dpreview.com)