 |
NVIDIA researchers have published a new paper detailing their latest artificial intelligence work, which involves generating photo-realistic portraits of humans that are indistinguishable from images of real people. The technology revolves around an alternative generator architecture for generative adversarial networks (GANs) that utilizes style transfer for producing the final result.
Though GANs have improved substantially in only a few years, the researchers say in their paper that the generators ‘continue to operate as black boxes, and despite recent efforts, the understanding of various aspects of the image synthesis process, e.g., the origin of stochastic features, is still lacking.’ That’s where the newly developed alternative architecture comes in.
The team’s style-based architecture enables GANs to generate new images based on photos of real subjects, but with a twist: their generator learns to distinguish between separate elements in the images on its own. In the video above, NVIDIA’s researchers demonstrate this technology by generating portraits based on separate elements from images of real people.
“Our generator thinks of an image as a collection of ‘styles,’ where each style controls the effects at a particular scale,” the team explains.
Image elements are split into three style categories: “Coarse,” “Middle,” and “Fine.” In terms of portraits, these categories include elements like facial features, hair, colors, eyes, the subject’s face shape, and more. The system is also able to target inconsequential variations, including elements like texture and hair curls/direction.
The video above demonstrates changes involving inconsequential variation on non-portrait images, which includes generating different patterns on a blanket, altering the hair on a cat, and subtly changing the background behind a car. The style-transfer GANs offer superior results to traditional GAN generator architecture, the researchers conclude, with the photo-realistic results underscoring their assessment.
The latest work further refines a technology that has been growing rapidly over only a few years. Though GANs have been used in the past to generate portraits, the results were far from photo-realistic. It’s possible that technology like this could one day be offered as a consumer or enterprise product for generating on-demand life-like images.
Articles: Digital Photography Review (dpreview.com)