Researchers with MIT’s Computer Science & Artificial Intelligence Laboratory (CSAIL) have published a study detailing the use of cameras and artificial intelligence to recreate hidden actions based only on the shadows they cast. The method produces fairly low-quality results at this time but may be refined for future computational photography purposes that include helping self-driving cars ‘see’ hidden objects in their environment.
Shadows can reveal the presence of things a person may not be able to directly see; in the most obvious example, someone could, for example, perceive that a person is standing around a nearby corner because of the shadow they cast on the sidewalk. Though humans can perceive the movement of objects using their shadows, we cannot determine their colors and may not be able to determine their shape.
The newly detailed MIT AI can, however, recreate videos that include hints about an object’s color and shape based on the shadows it produces. As demonstrated in the video above, the AI was surprisingly capable of recreating the movement and general shape of hands and forearms in motion out of view of the camera. As well, the algorithm generated a video of hands moving large blocks and a small ball, recreating part of each object’s color.
This is the latest example of researchers combining cameras and artificial intelligence to produce seemingly magical results. This past summer, for example, experts with Facebook Research and the University of Washington unveiled an algorithm that can generate ‘living’ animations from individual still images.
Articles: Digital Photography Review (dpreview.com)