Sony has shared the details of its new sensor array that it claims is the world’s first to feature artificial intelligence (AI) directly onboard the chipset.
Before diving into the details, it’s worth noting this sensor has been produced with industrial applications in mind, but as tends to be the case with much of Sony’s other sensor technologies, it’s not difficult to imagine seeing these AI-powered technologies in Sony’s smartphone and possibly even mirrorless camera sensors down the road.
 |
With that out of the way, let’s dive into the details. The IMX500 is a 12.3MP (4056 × 3040 pixels) backside-illuminated 1/2.3” sensor capable of 4K/60p video capture that features a new stacked design that puts the pixel chip atop the logic chip. The stacked design means the data captured from the pixel chip can immediately be parsed via Sony’s AI processing directly on the logic chip.
 |
Processing in the sensor not only removes the need for external hardware, it also means only the relevant data needs to be output, significantly reducing the amount of data that needs to be communicated to the cloud, reducing bandwidth and increasing speed. Sony says the new sensor can capture and process the image within 3.1 milliseconds when using the MobileNet V1 image analysis model.
 |
| IMX500 (left), IMX501 (right). |
In addition to faster, real-time object-tracking and processing, this setup also allows the system to export either image data and information or information alone. The ability to export the AI-derived information alone adds an additional layer of security that could be added to scenes where privacy is critical.
 |
| Illustration from Sony showing how the data output format can be customized to meet various needs. |
Again, Sony specifically mentions retail and industrial equipment use-cases for this new sensor technology, so it’s not likely we’ll be seeing this in consumer camera tech anytime soon.
The IMX500 (bare chip model) is expected to start shipping in April 2020 for 10,000 JPY (~$ 94), while the IMX501 (packaged product model) is expected to start shipping in June 2020 for 20,000 JPY (~$ 188).
Articles: Digital Photography Review (dpreview.com)