RSS
 

Posts Tagged ‘onchip’

Sony’s new ‘Intelligent Vision Sensor’ is the world’s first to feature on-chip AI processing

20 May

Sony has shared the details of its new sensor array that it claims is the world’s first to feature artificial intelligence (AI) directly onboard the chipset.

Before diving into the details, it’s worth noting this sensor has been produced with industrial applications in mind, but as tends to be the case with much of Sony’s other sensor technologies, it’s not difficult to imagine seeing these AI-powered technologies in Sony’s smartphone and possibly even mirrorless camera sensors down the road.

With that out of the way, let’s dive into the details. The IMX500 is a 12.3MP (4056 × 3040 pixels) backside-illuminated 1/2.3” sensor capable of 4K/60p video capture that features a new stacked design that puts the pixel chip atop the logic chip. The stacked design means the data captured from the pixel chip can immediately be parsed via Sony’s AI processing directly on the logic chip.

Processing in the sensor not only removes the need for external hardware, it also means only the relevant data needs to be output, significantly reducing the amount of data that needs to be communicated to the cloud, reducing bandwidth and increasing speed. Sony says the new sensor can capture and process the image within 3.1 milliseconds when using the MobileNet V1 image analysis model.

IMX500 (left), IMX501 (right).

In addition to faster, real-time object-tracking and processing, this setup also allows the system to export either image data and information or information alone. The ability to export the AI-derived information alone adds an additional layer of security that could be added to scenes where privacy is critical.

Illustration from Sony showing how the data output format can be customized to meet various needs.

Again, Sony specifically mentions retail and industrial equipment use-cases for this new sensor technology, so it’s not likely we’ll be seeing this in consumer camera tech anytime soon.

The IMX500 (bare chip model) is expected to start shipping in April 2020 for 10,000 JPY (~$ 94), while the IMX501 (packaged product model) is expected to start shipping in June 2020 for 20,000 JPY (~$ 188).

Articles: Digital Photography Review (dpreview.com)

 
Comments Off on Sony’s new ‘Intelligent Vision Sensor’ is the world’s first to feature on-chip AI processing

Posted in Uncategorized

 

Sony develops 22MP smartphone sensor with on-chip AF processing and video IS

17 Feb

Sony has announced a 22.5MP Exmor RS imaging sensor for smartphones. The new stacked 1/2.6″ CMOS sensor, called IMX318, is smaller than most used by current smartphones and includes on-chip processing for AF and video image stabilization. Sony has used pixels measuring 1.0µm, allowing for a smaller sensor with higher resolution to be included in likewise slimmer mobile devices. For comparison, this sensor is 20% smaller than the 1/2.3″ chip commonly used in smartphones.

Like many current smartphone sensors, the IMX318 uses phase-detection AF elements combined with contrast-detection to create ‘hybrid AF,’ and the company claims AF reporting 0.03 seconds. According to Sony, IMX318 has such an efficient data interface that pixels can be read at 30 fps from across the entire surface. This allows the company to use the whole 22.5MP resolution for video recording – although the footage is scaled to 4K – and full resolution stills to be captured during video shooting. HD movies can be recorded at up to 120 fps, and 720p footage at 240 fps.

While this sensor is most suitable for the smartphone market, the technology could soon move into larger sensors for compact, mirrorless and DSLR cameras. IMX318 will go on sale in May, according to Sony, at a price of ¥2000/$ 17.50/£12.30 per unit. For more information see the Sony website. Below, you can see the sensor’s image stabilization system in action. 

Press release:

Sony Announces a New Type 1/2.6 22.5 Megapixel Exmor RS™, the Industry’s First*1 Stacked CMOS Image Sensor with Built-in Hybrid Autofocus and 3-Axis Electronic Image Stabilization

Tokyo, Japan – Sony Corporation (“Sony”) is announcing the commercialization of a new Exmor RS™ image sensor for smartphones and other devices that require increasingly better cameras and thinner form factors. The IMX318 is a type 1/2.6 stacked CMOS image sensor with 22.5 effective megapixels, and it boasts a more compact size, greater image quality, and higher performance. This new Exmor RS is the industry’s first*1 to be equipped with built-in high-speed hybrid autofocus (“AF”), as fast as 0.03 seconds*2, and built-in 3-axis electronic image stabilization technology for video. Sony aims to begin shipments for this new CMOS image sensor in May, 2016.

*1: According to Sony research (as of February 16, 2016)
*2: The time it takes to bring the picture into focus when the distance from the sensor surface to the subject is reduced from 2m to 2cm, given high-light conditions (2,000lux or higher).

The IMX318 boasts a stacked structure and the industry’s smallest unit pixels, which measure 1.0?m (micrometers) in size. With this, the image sensor achieves a compact, type 1/2.6 size suitable for inclusion into smartphones, while still realizing a high resolution of 22.5 effective megapixels, among the top in the industry. This new image sensor not only boasts enhanced resolution, but it also matches its predecessor in image quality despite the IMX318’s smaller size and unit pixels (IMX230 predecessor model: type 1/2.4 sensor with 1.12?m unit pixel size). Additionally, Sony has succeeded in equipping the IMX318 with cutting-edge digital imaging features, namely high speed built-in hybrid AF and 3-axis electronic image stabilization for video, a first*1 for the industry. With this image sensor, the user can capture image stills of those decisive moments reliably in high quality, as well as high resolution video without fear of blur.

Main Features

1. The industry’s smallest unit pixel size at 1.0?m, achieving compact size (type 1/2.6) while realizing 22.5 effective megapixels high resolution

As smartphones grow ever thinner, so too are image sensors growing increasingly more compact. In line with this trend, Sony has developed a miniscule 1.0?m pixel sensor that, despite its small size, realizes high image quality. To accomplish this, Sony employed manufacturing technology that improves light utilization efficiency, as well as circuit design technology that eliminates noise, a root cause of deterioration in image quality. With this innovation, Sony has realized an image sensor that delivers high quality images, while also boasting small optical size and a high resolution of 22.5 megapixels. Night shots have long been a weak point for the compact cameras used in smartphones, with the lack of light translating into excessive visual noise. The IMX318 addresses this weakness and realizes beautiful photography in nighttime conditions.
*
No drop in image quality from the IMX230 predecessor model (Left: 1.12?m unit pixel size) to the IMX318 (Right: 1.0?m unit pixel size), despite the latter’s smaller size

2. The industry’s first*1 image sensor with built-in hybrid AF, as fast as 0.03 seconds*2, and 3-axis electronic image stabilization

Hybrid AF
Hybrid AF, which merges image plane phase detection AF with contrast detection AF, has previously been realized through the combination of an image sensor and an application processor. But with the IMX318, Sony has created the industry’s first*1 stacked CMOS image sensor with hybrid AF built into the sensor’s internal signal processor. The IMX318 leverages Sony’s high-speed AF technology that has been honed over many years, boasting an AF as fast as 0.03 seconds*2 (and as fast as 0.017 seconds when shooting video at 60fps). With this power in hand, the user can capture those decisive moments reliably in focus, whether shooting stills or video.
*
When shooting with hybrid AF: 0.03 seconds
(Time required for focusing)
Image plane phase detection AF: Utilizes dedicated image plane phase detection AF pixels embedded throughout the image sensor pixel array.
Contrast AF: Works off the contrast of the subject.

3-axis electronic image stabilization for video
With the IMX318, Sony has also created the industry’s first*1 stacked CMOS image sensor with image stabilization functionality built into its internal signal processor. This sensor leverages the image stabilization technology that Sony has cultivated over its years developing cameras, realizing smooth 4K*3 videos with little camera shake by making effectively use of the signal output obtained from the external 3-axis (pitch, yaw, and roll) gyro sensor*4. Sony’s unique image stabilization technology incorporated into the IMX318 corrects not only camera shake, but also lens distortion, making for more beautiful videos. Furthermore, because image stabilization is achieved by processing within the hardware of the image sensor, less power is used than when it is accomplished by software processing in the external application processor. Since it enables smooth video shooting, this image sensor is suited for incorporation into not only smartphones, but also a variety of other products that tend to generate substantial camera shake, such as aerial drones used for image capture.

*3: QFHD (3840 x 2160)
*4: Gyro sensor: instrument that determines angular velocity through inertia.

3. 4K and high frame rate video recording, through high-speed communications leveraging the latest MIPI specifications

Sony elected to adopt the MIPI (Mobile Industry Processor Interface) Alliance’s latest C-PHY 1.0/D-PHY 1.2 specifications for this image sensor’s interface. With this interface, the IMX318 is able to achieve more power efficient and faster data transmission from the image sensor to the application processor. As a result, even at the high resolution of 22.5 megapixels, transmission is realized for all pixels at 30fps. This enables seamless switching between high resolution video and still image photography, realizing the best of both worlds. Since the image sensor can transmit greater-than-4K resolution images for all pixels to the application processor, the user can capture 22.5 megapixel still images even while in the middle of recording video in 4K*3 at 30fps

Articles: Digital Photography Review (dpreview.com)

 
Comments Off on Sony develops 22MP smartphone sensor with on-chip AF processing and video IS

Posted in Uncategorized