Sony announces the world's first imaging sensors with AI capabilities baked in
The sensor uses stacked design to accommodate AI chip circuits
Only suitable for light AI inference
#IBMCodePatterns, a developer’s best friend.
#IBMCodePatterns provide complete solutions to problems that developers face every day. They leverage multiple technologies, products, or services to solve issues across multiple industries.
Click here to know moreAdvertisements
Sony today has announced the development of two new imaging sensors, the IMX500 and IMX501. What’s new about these sensors is that they pack an AI chip right into the sensor. Sony says the whole idea behind developing a sensor with an AI chip baked in is to circumvent the various issues that can come with cloud-based AI processing.
The Sony IMX500 is not a regular imaging sensor, but in fact, it utilizes Sony’s stacked sensor design. Sony had developed the ability to sandwich a DRAM layer between the optical layer of the sensor and its circuitry in order to increase the shooting buffer. This is what allowed smartphones to shoot at 960fps super slow-mo back when the feature was originally introduced. Now Sony has managed to pack an AI processing engine into this layer, instead of just having DRAM. The logic chip, as Sony calls it, includes an ISP to process the image signal, a DSP dedicated to AI signal processing and memory for the AI model. This approach has two distinct advantages. The sensor has a resolution of 12.3 megapixels and measures 1/2.3-inch diagonally.
Currently, most AI processing related to images happens on the cloud, which, according to Sony, can have privacy issues. Additionally, there’s the issue of connectivity, where if your phone loses its connection to the cloud, the AI processing comes to a halt. Sony’s approach alleviates some of these issues for sure. The AI chip can be programmed by the OEM with any kind of inference engine depending on where the sensor is deployed. Additionally, the sensor-chip assembly can be programmed to either output the image as a whole or forego the image data in favour of inference-based information. The fact that the on-board AI chip is programmable by the OEM truly makes the IMX500’s usage extremely flexible. It is interesting, however, that devices by Apple like the iPhone 11 Pro and those by Huawei like the Huawei P40 Pro utilise on-device AI processing and do not rely on the cloud.
The Sony IMX500 sounds like a sensor that could be used in a large variety of ways. For smartphones, it could take the load off of the main SoC by running inference right on the sensor, while the higher-level AI tasks can still be conducted using the cloud. The sensor can also be used in security cameras, where it could be programmed to, for example, identify whether the person entering the building is wearing a mask or not. Given times of the COVID, this could actually be a very useful thing. The possibilities, however, are limitless.
Sony has mentioned that the IMX500 has already been shipped to partner OEMs while the IMX501 will start shipping in June. We don’t expect to see this sensor in the smartphone space for now given that the industry is currently focused on the megapixel war once again.
Digit's resident camera nerd, (un)official product photographer and the Reviews Editor
Popular Mobile PhonesView All
Hot DealsView All
Digit caters to the largest community of tech buyers, users and enthusiasts in India. The all new Digit in continues the legacy of Thinkdigit.com as one of the largest portals in India committed to technology users and buyers. Digit is also one of the most trusted names when it comes to technology reviews and buying advice and is home to the Digit Test Lab, India's most proficient center for testing and reviewing technology products.
We are about leadership-the 9.9 kind! Building a leading media company out of India.And,grooming new leaders for this promising industry.