Methods and apparatus for performing analytics on image data

Abstract

Methods and apparatus for applying data analytics such as deep learning algorithms to sensor data. In one embodiment, an electronic device such as a camera apparatus including a deep learning accelerator (DLA) communicative with an image sensor is disclosed, the camera apparatus configured to evaluate unprocessed sensor data from the image sensor using the DLA. In one variant, the camera apparatus provides sensor data directly to the DLA, bypassing image signal processing in order to improve the effectiveness the DLA, obtain DLA results more quickly than using conventional methods, and further allow the camera apparatus to conserve power.

What is claimed is: 
     
         1 . An integrated circuit apparatus, comprising:
 a first processor apparatus configured to execute non-deterministic machine-based intelligence algorithms on first data obtained from a sensor apparatus to generate output relating to one or more aspects of the first data; and   a second processor apparatus configured to execute deterministic sensor data processing algorithms on the first data to produce human-cognizable data corresponding to the one or more aspects.   
     
     
         2 . The integrated circuit apparatus of  claim 1 , wherein the first and second processor apparatus comprise first and second integrated circuits optimized for their respective processing. 
     
     
         3 . The integrated circuit apparatus of  claim 2 , wherein the first and second integrated circuits optimized for their respective processing comprise an image signal processor and an FPGA, respectively. 
     
     
         4 . The integrated circuit apparatus of  claim 2 , wherein the first and second integrated circuits optimized for their respective processing comprise an image signal processor and an ASIC, respectively. 
     
     
         5 . The integrated circuit apparatus of  claim 1 , wherein the sensor apparatus comprises an image sensor. 
     
     
         6 . The integrated circuit apparatus of  claim 5 , wherein the human-cognizable data comprises an image, and wherein the image is output to a display for viewing by a human. 
     
     
         7 . The integrated circuit apparatus of  claim 5 , wherein the human-cognizable data comprises an image, and wherein the image is saved or transmitted to another computing device as an image file. 
     
     
         8 . The integrated circuit apparatus of  claim 5 , wherein image data captured by the sensor apparatus is processed with image signal processing including a color filter array, and wherein the image data processed with the color filter array is the first data obtained from the sensor apparatus. 
     
     
         9 . The integrated circuit apparatus of  claim 8 , wherein the color filter array is a Bayer filter or a red green blue (RGB) filter. 
     
     
         10 . A method of processing data generated from sensor apparatus, the method comprising:
 obtaining sensor data from a sensor apparatus;   processing the sensor data using a first processing entity to produce non-deterministic results; and   selectively processing the sensor data using a second processing entity to produce deterministic results.   
     
     
         11 . The method of  claim 10 , wherein the selectively processing the sensor data using a second processing entity to produce deterministic results is based at least in part on the non-deterministic results. 
     
     
         12 . The method of  claim 10 , wherein the processing the sensor data using a first processing entity comprises using a machine-learning based processing entity to process the sensor data. 
     
     
         13 . The method of  claim 12 , wherein the processing the sensor data using a machine-learning based processing entity comprises iterative or update processing of a plurality of successive sets of sensor data to converge on a learning hypothesis. 
     
     
         14 . The method of  claim 10 , wherein the processing the sensor data comprises processing the sensor data in an uncompressed, non-lossy format. 
     
     
         15 . The method of  claim 10 , wherein the selectively processing the sensor data using a second processing entity to produce deterministic results comprises (i) deleting at least a portion of the sensor data, and (ii) compressing a remaining non-deleted portion of the sensor data using a lossy compression algorithm. 
     
     
         16 . A device comprising:
 a sensor apparatus;   a deep learning accelerator in communication with the sensor apparatus; and   a signal processing apparatus in communication with the sensor apparatus,   wherein the sensor apparatus is configured to obtain sensor data and provide the sensor data to each of the deep learning accelerator and the signal processing apparatus,   wherein the deep learning accelerator is configured to process the sensor data using at least one machine learning algorithm to generate machine learning results based on the sensor data,   wherein the signal processing apparatus is configured to process the sensor data to output data comprising pixel values configured to be translated into an image, and   wherein the deep learning accelerator is configured to process the sensor data to generate the machine learning results independent of the signal processing apparatus processing the sensor data to output the data comprising the pixel values.   
     
     
         17 . The device of  claim 16 , wherein the sensor apparatus comprises an image sensor. 
     
     
         18 . The device of  claim 17 , wherein the machine learning results comprises at least one of a recognition of a human, a human face, a crowd of human bodies, an animal, an organism, a type of terrain, a street sign, a vehicle, a ball, a puck, a goal in a sports contest, a movement, an action, a throw of an object, a hit of an object, a fire, or a celestial object. 
     
     
         19 . The device of  claim 17 , wherein the deep learning accelerator is configured to generate the machine learning results based on:
 detection of a face in an image captured by the image sensor, and   matching the face to reference data stored in a memory accessible deep learning accelerator.   
     
     
         20 . The device of  claim 17 , wherein additional data not obtained by the sensor apparatus is input to the deep learning accelerator for use by the deep learning accelerator to generate the machine learning results. 
     
     
         21 . The device of  claim 20 , wherein the additional data comprises at least one of location information of the apparatus, time stamp data related to the sensor data, temperature data, accelerometer data, or reference information identifying the sensor apparatus from which the sensor data was obtained.