Vehicle surrounding awareness system

Abstract

The proposed innovative vehicle awareness surrounding system, based on radar sensing, provides visual information to the vehicle driver, with a maximum field of awareness larger than 300 degrees, typically better than 330 degrees, an affordable total system cost, by using two radar modules, each having a 180-degree field of view, with processing on the radar module. There is no explicit need for processing on domain controllers or vehicle processing units, proposed for autonomous driving. The proposed system utilizes artificial intelligence methodology using radar point cloud data for classifying the objects around the vehicle. An artificial intelligence-backed classification is executed, being previously trained by sets of actual radar measurements annotated data. Additionally, the proposed system may issue alerts to the driver, provide information to the vehicle control system to make autonomous actions, and wirelessly send surrounding data to the cloud. Moreover, the proposed system may provide parking support.

1 . System providing vehicle surrounding awareness, comprising of:
 two mmWave radar sensor modules, each having a 180-degree field of operation, where the field of operation is defined by a 6 dB antenna radiation diagram drop, having identical hardware components, and having the same respective hardware arrangement inside the said mmWave sensor modules, and having access over the vehicle's communication network to a display part, wherein the display part is positioned in the visual field of the driver;   wherein each of the said mmWave radar sensor modules is positioned on the vehicle's rear corners, where the rear side of the vehicle is defined as being opposite to the main driving direction side;   wherein the radiation diagram in the elevation plane of the said mm Wave radar sensor modules has maximum radiation in the horizontal plane, being parallel to the ground plane;   wherein the radiation diagram in the azimuth plane of the said mmWave radar sensor modules has maximum radiation in the plane being perpendicular to the back sides of said mmWave radar sensor modules;   wherein the back side of one said mmWave radar sensor is building an angle larger than 60 degrees with the back side of one said mmWave radar sensor modules;   wherein at least one of one said mmWave radar sensor modules has digital processing hardware functionality, performing a classification of objects, being detected by the said mmWave radar sensor modules, using radar sensor point cloud data, being previously generated by other functionality, within the said mmWave radar sensor modules;   wherein at least one of one said mmWave radar sensor modules is sending classification information about said objects around the vehicle, objects distances to vehicle, object relative positions to the vehicle, and relative speed of the vehicles; wherein basic classification categories are: passenger vehicles, large commercial vehicles not exceeding 10 meters in length and motorbikes;   wherein classification information is physically sent from at least one of one said mmWave radar sensor modules over a 4-pin connector;   wherein the said display part, being positioned in the visual field of the driver, uses the said classification information and shows dynamic positions of the said objects, wherein the objects are represented by artificial pictures, related to said basic classification categories, wherein the total view coverage in the azimuth plane is larger than 300 degrees.   
     
     
         2 . System according to  claim 1 ,
 wherein the said mmWave sensor modules have connectors with more than 4 pins.   
     
     
         3 . System according to  claim 2 ,
 wherein the said basic classification categories are extended to the detection of bicycles.   
     
     
         4 . System according to  claim 3 ,
 wherein the said basic classification categories are extended to the detection of long commercial vehicles, exceeding 10 meters in length.   
     
     
         5 . System according to  claim 4 ,
 wherein the said basic classification categories are extended to the detection of three-wheel commercial vehicles, whose length does not exceed 3 meters.   
     
     
         6 . System according to  claim 1 ,
 wherein the classification of objects around the vehicle having the said system is performed by artificial intelligence functionality using pre-calculated point cloud data,   wherein artificial intelligence functionality is making a classification decision, being previously trained by radar system annotation data, wherein the said radar system annotation data are related to the classification categories,   wherein the speed of objects around the vehicle is considered in conjunction with the point cloud radar data in the process of artificial intelligence computation.   
     
     
         7 . System according to  claim 1 ,
 wherein the surrounding awareness information is continuously updated on the said display and is visually accessible to the said driver.   
     
     
         8 . System according to  claim 1 ,
 where the surrounding awareness information is used to issue warnings to the said driver.   
     
     
         9 . System according to  claim 7 ,
 wherein the surrounding awareness information is sent by the said vehicle's wireless connectivity to the cloud, to be accessed by other systems.   
     
     
         10 . System according to  claim 1 ,
 where the surrounding awareness information is used by the vehicle infrastructure to perform actions autonomously, wherein actions change the dynamical behavior of the said vehicle.   
     
     
         11 . System according to  claim 1 ,
 where the surrounding awareness information is used additionally as a parking support and provides warning distances to the vehicle's nearby objects without the necessity to execute the classification of the nearby objects.   
     
     
         12 . Method implemented in a mmWave sensor signal processing HW for providing vehicle surrounding awareness using the system according to  claim 1 , the method comprising:
 performing a classification of objects around the vehicle to obtain a classification information; and   using said classification information to show dynamic positions of the objects relative to the vehicle.   
     
     
         13 . Method according to  claim 12 , wherein the classification of objects around the vehicle is performed by artificial intelligence functionality using pre-calculated point cloud data,
 wherein artificial intelligence functionality is making a classification decision, being previously trained by radar system annotation data, wherein the said radar system annotation data are related to the classification categories,   wherein the speed of objects around the vehicle is considered in conjunction with the point cloud radar data in the process of artificial intelligence computation.   
     
     
         14 . Method according to  claim 12 ,
 wherein said artificial intelligence functionality is using more than one algorithmic approaches: support vector machines (SVM) with decisions trees, multiply layer perception (MLP), convolutional neural network (CNN) and vision transformer (ViT), being applied on said radar point cloud data.   
     
     
         15 . Computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of  claim 12 .