Signal processing apparatus and augmented reality apparatus for vehicle having same

Abstract

A signal processing device and an augmented reality apparatus for a vehicle including the same are disclosed. The signal processing device according to an embodiment of the present disclosure includes: a memory to store map data; and a processor to generate a graphical overlay based on camera data from a camera, the map data, or sensor data from a sensor device, wherein the processor is configured to execute a plurality of virtual machines on an hypervisor, and generate a first graphical overlay through a first augmented reality engine or generate a second graphical overlay, including less data than the first graphical overlay, through a second augmented reality engine, in response to the first augmented reality engine being stopped, output the second graphical overlay instead of the first graphical overlay. Accordingly, an augmented reality-based overlay may be stably provided.

1 . A signal processing device comprising:
 a memory configured to store map data; and   a processor configured to generate a graphical overlay based on camera data from a camera, the map data, or sensor data from a sensor device,   wherein the processor is configured to:   execute a plurality of virtual machines on an executed hypervisor,   detect an object from the camera data through a first engine executed in one of the plurality of virtual machines,   transmit the detected object to a second engine through a shared memory based on the hypervisor,   generate guide information related to vehicle traveling based on the sensor data from the sensor device and the map data from the memory through a third engine executed in one virtual machine among the plurality of virtual machines,   transmit the generated guide information related to vehicle traveling to the second engine through the shared memory, and   generate a graphical overlay based on the generated guide information related to vehicle travel and the detected object through the second engine executed in some of the plurality of virtual machines and output the generated graphical overlay.   
     
     
         2 . The signal processing device of  claim 1 , wherein the processor is configured to superimpose the graphical overlay on the detected object. 
     
     
         3 . The signal processing device of  claim 1 , wherein a server virtual machine among the plurality of virtual machines is configured to detect the object from the camera data through the first engine, and
 wherein a guest virtual machine among the plurality of virtual machines is configured to generate the graphical overlay based on the detected object through the second engine and to output the generated graphical overlay.   
     
     
         4 . The signal processing device of  claim 3 , wherein a first guest virtual machine among the plurality of virtual machines is configured to operate for a first display, and
 wherein a second guest virtual machine among the plurality of virtual machines is configured to generate the graphical overlay based on the detected object through the second engine and to output the generated graphical overlay to an image projection device.   
     
     
         5 . The signal processing device of  claim 3 , wherein a first guest virtual machine among the plurality of virtual machines is configured to operate for a first display, and
 wherein a second guest virtual machine among the plurality of virtual machines is configured to operate for a second display and to generate the graphical overlay based on the detected object through the second engine and output the generated graphical overlay to the second display.   
     
     
         6 . The signal processing device of  claim 3 , wherein the guest virtual machine among the plurality of virtual machines is configured to generate the guide information related to vehicle traveling through the third engine based on the sensor data from the sensor device and the map data from the memory,
 wherein the second engine is configured to generate the graphical overlay based on the generated guide information related to vehicle traveling and the detected object and output the generated graphical overlay.   
     
     
         7 . The signal processing device of  claim 3 , wherein the server virtual machine among the plurality of virtual machines is configured to generate the guide information related to vehicle traveling through the third engine based on the sensor data from the sensor device and the map data from the memory,
 wherein the second engine is configured to generate the graphical overlay based on the generated guide information related to vehicle traveling and the detected object and output the generated graphical overlay.   
     
     
         8 . The signal processing device of  claim 1 , wherein an input and output resource manager executed based on the hypervisor is configured to receive the camera data from the camera or the sensor data from the sensor device and to provide the camera data or the sensor data to the first engine or the second engine. 
     
     
         9 . The signal processing device of  claim 1 , wherein the graphical overlay comprises a first layer including vehicle speed information and vehicle heading direction information, and a second layer including a preceding vehicle object and an indicator of a distance to a preceding vehicle. 
     
     
         10 . The signal processing device of  claim 1 , wherein the processor comprises a data interface configured to receive the camera data from the camera or the sensor data from the sensor device,
 wherein the second engine is configured to perform sensor fusion based on the camera data, the map data, or the sensor data, to perform geometric modeling based on a result of performing the sensor fusion, and to perform visualization based on a result of the modeling.   
     
     
         11 . The signal processing device of  claim 1 , wherein the processor comprises:
 a data interface configured to receive the camera data from the camera or the sensor data from the sensor device;   a sensor fusion processor configured to perform sensor fusion based on the camera data, the map data, or the sensor data;   a geometric modeling processor configured to perform geometric modeling based on a result of performing the sensor fusion; and   a visualization processor configured to perform visualization based on a result of the modeling.   
     
     
         12 . A signal processing device comprising:
 a memory configured to store map data; and   a processor configured to generate a graphical overlay based on camera data from a camera, the map data, or sensor data from a sensor device,   wherein the processor is configured to:   execute a plurality of virtual machines on an executed hypervisor,   detect an object from the camera data through a first engine executed in some of the plurality of virtual machines,   generate a graphical overlay based on the detected object through the second engine,   transmit the generated graphical overlay to another virtual machine among the plurality of virtual machines through a shared memory based on the hypervisor,   generate guide information related to vehicle traveling based on the sensor data from the sensor device and the map data from the memory through a third engine executed in one virtual machine among the plurality of virtual machines, and   transmit the generated guide information related to vehicle traveling to the second engine through the shared memory.   
     
     
         13 . The signal processing device of  claim 12 , wherein the processor is configured to superimpose the graphical overlay on the detected object. 
     
     
         14 . The signal processing device of  claim 1 , wherein a server virtual machine among the plurality of virtual machines is configured to detect the object from the camera data through the first engine, and to transmit the graphical overlay, generated through the second engine based on the detected object, to an augmented reality application executed in a guest virtual machine through a shared memory. 
     
     
         15 . The signal processing device of  claim 14 , wherein a first guest virtual machine among the plurality of virtual machines is configured to operate for a first display, and a second guest virtual machine among the plurality of virtual machines is configured to operate for a second display or an image projection device. 
     
     
         16 . The signal processing device of  claim 1 , wherein a server virtual machine among the plurality of virtual machines is configured to transmit the generated graphical overlay to a second guest machine through the shared memory. 
     
     
         17 . The signal processing device of  claim 15 , wherein a server virtual machine among the plurality of virtual machines is configured to transmit the generated graphical overlay to the first guest machine through the shared memory. 
     
     
         18 . The signal processing device of  claim 14 , wherein the server virtual machine among the plurality of virtual machines is configured to generate the guide information related to vehicle traveling through the third engine based on the sensor data from the sensor device and the map data from the memory,
 wherein the second engine is configured to generate the graphical overlay based on the generated guide information related to vehicle traveling and the detected object and output the generated graphical overlay.   
     
     
         19 . The signal processing device of  claim 14 , wherein an input and output resource manager executed based on the hypervisor is configured to receive the camera data from the camera or the sensor data from the sensor device and to provide the camera data or the sensor data to the first engine or the second engine. 
     
     
         20 . An augmented reality apparatus for a vehicle, the apparatus comprising:
 at least one camera; and   a signal processing device including a processor configured to generate a graphical overlay based on camera data from the camera,   wherein the signal processing device comprises:   a memory configured to store map data; and   a processor configured to generate a graphical overlay based on camera data from a camera, the map data, or sensor data from a sensor device,   wherein the processor is configured to:   execute a plurality of virtual machines on an executed hypervisor,   detect an object from the camera data through a first engine executed in one of the plurality of virtual machines,   transmit the detected object to a second engine through a shared memory based on the hypervisor,   generate guide information related to vehicle traveling based on the sensor data from the sensor device and the map data from the memory through a third engine executed in one virtual machine among the plurality of virtual machines,   transmit the generated guide information related to vehicle traveling to the second engine through the shared memory, and   generate a graphical overlay based on the generated guide information related to vehicle travel and the detected object through the second engine executed in some of the plurality of virtual machines and output the generated graphical overlay.