Teleoperation of autonomous vehicles

Abstract

An autonomous vehicle receives sensor data from one or more sensors of the autonomous vehicle, and generates a request for remote control of the autonomous vehicle by a computer system remote from the autonomous vehicle. Generating the request includes determining a quality metric associated with a network connection between the autonomous vehicle and the computer system, and upon determining that the quality metric is greater than a threshold quality level, transmitting, from the autonomous vehicle to the computer system the request for remote control and a first data item representing the sensor data. The first data item meets one or more conditions associated with the threshold quality level.

1 - 23 . (canceled) 
     
     
         24 . A method comprising:
 receiving, at a computer device, first vehicle telemetry data from each vehicle of a plurality of vehicles, wherein the vehicle telemetry data comprises, for each vehicle of the plurality of vehicles, an indication of a geographical location of the vehicle, and wherein at least one vehicle of the plurality of vehicles is an autonomous vehicle;   presenting, by a display device associated with the computer device, a user interface generated based on the first vehicle telemetry data, wherein the user interface comprises a graphical map and one or more first display elements, each first display element indicating at least a respective geographical location of a respective vehicle of the plurality of vehicles;   receiving, at the computer device, a first user input selecting a particular vehicle of the plurality of vehicles;   responsive to receiving the first user input:
 retrieving, by the computer device, second vehicle telemetry data from the selected vehicle, and 
 presenting, by the display device, a visual representation of at least a portion of the first vehicle telemetry data or a portion of the second vehicle telemetry data regarding the selected vehicle on the user interface, wherein the visual representation includes:
 raw data obtained from one or more sensors of the selected vehicle without substantially any post-processing, and 
 processed data obtained from the one or more sensors of the selected vehicle and subsequently manipulated. 
 
   
     
     
         25 . The method of  claim 24 , wherein the first vehicle telemetry data comprises, for each vehicle of the plurality of vehicles, at least one of
 geographical coordinates corresponding to the geographical location of the vehicle, or   an indication of an altitude of the vehicle.   
     
     
         26 . The method of  claim 24 , wherein the second vehicle telemetry data comprises one or more of:
 at least one of a video or an image captured by a sensor of the selected vehicle,   an indication of a velocity of the selected vehicle,   an indication of an orientation of the selected vehicle,   an indication of an operating condition of a computer system of the selected vehicle,   an indication of a condition of one or more batteries of the selected vehicle,   an indication of an energy consumption of the selected vehicle,   an indication of an operating condition of the selected vehicle,   information regarding an environment of the selected vehicle, or   information regarding a route of the selected vehicle.   
     
     
         27 . The method of  claim 26 , wherein the information regarding the environment of the selected vehicle comprises one or more of:
 an indication of one or more objects in proximity to the selected vehicle,   an indication of a weather in the environment, or   at least one of an indication of one or more parking spaces in the environment, or   an indication of a landmark in the environment.   
     
     
         28 . The method of  claim 24 , wherein at least one of the first vehicle telemetry data or the second vehicle telemetry data is textual data. 
     
     
         29 . The method of  claim 24 , wherein at least one of the first vehicle telemetry data) or the second vehicle telemetry data comprises one or more data items in a JavaScript Object Notation (JSON) format. 
     
     
         30 . The method of  claim 24 , further comprising:
 receiving, at a computer device from a first vehicle of the plurality vehicles, an indication of an abnormal operation of the first vehicle;   responsive to receiving the indication of the abnormal operation of the first vehicle, presenting, by the display device, an alert regarding the abnormal operation of the first vehicle on the user interface.   
     
     
         31 . The method of  claim 30 , wherein the indication of the abnormal operation of the first vehicle is at least one of an indication of an interruption of a network connection between the first vehicle and the computer device or an indication that a path of the first vehicle is obstructed. 
     
     
         32 . The method of  claim 24 , wherein the user interface comprises a second display element indicating at least one of the vehicles of the plurality of vehicles in a list. 
     
     
         33 . The method of  claim 24 , wherein the visual representation of at least the portion of the first vehicle telemetry data or the portion of the second vehicle telemetry data regarding the selected vehicle comprises presenting a video captured by a sensor of the selected vehicle and one or more graphical elements overlaid on the video, the one or more graphical elements indicating one or more detected objects in the video. 
     
     
         34 . The method of  claim 24 , wherein the user interface comprises, for at least one autonomous vehicle of the plurality of vehicles, an indication of an assigned task of the autonomous vehicle. 
     
     
         35 . The method of  claim 24 , further comprising:
 receiving, at the computer device, a second user input comprising one or more search conditions with respect to the plurality of vehicles;   determining, at the computer device, one or more vehicles of the plurality of vehicles meeting the one or more search conditions; and   presenting, by the display device, a visual representation of the one or more vehicles meeting the one or more search conditions on the user interface.   
     
     
         36 . The method of  claim 35 , wherein the one or more search conditions comprise an indication of a service facility associated with one or more autonomous vehicles, and
 wherein determining the one or more vehicles meeting the one or more search conditions comprises identifying one or more vehicles in proximity to the service facility.   
     
     
         37 . A computer system comprising:
 one or more processors;   memory; and   one or more programs stored in memory, the one or more programs including instructions which, when executed by the one or more processors, cause the computer system to perform operations comprising:
 receiving first vehicle telemetry data from each vehicle of a plurality of vehicles, wherein the vehicle telemetry data comprises, for each vehicle of the plurality of vehicles, an indication of a geographical location of the vehicle, and wherein at least one vehicle of the plurality of vehicles is an autonomous vehicle; 
 presenting, by a display device associated with the computer system, a user interface generated based on the first vehicle telemetry data, wherein the user interface comprises a graphical map and one or more first display elements, each first display element indicating at least a respective geographical location of a respective vehicle of the plurality of vehicles; 
 receiving a first user input selecting a particular vehicle of the plurality of vehicles; 
 responsive to receiving the first user input:
 retrieving second vehicle telemetry data from the selected vehicle, and 
 presenting, by the display device, a visual representation of at least a portion of the first vehicle telemetry data or a portion of the second vehicle telemetry data regarding the selected vehicle on the user interface, wherein the visual representation includes:
 raw data obtained from one or more sensors of the selected vehicle without substantially any post-processing, and 
 processed data obtained from the one or more sensors of the selected vehicle and subsequently manipulated. 
 
 
   
     
     
         38 . A non-transitory computer-readable storage medium comprising one or more programs for execution by one or more processors of a computer device, the one or more programs including instructions which, when executed by the one or more processors, cause the computer device to perform operations comprising:
 receiving first vehicle telemetry data from each vehicle of a plurality of vehicles, wherein the vehicle telemetry data comprises, for each vehicle of the plurality of vehicles, an indication of a geographical location of the vehicle, and wherein at least one vehicle of the plurality of vehicles is an autonomous vehicle;   presenting, by a display device associated with the computer device, a user interface generated based on the first vehicle telemetry data, wherein the user interface comprises a graphical map and one or more first display elements, each first display element indicating at least a respective geographical location of a respective vehicle of the plurality of vehicles;   receiving a first user input selecting a particular vehicle of the plurality of vehicles;   responsive to receiving the first user input:
 retrieving second vehicle telemetry data from the selected vehicle, and 
 presenting, by the display device, a visual representation of at least a portion of the first vehicle telemetry data or a portion of the second vehicle telemetry data regarding the selected vehicle on the user interface, wherein the visual representation includes:
 raw data obtained from one or more sensors of the selected vehicle without substantially any post-processing, and 
 processed data obtained from the one or more sensors of the selected vehicle and subsequently manipulated.