Autonomous Map Traversal with Waypoint Matching

Abstract

A robot includes a drive system configured to maneuver the robot about an environment and data processing hardware in communication with memory hardware and the drive system. The memory hardware stores instructions that when executed on the data processing hardware cause the data processing hardware to perform operations. The operations include receiving image data of the robot maneuvering in the environment and executing at least one waypoint heuristic. The at least one waypoint heuristic is configured to trigger a waypoint placement on a waypoint map. In response to the at least one waypoint heuristic triggering the waypoint placement, the operations include recording a waypoint on the waypoint map where the waypoint is associated with at least one waypoint edge and includes sensor data obtained by the robot. The at least one waypoint edge includes a pose transform expressing how to move between two waypoints.

What is claimed is: 
     
         1 . A legged robot comprising:
 a body;   a plurality of legs coupled to the body, the plurality of legs comprising a right front leg, a right hind leg, a left front leg, and a left hind leg;   a vision system disposed on the body, the vision system comprising:
 a first image sensor and a second image sensor located between the right front leg and the right hind leg and horizontally mounted along a length of a right side of the body, wherein the first image sensor and the second image sensor are different types of image sensors; 
 a third image sensor and a fourth image sensor located between the left front leg and the left hind leg and horizontally mounted along a length of a left side of the body, wherein the third image sensor and the fourth image sensor are different types of image sensors; and 
 one or more front image sensors located between the right front leg and the left front leg on a front end portion of the body, 
 wherein the vision system is configured to obtain image data captured from the first image sensor, the second image sensor, the third image sensor, the fourth image sensor, and the one or more front image sensors; and 
   a computing system configured to process the image data to generate a map for controlling at least one of the right front leg, the right hind leg, the left front leg, or the left hind leg to maneuver the legged robot about an environment.   
     
     
         2 . The legged robot of  claim 1 , wherein at least one of the first image sensor, the second image sensor, the third image sensor, the fourth image sensor, or the one or more front image sensors comprises one or more of a stereo camera, a lidar sensor, or a ladar sensor. 
     
     
         3 . The legged robot of  claim 1 , wherein at least one of the first image sensor, the second image sensor, the third image sensor, the fourth image sensor, or the one or more front image sensors comprises a three-dimensional volumetric image sensor. 
     
     
         4 . The legged robot of  claim 1  further comprising an arm coupled to the body. 
     
     
         5 . The legged robot of  claim 1  further comprising an arm coupled to the body, wherein the vision system further comprises a movable image sensor disposed on the arm. 
     
     
         6 . The legged robot of  claim 1 , wherein at least one of the first image sensor, the second image sensor, the third image sensor, the fourth image sensor, or the one or more front image sensors is pivotable. 
     
     
         7 . The legged robot of  claim 1 , wherein the vision system further comprises a fifth image sensor located between the right front leg and the right hind leg or between the left front leg and the left hind leg. 
     
     
         8 . The legged robot of  claim 1 , wherein each of the first image sensor, the second image sensor, the third image sensor, and the fourth image sensor has a housing with a longitudinal axis extending along a length of the body on a respective side of the body. 
     
     
         9 . The legged robot of  claim 1 , wherein the one or more front image sensors comprises a column of image sensors. 
     
     
         10 . The legged robot of  claim 1 , wherein the one or more front image sensors comprises three image sensors located between the right front leg and the left front leg. 
     
     
         11 . The legged robot of  claim 1 , wherein the one or more front image sensors extend along a front end portion of the body. 
     
     
         12 . A computer-implemented method comprising:
 receiving, by data processing hardware, image data from a vision system disposed on a body of a legged robot, the legged robot comprising a right front leg, a right hind leg, a left front leg, and a left hind leg each coupled to the body, wherein receiving the image data from the vision system comprises:
 receiving a first portion of the image data from a first image sensor and a second image sensor located between the right front leg and the right hind leg and horizontally mounted along a length of a right side of the body, wherein the first image sensor and the second image sensor are different types of image sensors, 
 receiving a second portion of the image data from a third image sensor and a fourth image sensor located between the left front leg and the left hind leg and horizontally mounted along a length of a left side of the body, wherein the third image sensor and the fourth image sensor are different types of image sensors, and 
 receiving a third portion of the image data from one or more front image sensors located between the right front leg and the left front leg on a front end portion of the body; 
   processing, by the data processing hardware, the image data to generate a map for controlling at least one of the right front leg, the right hind leg, the left front leg, or the left hind leg to maneuver the legged robot about an environment; and   communicating, by the data processing hardware, the map to a drive system of the legged robot to maneuver the legged robot about the environment using the map.   
     
     
         13 . The method of  claim 12 , wherein receiving the image data further comprises receiving a fourth portion of the image data from a fifth image sensor located between the right front leg and the right hind leg or between the left front leg and the left hind leg. 
     
     
         14 . The method of  claim 12 , wherein each of the first image sensor, the second image sensor, the third image sensor, and the fourth image sensor has a housing with a longitudinal axis extending along a length of the body on a respective side of the body. 
     
     
         15 . The method of  claim 12 , wherein the one or more front image sensors extend along a front end portion of the body. 
     
     
         16 . The method of  claim 12 , wherein at least one of the first image sensor, the second image sensor, the third image sensor, the fourth image sensor, or the one or more front image sensors comprises one or more of a stereo camera, a lidar sensor, or a ladar sensor. 
     
     
         17 . A system comprising:
 data processing hardware; and   memory in communication with the data processing hardware, the memory storing instructions that when executed by the data processing hardware cause the data processing hardware to:
 receive image data from a vision system disposed on a body of a legged robot, the legged robot comprising a right front leg, a right hind leg, a left front leg, and a left hind leg each coupled to the body, wherein to receive the image data from the vision system, execution of the instructions by the data processing hardware causes the data processing hardware to:
 receive a first portion of the image data from a first image sensor and a second image sensor located between the right front leg and the right hind leg and horizontally mounted along a length of a right side of the body, wherein the first image sensor and the second image sensor are different types of image sensors, 
 receive a second portion of the image data from a third image sensor and a fourth image sensor located between the left front leg and the left hind leg and horizontally mounted along a length of a left side of the body, wherein the third image sensor and the fourth image sensor are different types of image sensors, and 
 receive a third portion of the image data from one or more front image sensors located between the right front leg and the left front leg on a front end portion of the body, 
 
 process the image data to generate a map for controlling at least one of the right front leg, the right hind leg, the left front leg, or the left hind leg to maneuver the legged robot about an environment; and 
 communicate the map to a drive system of the legged robot to maneuver the legged robot about the environment using the map. 
   
     
     
         18 . The system of  claim 17 , wherein at least one of the first image sensor, the second image sensor, the third image sensor, the fourth image sensor, or the one or more front image sensors comprises one or more of a stereo camera, a lidar sensor, or a ladar sensor. 
     
     
         19 . The system of  claim 17 , wherein the legged robot further comprises an arm coupled to the body, wherein the vision system comprises a fifth image sensor disposed on the arm. 
     
     
         20 . The system of  claim 17 , wherein the one or more front image sensors comprises a column of image sensors.