Method and system for operating a self-propelled vehicle according to scene images

Abstract

The present disclosure relates to a robotic system including one or more self-propelled motorized vehicles (SPMV) whose motion is controlled in accordance with electronic image data acquired by one or more observing camera(s) configured to image a scene including the SPMV. In some embodiments, the SPMV includes one or more on-board lights, and the SPMV is operated according to analyzing images acquired by the observing camera before and after an illumination transition of one or more of the point lights. Some embodiments relate techniques to computer gaming and/or to stereoscopic image processing techniques.

1 . A gaming system for providing a gaming service to a user, the gaming system comprising: a. electronic control circuitry; b. a user-directly-controlled self-propelled motorized vehicle (SPMV) operative to move responsively to wirelessly-received user-generated direct commands provided by mechanical motion and/or brainwaves of the user; c. an array of one or more cameras configured to generate an electronic image a scene including the user-directly-controlled SPMV; and d. a computer-directly-controlled SPMV operative to move responsively to computer-generated direct commands that generated by the electronic control circuitry in accordance with: i) one or more game objectives; and ii) a position or orientation, within the electronic image of the scene, of the user-directly-controlled SPMV. 
     
     
         2 . The system of  claim 1 , wherein the electronic control circuitry generates commands to control translational or rotational movement of computer-directly-controlled SPMV in accordance with at least one of: i) a distance between computer-directly-controlled SPMV and/or user-directly-controlled SPMV and a foreign object as determined in accordance with a Euclidian scene reconstruction of the electronic scene image; ii) historical and/or present and/or predicted future contents of a Euclidian world description data structure as determined in accordance with a Euclidian scene reconstruction of the electronic scene image; iii) historical and/or present and/or predicted contents of a game world description data structure. 
     
     
         3 . The system of  claim 1 , wherein the electronic control circuitry includes game strategy circuitry for enforcing one or more of the one or more of the game objectives. 
     
     
         4 . The system of  claim 1 , wherein the electronic control circuitry is operative to: i) detect the user-generated direct commands according to mechanical motion of a user control device or according to a detected gesture of the or a portion thereof; ii) wirelessly transmit the detected commands to the user-directly-controlled SPMV. 
     
     
         5 . The system of  claim 1 , wherein the user control device is selected from the group consisting of a joystick, a mouse, a keyboard, and an accelerometer. 
     
     
         6 . The system of  claim 1 , wherein the electronic control circuitry is operative generate the computer-generated direct commands for controlling the computer-directly-controlled SPMV in accordance with game rules of and/or strategy directives for a game selected from the group consisting of: a) a shooting game; b) a ball game; and c) a hand-to-hand combat game. 
     
     
         7 . The system of  claim 1 , wherein the gaming objective is selected from the group consisting of: a) an objective to score or a goal with a ball or puck; b) an objective to block a goal from being scored with a ball or puck; c) an objective to score or prevent a touchdown or field goal; d) an objective to score a hit against combat game vehicle with a projectile or a beam of light; e) an objective to reduce a probability of a hit being scored against a combat game vehicle with a projectile or a beam of light; and f) an objective to move or grab a game prop with computer SPMV is the game prop is grabbed by user SPMV. 
     
     
         8 . A method of operating an electronic device mounted with onboard lights (EDMOL) such as, but not limited to, a self-propelled motorized vehicle (SPMV), such device including one or more onboard lights operative to sequentially effect a plurality of illumination transitions, each transition modifying brightness and/or color of one or more of the onboard lights, the method comprising: a) electronically controlling the onboard light(s) of the EDMOL to induce an illumination transition that modifies brightness and/or color of one or more of the onboard lights; b) comparing first and second electronic images acquired by the camera, the first image being a pre-transition electronic image describing the EDMOL before the illumination transition and the second electronic image being a post transition electronic image describing the SPMV after the illumination transition; and c) in accordance with results of the comparing and in accordance with the camera calibration data defining the map between the pixel-image locations and the real-world locations, providing real-world Euclidean location information to electronic circuitry operating the EDMOL. 
     
     
         9 . The method of  claim 8  where the method further comprises computing, according to a first illumination transition set of one or more illumination transitions as described by the image time series, the camera calibration data itself including extrinsic camera calibration data for the observing camera, the camera calibration data relating pixel-image locations to Euclidian locations. 
     
     
         10 . The method of  claim 9  where the EDMOL controlled is a self-propelled motorized vehicle (SPMV) including one or more onboard lights operative to sequentially effect a plurality of illumination transitions, each transition modifying brightness and/or color of one or more of the onboard lights, the method further a) calculating: i) SPMV motor calibration data relating SPMV motor energy inputs to Euclidian displacements describing movement of the SPMV or a portion thereof; ii) servo motor calibration data for a servo on which the observing camera is mounted, the servo calibration data relating servo motor energy inputs to perceived Euclidian displacements of the SPMV as perceived by the servo-moved camera moved the servo; b) determining, according to: i) a second illumination transition set of one or more illumination transitions as described by the image time series; and ii) camera and/or SPMV motor and/or servo motor calibration data, a Euclidian location of the SPMV; and c) controlling rotational and/or translational movement of the SPMV or a portion thereof according to the determined Euclidian location. 
     
     
         11 . The method of  claim 10  wherein the method further comprises: a) sending one or more commands to the SPMV or to a servo on which the camera is mounted to induce translational or rotational movement of the SPMV or a portion thereof relative to the observing camera b) obtaining a time series of images of a scene including the SPMV, each image being generated by the observing camera and associated with a different location in appearance space (R, .PHI., I) for the SPMV that is provided according to the commands of step (a); c) computing, according to differences in the appearance of the SPMV in at least some of the images of the time series, calibration data including at least one of: i) camera calibration data including extrinsic camera calibration data for the observing camera, the camera calibration data relating pixel-image locations to Euclidian locations; ii) SPMV motor calibration data relating SPMV motor energy inputs to Euclidian displacements describing movement of the SPMV or a portion thereof; iii) servo motor calibration data for a servo on which the observing camera is mounted, the servo calibration data relating servo motor energy inputs to perceived Euclidian displacements of the SPMV as perceived by the servo-moved camera moved the servo; d) receiving from the client device or the client application a Euclidian command(s) for the SPMV; e) translating the Euclidian movement command(s) to one or more motor command(s) according to the calibration data; f) sending the motor commands of step (e) to the SPMV; g) performing a Euclidian scene reconstruction of one or more images of the time series according to the calibration data; and h) providing a description of the Euclidian scene reconstruction to the client device or client application. It is now disclosed for the first time a method of providing access for a client device or client application to a self-propelled motorized vehicle (SPMV) including one or more onboard lights operative to sequentially effect a plurality of illumination transitions, each transition modifying brightness and/or color of one or more of the onboard lights, the method comprising: a) obtaining a time series of images of a scene including the SPMV, each image being generated by an observing camera observing the scene; b) computing, according to a illumination transition set of one or more illumination transitions as described by the image time series, calibration data including at least one of: i) camera calibration data including extrinsic camera calibration data for the observing camera, the camera calibration data relating pixel-image locations to Euclidian locations; ii) SPMV motor calibration data relating SPMV motor energy inputs to Euclidian displacements describing movement of the SPMV or a portion thereof; iii) servo motor calibration data for a servo on which the observing camera is mounted, the servo calibration data relating servo motor energy inputs to perceived Euclidian displacements of the SPMV as perceived by the servo-moved camera moved the servo; c) receiving from the client device or the client application a Euclidian command(s) for the SPMV; d) translating the Euclidian movement command(s) to one or more motor command(s) according to the calibration data; e) sending the motor commands of step (d) to the SPMV; f) performing a Euclidian scene reconstruction of one or more images of the time series according to the calibration data; and g) providing a description of the Euclidian scene reconstruction to the client device or client application. 
     
     
         12 . The method of  claim 8  wherein, i) the camera is mounted on a servo assembly and which subjects the camera to the mechanical rotation according to a delivered power parameter describing delivered power which is delivered to one or more motors of the servo assembly; ii) the candidate rotation angles are selected according to a relationship between the power parameter and an estimated rotation provided by the servo assembly. 
     
     
         13 . The method of  claim 8 , wherein the method further comprises: f) controlling translational and/or rotational motion of an SPMV in a field of view of the camera in accordance with the computed post-rotation camera external calibration data. 
     
     
         14 . The method of  claim 8  wherein a) the onboard light(s) represent a plurality of lights and b) the relative real-world locations of the lights is known and c) the information regarding the relative real-world locations of the lights is utilized by the method to provide real-world Euclidean location information to electronic circuitry operating the EDMOL. 
     
     
         15 . The method of  claim 9  wherein a) the onboard light(s) represent a plurality of lights and b) the relative real-world locations of the lights is known and c) the information regarding the relative real-world locations of the lights is utilized by the method to provide real-world Euclidean location information to electronic circuitry operating the EDMOL. 
     
     
         16 . The method of  claim 10  wherein a) the onboard light(s) represent a plurality of lights and b) the relative real-world locations of the lights is known and c) the information regarding the relative real-world locations of the lights is utilized by the method to provide real-world Euclidean location information to electronic circuitry operating the SPMV. 
     
     
         17 . A method of operating a self-propelled motorized vehicle (SPMV)  120  including one or more electronically controlled onboard mechanical shutters  124  that effect mechanical shutter transition(s) that modifies color appearance of a location on SPMV housing, the SPMV located within a scene observed by an observing electronic camera  110 , the method comprising: a) obtaining first and second electronic images acquired by the camera, the first image being a pre-transition electronic image IMG.sub.PRE describing the SPMV  120  before the mechanical shutter transition and the second electronic image being a post-transition electronic image IMG.sub.POST describing the SPMV  120  after the mechanical shutter transition; and b) comparing the first and second electronic images to determine for each onboard mechanical shutter assembly of one or more of the onboard mechanical shutter assemblies, a respective pixel location within the first and/or second image c) determining, from the pixel location(s) and camera calibration data for the camera, a respective Euclidian location for each onboard mechanical shutter of the one or more onboard mechanical shutter(s); and d) in accordance with the determined Euclidian location(s) of the on-board mechanical shutter(s), electronically controlling rotational and/or translational movement of the SPMV or a portion thereof.