Methods and apparatus for controlling automated vehicles in an environment using virtual approved pathways

Abstract

Systems and methods for controlling or guiding one or more automated vehicles and/or people (VOP) in an environment using virtual approved pathways (VAPs). Methods include determining a set of parameters associated with automated vehicles and/or people operating within an environment, including periodically obtaining a first set of parameters from a plurality of data sources deployed within an environment, in real-time.

What is claimed: 
     
         1 . A method for controlling or guiding one or more automated vehicles or people (VOPs) in an environment using virtual approved pathways (VAPs), comprising:
 determining, with a processor, a set of parameters associated with a vehicle or person (VOP), wherein the set of parameters comprise at least one of: a VOP class, capabilities, characteristics, and requirements associated with the VOP, and wherein the VOP is dynamically configured or guided to navigate from a source point to a destination point based on one or more tasks requested;   continuously obtaining, with the processor, a first set of parameters from a plurality of data sources deployed within the environment, in real-time, wherein the first set of parameters comprise sensor data, one or more properties associated with a plurality of objects, one or more navigation objectives, a plurality of pre-defined rules associated with each object within the environment, a plurality of priority levels, and one or more user-defined requirements, wherein the plurality of objects comprises at least one of sensors, one or more VOPs, one or more load objects, one or more cart objects, and one or more obstacles, one or more computing devices, one or more user devices, one or more real-time location system (RTLS) tags;   determining, with the processor, one or more navigation plans for capable VOPs predicted to be available at a requested time, and dynamically configured to navigate from the source point to the destination point based on the obtained first set of parameters, wherein each of the one or more navigation plans corresponds to at least one of a plurality of classes of VOPs, a plurality of environmental conditions, the plurality of pre-defined rules and the plurality of priority levels, the one or more properties associated with each object within the environment, the one or more navigation objectives and the one or more user-defined requirements;   correlating, with the processor, each of the determined one or more navigation plans with the one or more tasks to be performed by the one or more VOPs, at specific times, the at least one of the obtained first set of parameters and the determined set of VOP parameters, using a data driven model;   determining, with the processor, an optimal navigation plan for the VOP based on the correlation, wherein the optimal navigation plan comprises at least one Virtual Approved Pathway (VAP) connecting the source point with the destination point via a plurality of path points, dynamic properties to be configured with the VOP, and one or more operational rules to be followed by the VOP while navigating in the at least one Virtual Approved Pathway (VAP) navigating, with the processor, the VOP within the environment based on the determined optimal navigation plan, wherein the one or more VOPs are guided by one or more waypoints acting as path indicators;   determining, with the processor, whether the optimal navigation plan is to be updated based on the plurality of environmental conditions, a current position of the destination point, a predicted position of the destination point, possible movement of the destination point, and the first set of parameters; and   updating, with the processor, at least a portion of the optimal navigation plan based on the plurality of environmental conditions and the first set of parameters obtained in real-time.   
     
     
         2 . The method of  claim 1 , further comprising:
 simulating, with the processor, the determined optimal navigation plan in a virtual environment for validating the determined optimal navigation plan, wherein the virtual environment emulates a physical environment; and   deploying, with the processor, the determined optimal navigation plan in the physical environment based on results of simulation.   
     
     
         3 . The method of  claim 1 , further comprising:
 controlling or guiding, with the processor, the VOP based on the updated optimal navigation plan, the plurality of environmental conditions and the first set of parameters obtained in real-time.   
     
     
         4 . The method of  claim 1 , wherein determining the optimal navigation plan for the VOP based on the correlation comprises:
 retrieving, with the processor, one or more Virtual Approved Pathways (VAPs) within the environment from at least one of the one or more VOPs and the plurality of objects, using real time location systems (RTLS) and one or more sensors;   determining, with the processor, one or more zones and stations existing within the environment based on the first set of parameters, wherein the one or more zones comprise one of a free roaming zone, a no-go zone, and an intersection zone, wherein the VOP is configured or guided to choose a desired trajectory when in free-roaming zone until the VOP exits the free roaming zone, and wherein the VOP is configured or guided to restrict navigating via the no-go zone, and wherein the VOP is configured or guided to modify its behavior, such as performing a temporarily halt and then traveling at reduced speed, while transversing the intersection zone;   determining, with the processor, one or more best possible trajectories for navigation of the VOP based on the obtained one or more VAPs and the one or more zones and stations, wherein each trajectory comprises the plurality of path points, and wherein each trajectory is guided by the one or more waypoints;   identifying, with the processor, an optimal trajectory among the determined one or more best possible trajectories for the VOP based on one of the plurality of classes of VOPs, the plurality of environmental conditions, the plurality of pre-defined rules and the plurality of priority levels, the one or more properties associated with each object within the environment, the one or more navigation objectives and the one or more user-defined requirements;   determining, with the processor, the one or more operational rules to be followed by the VOP based on the identified optimal trajectory;   determining, with the processor, the dynamic properties to be configured with the VOP based on the determined one or more operational rules;   defining, with the processor, the at least one Virtual Approved Pathway (VAP) connecting the source point with the destination point via the plurality of path points; and   defining, with the processor, at least one priority level of the plurality of priority levels for the VOP based on the defined VAP, the one or more operational rules, and the dynamic properties to be configured with the VOP.   
     
     
         5 . The method of  claim 1 , wherein defining the at least one Virtual Approved Pathway (VAP) connecting the source point with the destination point via the plurality of path points comprises:
 identifying, with the processor, the plurality of path points between the source point and the destination point using a Real-Time Location System (RTLS); and   generating, with the processor, a route from the source point to the destination point, wherein the route comprises the identified plurality of path points.   
     
     
         6 . The method of  claim 4 , wherein determining whether the optimal navigation plan is to be updated based on the plurality of environmental conditions, the current position of the destination point, the predicted position of the destination point, the possible movement of the destination point, and the first set of parameters comprises:
 continuously monitoring, with the processor, current location of the VOP relative to the identified optimal trajectory to determine whether the VOP deviates greater than a threshold distance value from the optimal trajectory of the at least one Virtual Approved Pathway (VAP);   determining, with the processor, the plurality of environmental conditions, the current position of the destination point, the predicted position of the destination point, and the possible movement of the destination point, wherein the plurality of environmental conditions are determined using one or more sensors present within the environment, wherein the one or more sensors comprise sensors associated with the one or more VOPs, and wherein the current position of the destination point is determined using at least one of the RTLS and peer VOPs currently deployed in the environment;   determining, with the processor, one or more possible collision events on the at least one Virtual Approved Pathway (VAP) based on the determined plurality of environmental conditions and the first set of parameters obtained in real-time; and   creating, with the processor, one of a temporary VAP in addition to a current VAP, an updated VAP, and a new VAP for the VOP, based on the determined one or more possible collision events, the plurality of environmental conditions, the current position of the destination point, the predicted position of the destination point, and the possible movement of the destination point, wherein the temporary VAP inherits specific properties and rules from the at least one VAP.   
     
     
         7 . The method of  claim 6 , further comprising:
 generating, with the processor, visual representations of the environment in real-time using at least one of RTLS tags and the one or more sensors deployed within the environment, wherein the visual representations comprise a plurality of virtual elements corresponding to each of the plurality of objects deployed in the environment and wherein the visual representations comprise of a map of the environment;   determining, with the processor, the properties and movements associated with each of the plurality of virtual elements depicted in the generated visual representations, wherein each virtual element of the plurality of virtual elements is configured to perform one of inherit and overrule one of properties and rules of other virtual elements based on the plurality of environmental conditions and based on an absolute location of the virtual element within the environment and/or based on a relative position of the virtual element relative to some or all of the other virtual elements;   determining, with the processor, a plurality of rules associated with each of the plurality of virtual elements depicted in the generated visual representations;   comparing, with the processor, the determined properties, the movements, and the plurality of rules associated with the plurality of virtual elements with corresponding pre-defined properties, pre-defined movements, and pre-defined rules stored in a database; and   determining, with the processor, whether the optimal navigation plan is to be updated based on the comparison.   
     
     
         8 . The method of  claim 7 , wherein generating the visual representations of the environment in real time comprises:
 creating, by the VOP, at least one Simultaneous Locating and Mapping (SLAM) map of the environment while navigating along the at least one VAP using one or more RTLS tags; and   transmitting, by the VOP, the created SLAM map to a control system or to the one or more VOPs within the environment.   
     
     
         9 . The method of  claim 7 , wherein generating the visual representations of the environment in real-time comprises:
 generating, by a processor of a user device, an augmented-reality (AR) based representation as a visual representation of a physical environment, wherein the AR-based representation comprises the plurality of virtual elements emulating the plurality of objects within the physical environment, the one or more VAPs, one or more destinations, trajectories, current and historical locations of the plurality of virtual elements, the properties and rules associated with the plurality of virtual elements; and   obtaining, with the processor, user interaction with the plurality of virtual elements via the generated AR-based representation, wherein the user interaction comprises at least one of:   visualizing, with the processor, location and movements of static elements and dynamic elements in real-time, at specific past and future points in time, wherein the static elements correspond to the one or more VAPs, the one or more zones, and one or more stations defined within the environment, respective rules and properties associated with each of the one or more VAPs, the one or more zones and stations, and wherein the dynamic elements correspond to the one or more destinations, the plurality of environmental conditions, tasks, destination points, targets, temporary VAPs, trajectories, and travel intent, and wherein the visualization comprises one of replaying the past locations of the plurality of virtual elements at previous points in time, replaying movements of the plurality of virtual elements during past periods in time, future locations of the plurality of virtual elements predicted to be at future periods in time;   accessing and modifying, with the processor, the properties and rules associated with the plurality of virtual elements; and   interacting and modifying, with the processor, the plurality of virtual elements on the AR-based representation.   
     
     
         10 . The method of  claim 9 , wherein generating the augmented-reality (AR) based representation as one of the visual representation of the physical environment comprises:
 projecting, with the processor, the AR-based representation onto an AR capable device associated with a user, wherein the AR-based representation comprises the plurality of virtual elements, the properties and the rules associated with the plurality of virtual elements superimposed in real-time, onto a graphical user interface screen of the AR capable device.   
     
     
         11 . The method of  claim 1 , further comprising:
 identifying, with the processor, potential conflicts between the Virtual Approved Pathways (VAPs) and capabilities of a specific class of VOP, based on the dynamic properties of that class of the VOP;   visually representing, with the processor, the identified conflicts as graphical representations on a graphical user interface; and   generating, with the processor, one or more solutions for rectification of the identified potential conflicts and directing tasks and activities to be performed by the VOP; and   receiving, with the processor, a user-approved solution from a user from among the generated one or more solutions.   
     
     
         12 . The method of  claim 1 , wherein updating at least the portion of the optimal navigation plan based on the plurality of environmental conditions and the first set of parameters obtained in real-time comprises one of:
 modifying, with the processor, at least one of a speed, a direction, and a rate of travel of the VOP; and   halting, with the processor, movement of the VOP in a current position until the determined one or more possible collision events are resolved.   
     
     
         13 . The method of  claim 1 , further comprising:
 identifying, with the processor, a current state of a VAP section based on the plurality of environmental conditions and the first set of parameters obtained in real-time, wherein the current state comprises temporarily unavailable; and   actively launching, with the processor, one or more available VOPs to current location of the VAP section for validating the identified current state, re-determine the plurality of environmental conditions and re-transmit the first set of parameters in real-time, wherein the current state of the VAP section can be determined to either remain temporarily unavailable, or to be available again to be included in the creation of navigation plans.   
     
     
         14 . The method of  claim 1 , further comprising:
 detecting, with the processor, patterns, behaviors, and trends associated with the obtained first set of parameters, the plurality of environmental conditions, and one or more possible collision events;   training, with the processor, a dataset based on the detected patterns, behaviors and trends associated with the obtained first set of parameters, the plurality of environmental conditions, and the one or more possible collision events; and   tuning, with the processor, the dynamic properties and the one or more operational rules of the VOP based on the trained dataset.   
     
     
         15 . The method of  claim 1 , further comprising:
 continuously monitoring, with the processor, the first set of parameters obtained in real-time;   continuously monitoring, with the processor, the Virtual Approved Pathways (VAPs), the dynamic properties of the VOP and the one or more operational rules;   continuously monitoring, with the processor, the plurality of environmental conditions within the environment;   continuously monitoring, with the processor, the current position of the destination point, the predicted position of the destination point, and the possible movement of the destination point;   continuously training, with the processor, a dataset based on the monitored first set of parameters, the at least one VAP, the dynamic properties of the VOP and the one or more operational rules, the plurality of environmental conditions, the current position of the destination point, the predicted position of the destination point, and the possible movement of the destination point; and   updating, with the processor, a VAP database with the trained dataset, wherein the VAP database is maintained to store the dynamic properties, routes of the VAPs and operational rules associated with the VAPs.   
     
     
         16 . The method of  claim 1 , wherein updating at least the portion of the optimal navigation plan based on the plurality of environmental conditions and the first set of parameters obtained in real-time comprises one of:
 dynamically tuning, with the processor, one or more VAPs based on the obtained first set of parameters and the plurality of environmental conditions, wherein the one or more VAPs are pre-recorded using one of RTLS Tags and user-driven inputs, wherein the tuning comprises one of dynamically correcting, dynamically smoothening, dynamically straightening, dynamically aligning, and dynamically connecting the one or more VAPs; and   displaying, with the processor, parts of the one or more VAPs as visual highlights when the parts of the one or more VAPs are identified to be unsuitable.   
     
     
         17 . The method of  claim 1 , wherein updating at least the portion of the optimal navigation plan based on the plurality of environmental conditions and the first set of parameters obtained in real-time comprises at least one of:
 modifying, with the processor, at least one of: a route of the at least one VAP and the properties associated with the at least one VAP in real-time based on the determined plurality of environmental conditions; and   modifying, with the processor, a priority level assigned to the at least one VAP based on the determined plurality of environmental conditions.   
     
     
         18 . The method of  claim 1 , wherein updating at least the portion of the optimal navigation plan based on the plurality of environmental conditions and the first set of parameters obtained the at real-time comprises:
 generating, with the processor, one or more suggestions to correct at least one of multiple current VAPs, the properties, and the rules of a current VAP.   
     
     
         19 . The method of  claim 1 , further comprising:
 determining, with the processor, compatibility of the VOP for navigation by mapping the dynamic properties and the one or more operational rules of the VOP with the properties and rules of the at least one VAP; and   navigating, with the processor, the VOP, based on the determined compatibility, wherein the determined compatibility comprises one of directionality restrictions, speed limitations, distance maintenance from other objects, and a lane assignment within the at least one VAP.   
     
     
         20 . The method of  claim 4 , wherein the sensor data is obtained by at least one of the one or more sensors carried by the one or more VOPs being directed by a control system, and the one or more sensors carried by other VOPs operating within the environment. 
     
     
         21 . The method of  claim 1 , wherein the one or more operational rules comprise virtual approved pathway (VAP) rules, travel restrictive rules for specific classes of VOPs, traffic rules, event-based rules, environmental condition-based rules, activity-based rules, zone based rules, station-based rules, time-based rules, and pathway based rules, and wherein the one or more operational rules control behaviors of the one or more VOPs while navigating along respective one or more VAPs, wherein the one or more operational rules correspond to rules which the VOP is required to comply with, based on the rules which are applicable to the VOP along an optimal trajectory navigated by the VOP while performing a task of the one or more tasks. 
     
     
         22 . The method of  claim 4 , further comprising:
 evaluating, with the processor, one or more possible combinations between available VOPs, available VAPs, and possible trajectories within the environment, to decide which VOP of the available VOPs is best positioned to perform a task;   matching, with the processor, existing properties and rules associated with the available VAPs to properties and rules associated with the available VOPs and properties and rules associated with one of the task to be performed, one or more loads involved, one or more carts involved, and the plurality of environmental conditions existing and predicted at a time when the task is to be performed; and   determining, with the processor, the optimal trajectory for a best suited VOP to perform the task based on the matching.   
     
     
         23 . The method of  claim 4 , wherein retrieving the one or more Virtual Approved Pathways (VAPs) within the environment from at least one of the one or more VOPs and the plurality of objects, comprises:
 recording, with the processor, one or more VAPs by navigating the one or more VOPs within the environment.   
     
     
         24 . The method of  claim 1 , further comprising:
 automatically tuning pre-recorded pathways captured by at least one of RTLS tags or hand-drawn on a graphical user interface, wherein the tuning comprises at least one of automatically straightening, automatically smoothening, automatically aligning, and automatically connecting at least one of the pre-recorded pathways and the hand-drawn pre-recorded pathways; and   storing the automatically tuned pre-recorded pathways as one or more VAPs in a VAP database.