Vision-based approach and landing system

Abstract

A vision-based navigation method that does not depend on GPS signal for landing AAM aircraft. The Vision-Based Approach and Landing System (VALS) uses images captured from a camera for Advanced Air Mobility (AAM) approach and landing in GPS-denied environments, which offers a potential Alternative Position, Navigation, and Timing (APNT) solution. VALS utilizes a computer vision algorithm called Coplanar Pose from Orthography and Scaling with Iterations (COPOSIT) to estimate the position and orientation of the camera based on coplanar features. VALS also includes an extended Kalman filter that uses IMU measurements in a prediction step and the COPOSIT estimation results in a correction element. Combining IMU with vision creates a sensor fusion navigation solution for GPS-denied environments.

What is claimed is: 
     
         1 . A method of landing an aircraft without usage of a GPS signal, comprising:
 receiving an image from a camera attached to an aircraft;   determining a correspondence between estimated landmark locations based on the image and predicted landmark locations to produce a plurality of world coordinates of best matches for at least four coplanar points in the image;   applying a COPOSIT method to the plurality of world coordinates of best matches for the at least four coplanar points to produce an estimated aircraft position and orientation;   receiving, by a Kalman filter, the estimated aircraft position and orientation, and further receiving input from an IMU, the IMU input representing acceleration and angular body rates; and   generating, by the Kalman filter, a state estimation of the aircraft, the state estimation including a corrected estimated position, a corrected estimated velocity, and a corrected estimated orientation of the aircraft, the Kalman filter operating without using input of the GPS signal, the state estimation configured to be outputted to a flight control system of the aircraft for landing the aircraft without using input of the GPS signal.   
     
     
         2 . The method of  claim 1 , further comprising:
 detecting features in the image, wherein the features correspond to fiducials; and   estimating the landmark locations based on the detected features; and   predicting the predicted landmark locations.   
     
     
         3 . The method of  claim 2 , wherein the detecting features is performed by Hough circle detection. 
     
     
         4 . The method of  claim 2 , wherein the detecting features is performed by Harris corner detection. 
     
     
         5 . The method of  claim 1 , wherein method is performed by a simulator to simulate aircraft flight using computer-generated terrain and fiducials. 
     
     
         6 . The method of  claim 1  wherein the aircraft position and orientation are an output measurement matrix of the COPOSIT module, and wherein the output measurement matrix is set to zero when no COPOSIT data is available and the zeroed output measurement matrix is input to the Kalman filter. 
     
     
         7 . The method of  claim 1 , wherein the corrected estimated position, the corrected estimated velocity, and the corrected estimated orientation are repeatedly determined over time based on new received images from the camera and are repeatedly inputted into a flight control system to facilitate landing the aircraft. 
     
     
         8 . The method of  claim 1 , wherein the method is performed in near real time. 
     
     
         9 . The method of  claim 1 , further comprising:
 obtaining real world telemetry data and videos from a drone flight test, and   simulating the method using the obtained real world telemetry data and videos as input to the simulation.   
     
     
         10 . A system for landing an aircraft without usage of a GPS signal, the system comprising:
 a camera attached to an aircraft;   a feature corresponder to determine a correspondence between estimated landmark locations based on the image and predicted landmark locations to produce a plurality of world coordinates of best matches for at least four coplanar points in the image;   a COPOSIT module for applying a COPOSIT method to the plurality of world coordinates of best matches for the at least four coplanar points to produce estimated aircraft position and orientation; and   a Kalman filter that receives the estimated aircraft position and orientation and further receives input from an IMU, the input representing acceleration and angular body rates, the Kalman filter generating a corrected estimated position, a corrected estimated velocity, and a corrected estimated orientation of the aircraft, the Kalman filter operating without using input of the GPS signal, the state estimation configured to be outputted to a flight control system of the aircraft for landing the aircraft without using input of the GPS signal.   
     
     
         11 . The system of  claim 10 ,
 wherein the determiner further comprises:
 detecting features in the image, wherein the features correspond to fiducials, and 
 estimating the landmark locations based on the detected features; and 
   a predictor to predict the predicted landmark locations.   
     
     
         12 . The system of  claim 11 , wherein the detecting features is performed by Hough circle detection. 
     
     
         13 . The system of  claim 12 , wherein the detecting features is performed by Harris corner detection. 
     
     
         14 . The system of  claim 12 , wherein method is performed by a simulator to simulate aircraft flight using computer-generated terrain and fiducials. 
     
     
         15 . The system of  claim 10 , wherein the aircraft position and orientation are an output measurement matrix of the COPOSIT module, and wherein the output measurement matrix is set to zero when no COPOSIT data is available and the zeroed output measurement matrix is input to the Kalman filter. 
     
     
         16 . The system of  claim 10 , wherein the corrected estimated position, the corrected estimated velocity, and the corrected estimated orientation are repeatedly determined over time based on new received image from the camera and are repeatedly inputted into a flight control system to facilitate landing the aircraft. 
     
     
         17 . The system of  claim 10 , wherein the system operates in near real time. 
     
     
         18 . A computer program product capable for landing an aircraft without usage of a GPS signal, comprising instructions stored in a memory to:
 receive an image from a camera attached to an aircraft;   determine a correspondence between estimated landmark locations based on the image and between predicted landmark locations to produce a plurality of world coordinates of best matches for at least four coplanar points in the image;   apply a COPOSIT method to the plurality of world coordinates of best matches for the at least four coplanar points to produce estimated aircraft position and orientation; and   receive, by a Kalman filter, the estimated aircraft position and orientation and further receiving input from an IMU representing acceleration and angular body rates, the Kalman filter generating a state estimation of the aircraft, the state estimation including a corrected estimated position, a corrected estimated velocity, and a corrected estimated orientation of the aircraft, the Kalman filter operating without using input of a GPS signal, the state estimation configured to be outputted to a flight control system of the aircraft for landing the aircraft without using input of the GPS signal.   
     
     
         19 . The computer program product of  claim 18 , further comprising instructions to:
 detect features in the image, wherein the features correspond to fiducials, wherein the detecting features is performed by Hough circle detection.   
     
     
         20 . The computer program product of  claim 18 , further comprising instructions to:
 detect features in the image, wherein the features correspond to fiducials, wherein the detecting features is performed by Harris corner detection.