Traffic light recognition method and apparatus

Abstract

This application discloses a traffic light recognition method and apparatus in the field of artificial intelligence that relate to an automated driving technology. The recognition apparatus determines location information of n first traffic lights in a first image based on current positioning information of a vehicle, a calibration parameter of an image sensor, and prestored map data; and detects the first image by using an image detection method, to obtain location information of m second traffic lights in the first image and indication information of the m second traffic lights. This effectively reduces a traffic light recognition error, and improves recognition accuracy.

What is claimed is: 
     
       1. A traffic light recognition method, comprising:
 obtaining current positioning information of a vehicle and a first image, wherein the first image is currently obtained by using an image sensor; 
 determining location information of n first traffic lights in the first image based on the current positioning information of the vehicle, a calibration parameter of the image sensor, and prestored map data, wherein n is an integer greater than or equal to 1; 
 detecting the first image by using an image detection method, to obtain location information of m second traffic lights in the first image and indication information of the m second traffic lights, wherein the indication information comprises at least one of a color and semantic indication information, and m is an integer greater than or equal to 1; 
 determining, based on the location information of the n first traffic lights in the first image and the location information of the m second traffic lights in the first image and based on a preset error constraint, a first traffic light and a second traffic light that match each other; and 
 obtaining indication information of the second traffic light that matches the first traffic light. 
 
     
     
       2. The method according to  claim 1 , wherein the determining, based on the location information of the n first traffic lights in the first image and the location information of the m second traffic lights in the first image and based on the preset error constraint, the first traffic light and the second traffic light that match each other comprises:
 step A: selecting i first traffic lights from the n first traffic lights, and generating a first subimage group, wherein the first subimage group comprises k subimages, each subimage in the first subimage group comprises a first location or a first location relationship, the first location is used to indicate locations, in the first image, of the i first traffic lights comprised in the subimage, the first location relationship is used to indicate a relationship between the locations, in the first image, of the i first traffic lights comprised in the subimage, and wherein k=C n   i , and 1≤i≤min(n, m); 
 step B: selecting i second traffic lights from the m second traffic lights, and generating a second subimage group, wherein the second subimage group comprises p subimages, each subimage in the second subimage group comprises a second location or a second location relationship, the second location is used to indicate locations, in the first image, of the i second traffic lights comprised in the subimage, the second location relationship is used to indicate a relationship between the locations, in the first image, of the i second traffic lights comprised in the subimage, and wherein p=C m   i ; and 
 step C: determining whether a pair of subimages matching each other exists in the first subimage group and the second subimage group, wherein an error between a first location and a second location respectively comprised in the pair of subimages matching each other is less than the preset error constraint, or an error between a first location relationship and a second location relationship respectively comprised in the pair of subimages matching each other is less than the preset error constraint; 
 wherein if a pair of subimages matching each other exists, determining, based on the pair of subimages matching each other, the first traffic light and the second traffic light that match each other. 
 
     
     
       3. The method according to  claim 2 , wherein the determining, based on the pair of subimages matching each other, the first traffic light and the second traffic light that match each other comprises:
 selecting, among at least two pairs of subimages matching each other that exist, a pair of subimages matching each other from the at least two pairs of subimages matching each other, wherein the selected pair of subimages matching each other respectively comprises a first subimage in the first subimage group and a second subimage in the second subimage group; and 
 determining that a j th  first traffic light arranged in a preset order in the first subimage matches a j th  second traffic light arranged in the preset order in the second subimage; 
 wherein j∈[1, i]. 
 
     
     
       4. The method according to  claim 2 , wherein the method further comprises:
 if no pair of subimages matching each other exists, re-determining a value of i, and performing step A, step B, and step C based on the re-determined value of i. 
 
     
     
       5. The method according to  claim 2 , wherein the error constraint is a geometric error and a distance error, and the determining whether a pair of subimages matching each other exists in the first subimage group and the second subimage group comprises:
 calculating a geometric error and a distance error between a third subimage and a fourth subimage, wherein the third subimage is any subimage in the first subimage group, and the fourth subimage is any subimage in the second subimage group; and 
 in response to the geometric error meeting a preset condition and the distance error being less than a preset distance, determining that the third subimage and the fourth subimage match each other; 
 wherein the geometric error comprises at least one of a scale error or an angle error; 
 wherein the scale error is used to represent a ratio of a first length to a second length, the first length is a length between a j th  traffic light and a (j+1) th  traffic light that are arranged in the preset order in the third subimage, and the second length is a length between a j th  traffic light and a (j+1) th  traffic light that are arranged in the preset order in the fourth subimage; 
 wherein the angle error is used to represent a difference between a first angle and a second angle, the first angle is an angle between a first connection line and a preset coordinate axis in a coordinate system of the first image, the second angle is an angle between a second connection line and the preset coordinate axis, the first connection line is a connection line between the j th  traffic light and the (j+1) th  traffic light that are arranged in the preset order in the third subimage, and the second connection line is a connection line between the j th  traffic light and the (j+1) th  traffic light that are arranged in the preset order in the fourth subimage, wherein j∈[1, i]; 
 wherein the distance error is used to indicate a distance between the j th  traffic light arranged in the preset order in the third subimage and the j th  traffic light arranged in the preset order in the fourth subimage; and 
 wherein the geometric error meeting the preset condition comprises: when the geometric error comprises the scale error, the scale error is less than or equal to a first preset threshold; and when the geometric error comprises the angle error, the angle error is less than or equal to a second preset threshold. 
 
     
     
       6. The method according to  claim 5 ,
 wherein the geometric error further comprises a semantic error, and the semantic error is used to indicate confidence of semantic information of the j th  traffic light arranged in the preset order in the fourth subimage relative to semantic information of the j th  traffic light arranged in the preset order in the third subimage; and 
 wherein the geometric error meeting the preset condition further comprises the semantic error being less than or equal to a third preset threshold. 
 
     
     
       7. The method according to  claim 5 , wherein the selecting, among the at least two pairs of subimages matching each other exist, a pair of subimages matching each other from the at least two pairs of subimages matching each other comprises:
 selecting, from the at least two pairs of subimages matching each other, a pair of subimages having a minimum sum of a geometric error and a distance error. 
 
     
     
       8. The method according to  claim 2 , wherein only one pair of subimages matching each other exists, and the pair of subimages matching each other respectively comprises a first subimage in the first subimage group and a second subimage in the second subimage group, and wherein the determining, based on the pair of subimages matching each other, the first traffic light and the second traffic light that match each other comprises:
 determining that a j th  first traffic light arranged in a preset order in the first subimage matches a j th  second traffic light arranged in the preset order in the second subimage, 
 wherein j∈[1, i]. 
 
     
     
       9. The method according to  claim 1 , wherein the detecting the first image by using the image detection method, to obtain the location information of the m second traffic lights in the first image and the indication information of the m second traffic lights comprises:
 detecting the first image by using a preset neural network model, to obtain the location information of the m second traffic lights in the first image and the indication information of the m second traffic lights. 
 
     
     
       10. An apparatus, comprising:
 one or more processors; and 
 a memory that is coupled to the one or more processors, wherein the memory is configured to store computer program code, the computer program code comprises instructions, and when the one or more processors execute the instructions, the apparatus performs a traffic light recognition method comprising:
 obtaining current positioning information of a vehicle and a first image, wherein the first image is currently obtained by using an image sensor; 
 determining location information of n first traffic lights in the first image based on the current positioning information of the vehicle, a calibration parameter of the image sensor, and prestored map data, wherein n is an integer greater than or equal to 1; 
 detecting the first image by using an image detection method, to obtain location information of m second traffic lights in the first image and indication information of the m second traffic lights, wherein the indication information comprises at least one of a color and semantic indication information, and m is an integer greater than or equal to 1; 
 determining, based on the location information of the n first traffic lights in the first image and the location information of the m second traffic lights in the first image and based on a preset error constraint, a first traffic light and a second traffic light that match each other; and 
 obtaining indication information of the second traffic light that matches the first traffic light. 
 
 
     
     
       11. The apparatus according to  claim 10 , wherein the determining, based on the location information of the n first traffic lights in the first image and the location information of the m second traffic lights in the first image and based on the preset error constraint, the first traffic light and the second traffic light that match each other comprises:
 step A: selecting i first traffic lights from the n first traffic lights, and generating a first subimage group, wherein the first subimage group comprises k subimages, each subimage in the first subimage group comprises a first location or a first location relationship, the first location is used to indicate locations, in the first image, of the i first traffic lights comprised in the subimage, the first location relationship is used to indicate a relationship between the locations, in the first image, of the i first traffic lights comprised in the subimage, and wherein k=C n   i , and 1≤i≤min(n, m); 
 step B: selecting i second traffic lights from the m second traffic lights, and generating a second subimage group, wherein the second subimage group comprises p subimages, each subimage in the second subimage group comprises a second location or a second location relationship, the second location is used to indicate locations, in the first image, of the i second traffic lights comprised in the subimage, the second location relationship is used to indicate a relationship between the locations, in the first image, of the i second traffic lights comprised in the subimage, and wherein p=C m   i ; and 
 step C: determining whether a pair of subimages matching each other exists in the first subimage group and the second subimage group, wherein an error between a first location and a second location respectively comprised in the pair of subimages matching each other is less than the preset error constraint, or an error between a first location relationship and a second location relationship respectively comprised in the pair of subimages matching each other is less than the preset error constraint; 
 wherein if a pair of subimages matching each other exists, determining, based on the pair of subimages matching each other, the first traffic light and the second traffic light that match each other. 
 
     
     
       12. The apparatus according to  claim 11 , wherein the determining, based on the pair of subimages matching each other, the first traffic light and the second traffic light that match each other comprises:
 selecting, among at least two pairs of subimages matching each other that exist, a pair of subimages matching each other from the at least two pairs of subimages matching each other, wherein the selected pair of subimages matching each other respectively comprises a first subimage in the first subimage group and a second subimage in the second subimage group; and 
 determining that a j th  first traffic light arranged in a preset order in the first subimage matches a j th  second traffic light arranged in the preset order in the second subimage; 
 wherein j∈[1, i]. 
 
     
     
       13. The apparatus according to  claim 11 , wherein the method further comprises:
 if no pair of subimages matching each other exists, re-determining a value of i, and performing step A, step B, and step C based on the re-determined value of i. 
 
     
     
       14. The apparatus according to  claim 11 , wherein the error constraint is a geometric error and a distance error, and the determining whether a pair of subimages matching each other exists in the first subimage group and the second subimage group comprises:
 calculating a geometric error and a distance error between a third subimage and a fourth subimage, wherein the third subimage is any subimage in the first subimage group, and the fourth subimage is any subimage in the second subimage group; and 
 in response to the geometric error meeting a preset condition and the distance error being less than a preset distance, determining that the third subimage and the fourth subimage match each other; 
 wherein the geometric error comprises at least one of a scale error or an angle error; 
 wherein the scale error is used to represent a ratio of a first length to a second length, the first length is a length between a j th  traffic light and a (j+1) th  traffic light that are arranged in the preset order in the third subimage, and the second length is a length between a j th  traffic light and a (j+1) th  traffic light that are arranged in the preset order in the fourth subimage; 
 wherein the angle error is used to represent a difference between a first angle and a second angle, the first angle is an angle between a first connection line and a preset coordinate axis in a coordinate system of the first image, the second angle is an angle between a second connection line and the preset coordinate axis, the first connection line is a connection line between the j th  traffic light and the (j+1) th  traffic light that are arranged in the preset order in the third subimage, and the second connection line is a connection line between the j th  traffic light and the (j+1) th  traffic light that are arranged in the preset order in the fourth subimage, wherein j∈[1, i]; 
 wherein the distance error is used to indicate a distance between the j th  traffic light arranged in the preset order in the third subimage and the j th  traffic light arranged in the preset order in the fourth subimage; and 
 wherein the geometric error meeting the preset condition comprises: when the geometric error comprises the scale error, the scale error is less than or equal to a first preset threshold; and when the geometric error comprises the angle error, the angle error is less than or equal to a second preset threshold. 
 
     
     
       15. The apparatus according to  claim 14 ,
 wherein the geometric error further comprises a semantic error, and the semantic error is used to indicate confidence of semantic information of the j th  traffic light arranged in the preset order in the fourth subimage relative to semantic information of the j th  traffic light arranged in the preset order in the third subimage; and 
 wherein the geometric error meeting the preset condition further comprises the semantic error being less than or equal to a third preset threshold. 
 
     
     
       16. The apparatus according to  claim 14 , wherein the selecting, among the at least two pairs of subimages matching each other exist, a pair of subimages matching each other from the at least two pairs of subimages matching each other comprises:
 selecting, from the at least two pairs of subimages matching each other, a pair of subimages having a minimum sum of a geometric error and a distance error. 
 
     
     
       17. The apparatus according to  claim 11 , wherein only one pair of subimages matching each other exists, and the pair of subimages matching each other respectively comprises a first subimage in the first subimage group and a second subimage in the second subimage group, and wherein the determining, based on the pair of subimages matching each other, the first traffic light and the second traffic light that match each other comprises:
 determining that a j th  first traffic light arranged in a preset order in the first subimage matches a j th  second traffic light arranged in the preset order in the second subimage, 
 wherein j∈[1, i]. 
 
     
     
       18. The apparatus according to  claim 10 , wherein the detecting the first image by using the image detection method, to obtain the location information of the m second traffic lights in the first image and the indication information of the m second traffic lights comprises:
 detecting the first image by using a preset neural network model, to obtain the location information of the m second traffic lights in the first image and the indication information of the m second traffic lights. 
 
     
     
       19. Anon-transitory computer-readable storage medium comprising instructions that, run by a processor on a recognition apparatus, the recognition apparatus is enabled to perform a traffic light recognition method comprising:
 obtaining current positioning information of a vehicle and a first image, wherein the first image is currently obtained by using an image sensor; 
 determining location information of n first traffic lights in the first image based on the current positioning information of the vehicle, a calibration parameter of the image sensor, and prestored map data, wherein n is an integer greater than or equal to 1; 
 detecting the first image by using an image detection method, to obtain location information of m second traffic lights in the first image and indication information of the m second traffic lights, wherein the indication information comprises at least one of a color and semantic indication information, and m is an integer greater than or equal to 1; 
 determining, based on the location information of the n first traffic lights in the first image and the location information of the m second traffic lights in the first image and based on a preset error constraint, a first traffic light and a second traffic light that match each other; and 
 obtaining indication information of the second traffic light that matches the first traffic light.