Systems and methods for machine learning-informed automated recording of time activities with an automated electronic time recording system or service

Abstract

A system and method for a machine learning-based automated electronic time recording for personnel includes identifying, via a scene capturing device, a representation of a time recording space; identifying a body having a time recording pose within the time recording space based on an assessment of the representation of the time recording space; extracting a plurality of distinct features from the representation of the time recording space based on identifying the body having the time recording pose; executing automated user-recognition based on the extracting of the plurality of distinct features; executing automated time recording recognition based on the extracting of the plurality of distinct features; and executing automated electronic time recording, via a time recording application based on the automated user-recognition and the automated time recording recognition.

I claim: 
     
         1 . A method for machine learning-based automated electronic time recording, the method comprising:
 (1) capturing, via a camera, a plurality of image frames of a time recording event;   (2) detecting within the plurality of image frames, by one or more computers executing a body detection engine, a human body having a time recording pose,   (3) wherein in response to detecting the human body, extracting a plurality of distinct features from the plurality of image frames including:
 (3-A) extracting, from the plurality of image frames, a first distinct portion of the human body comprising at least a cropped image of a head segment or a facial segment of the human body; and 
 (3-B) extracting, from the plurality of image frames, a second distinct portion of the human body comprising at least a cropped image of a hand segment of the human body that is associated with a detected time recording gesture; 
   (4) executing, by the one or more computers executing a facial recognition model, an automated employee-identification process that receives the cropped image of the head segment or facial segment of the human body as input, wherein executing the automated employee-identification process includes:
 (4-A) transforming, by the one or more computers executing the facial recognition model, the facial features of the human body to a facial feature vector comprising distinct numerical vector representation of the facial features of the human body and matching the facial feature vector to a given facial feature vector stored in association with a recognized employee record; and 
 (4-B) computing, by the one or more computers, an employee-identification inference comprising an employee identifier value for the human body based on the matching the facial feature vector to the given facial feature vector; 
   (5) executing, by the one or more computers executing a time recording machine learning model, an automated time recording-recognition process that receives the cropped image of the hand segment as input and generates a hand pose estimation vector to classify a time recording action performed by the human body, wherein executing the automated time recording-recognition process includes:
 (5-A) computing whether the hand pose estimation vector matches one of a plurality of stored reference hand pose vectors associated with distinct time recording actions; generating, by a time recording machine learning model, a time recording action inference for the human body based on a model input comprising extracted features of the second distinct portion of the human body, wherein:
 (5-A-1) generating the time recording action inference for the human body includes predicting a hand gesture classification inference for the human body, 
 (5-A-2) the time recording action inference generated for the human body indicates that the hand segment associated with the human body corresponds to a first hand gesture, and 
 
 (5-B) identifying, by the one or more computers, a time recording code of a plurality of time recording codes corresponding to a reference hand pose vector that matches the hand pose estimation vector; 
   (6) automatically executing, by the one or more computers executing a time recording computer application, an automated electronic time recording event for the human body based on time recording inputs of (i) the employee identifier value and (ii) the time recording code associated with the human body, wherein:
 executing, by the one or more computers, the automated electronic time recording event automatically creating an entry in an electronic user account associated with the employee identification value of a time recording action determined by the time recording code; and 
   in response to executing the automated electronic time recording event, transmitting, by the one or more computers, a confirmation notification to a computing device associated with the employee identifier value that indicates a successful registration of the time recording event.   
     
     
         2 . The method of  claim 1 , wherein the employee-identification machine learning model comprises a neural network (NN) and/or a transformer model. 
     
     
         3 . The method of  claim 1 , wherein:
 the automated electronic time recording event executed for the human body corresponds to a clock-in time recording event, and   the method further comprising:
 after executing the clock-in time recording event for the human body, visually indicating, via a computer display, to the human body that the clock-in time recording event was successfully registered to the time recording application. 
   
     
     
         4 . The method of  claim 1 , wherein:
 the one or more computers identified that the time recording code associated with the body corresponds to a clock-in time recording code based on the one or more computers identifying that the first hand gesture is digitally mapped to the clock-in time recording code.   
     
     
         5 . The method of  claim 1 , wherein the time recording machine learning model comprises a neural network (NN) and/or a transformer model that is trained based on one or more second training corpora comprising a plurality of images of distinct hand-based time recording gestures. 
     
     
         6 . The method of  claim 1 , wherein:
 generating the employee-identification inference for the human body includes predicting a facial classification for the human body,   the facial classification for the human body includes a distinct facial image value and an associated degree of confidence, and   identifying the employee identifier value for the human body includes:
 performing a search, using the employee-identification inference generated for the human body, at a data structure comprising a plurality of distinct employee identifier values digitally associated with a plurality of employee-identification data; 
 returning a distinct employee identifier value for the human body based on the search; and 
 digitally linking the human body to a corresponding distinct employee identifier value. 
   
     
     
         7 . The method of  claim 1 , further comprising:
 implementing a time recording ensemble of machine learning models comprising:
 (1) the employee-identification machine learning model, and 
 (2) the time recording machine learning model, 
   wherein the time recording ensemble of machine learning models output the employee-identification inference and the time recording action inference for the human body.   
     
     
         8 . The method of  claim 1 , wherein:
 the employee identifier value identified for the human body is associated with a distinct employee account that is electronically accessible to the time recording application,   before executing the automated electronic recording event for the human body, the distinct employee account associated with the identified employee identifier value is in a first time recording state, and   the time recording application automatically and electronically changes the distinct employee account to a second time recording state that is distinct from the first time recording state.   
     
     
         9 . The method of  claim 1 , wherein:
 the hand gesture classification inference generated for the human body includes a distinct gesture image value and an associated degree of confidence, and   identifying the time recording code for the human body includes:
 performing an automated search by the time recording computer application, using the time recording action inference generated for the human body, at a data structure comprising a plurality of distinct gesture image values associated with the plurality of time recording codes; 
 returning the time recording code for the human body based on the search; and 
 digitally linking the human body to a corresponding time recording code. 
   
     
     
         10 . The method of  claim 1 , wherein:
 the time recording computer application dynamically creates the entry in the electronic user account as the human body is moving through the time recording event.   
     
     
         11 . The method of  claim 1 , wherein:
 the employee identifier value for the human body is identifiable when the human body has been enrolled into an automated electronic time recording system, and   the employee identifier value for a subject distinct body is not identifiable when the subject distinct body has not previously enrolled into the automated electronic time recording system.   
     
     
         12 . The method of  claim 1 , wherein the automated employee-recognition and the automated time recording-recognition are simultaneously executed by the one or more computers executing an automated electronic time recording system. 
     
     
         13 . The method of  claim 1 , wherein:
 a second distinct body comprises a first hand and a second hand,   the second distinct body is determined to be in the time recording pose when a respective hand of the second distinct body is detected above a head of the second distinct body, and   extracting the second distinct portion for the second distinct body includes:
 in accordance with a determination that the first hand of the second distinct body is detected above the head of the second distinct body, extracting the first hand of the second distinct body without extracting the second hand of the second distinct body; and 
 in accordance with a determination that the second hand of the second distinct body is detected above the head of the second distinct body, extracting the second hand of the second distinct body without extracting the first hand of the second distinct body. 
   
     
     
         14 . The method of  claim 1 , wherein the time recording event includes a plurality of time recording zones, the method further comprising:
 identifying a location of the human body within the time recording event based on the assessment of the plurality of image frames of the time recording event;   determining a time recording zone of the plurality of time recording zones associated with the human body based on the location of the human body,   wherein executing the automated electronic time recording event for the human body is further based on an input of the time recording zone associated with the human body.   
     
     
         15 . The method of  claim 1 , further comprising:
 after extracting the first distinct portion or the second distinct portion of the human body:
 identifying that the first distinct portion or the second distinct portion of the human body does not satisfy an image resolution threshold; and 
 in response to identifying that the first distinct portion or the second distinct portion of the human body does not satisfy the image resolution threshold:
 forgoing executing the automated employee-recognition for the human body; 
 forgoing executing the automated time recording-recognition for the human body; 
 forgoing executing the automated electronic time recording event for the human body; and 
 providing at least a portion of the plurality of image frames of the time recording event to a predetermined entity to assess a time recording intent of the human body. 
 
   
     
     
         16 . The method of  claim 1 , wherein the body detection engine processes image frames in parallel to accelerate detection latency, and the time recording machine learning model applies an adaptive thresholding mechanism to minimize false positives in hand gesture classification. 
     
     
         17 . The method of  claim 1 , wherein the automated time recording-recognition process enables multiple users to execute distinct time recording actions simultaneously without requiring physical interaction with a timekeeping device. 
     
     
         18 . A computer-program product comprising a non-transitory machine-readable storage medium storing computer instructions that, when executed by one or more processors, perform operations comprising:
 (1) capturing, via a camera, a plurality of image frames of a time recording event;   (2) detecting within the plurality of image frames, by one or more computers executing a body detection engine, a human body having a time recording pose;   (3) wherein in response to detecting the human body, extracting a plurality of distinct features from the plurality of image frames including:
 (3-A) extracting, from the plurality of image frames, a first distinct portion of the human body comprising at least a cropped image of a head segment or a facial segment of the human body; and 
 (3-B) extracting, from the plurality of image frames, a second distinct portion of the human body comprising at least a cropped image of a hand segment of the human body that is associated with a detected time recording gesture; 
   (4) executing, by the one or more computers executing a facial recognition model, an automated employee-identification process that receives the cropped image of the head segment or facial segment of the human body as input, wherein executing the automated employee-identification process includes:
 (4-A) transforming, by the one or more computers executing the facial recognition model, the facial features of the human body to a facial feature vector comprising distinct numerical vector representation of the facial features of the human body and matching the facial feature vector to a given facial feature vector stored in association with a recognized employee record; and 
 (4-B) computing, by the one or more computers, an employee-identification inference comprising an employee identifier value for the human body based on the matching the facial feature vector to the given facial feature vector; 
   (5) executing, by the one or more computers executing a time recording machine learning model, an automated time recording-recognition process that receives the cropped image of the hand segment as input and generates a hand pose estimation vector to classify a time recording action performed by the human body, wherein executing the automated time recording-recognition process includes:
 (5-A) computing whether the hand pose estimation vector matches one of a plurality of stored reference hand pose vectors associated with distinct time recording actions; generating, by a time recording machine learning model, a time recording action inference for the human body based on a model input comprising extracted features of the second distinct portion of the human body, wherein:
 (5-A-1) generating the time recording action inference for the human body includes predicting a hand gesture classification inference for the body, 
 (5-A-2) the time recording action inference generated for the human body indicates that the hand segment associated with the human body corresponds to a first hand gesture, and 
 
 (5-B) identifying, by the one or more computers, a time recording code of a plurality of time recording codes corresponding to a reference hand pose vector that matches the hand pose estimation vector; 
   (6) automatically executing, by the one or more computers executing a time recording computer application, an automated electronic time recording event for the human body based on time recording inputs of (i) the employee identifier value and (ii) the time recording code associated with the human body, wherein:
 executing, by the one or more computers, the automated electronic time recording event automatically creating an entry in an electronic user account associated with the employee value of a time recording action determined by the time recording code; and 
   in response to executing the automated electronic time recording event, transmitting, by the one or more computers, a confirmation notification to a computing device associated with the employee identifier value that indicates a successful registration of the time recording event.