High-risk passage automation in a digital transaction management platform

Abstract

A document execution engine receives a training set of data including training documents that each include one or more passages associated with a passage type and a level of risk. The document execution engine trains a machine learned model based on the training set. The trained machine learned model, when applied to subsequently identified passages within documents in the document execution environment, can identify a passage with above threshold levels of risk (e.g., a high-risk passage) based on a passage type of the passage. The trained machine learned model can then provide for display the high-risk passage and a related passage of the same passage type from a second document within the document execution environment to the user via a document passage comparison interface. Differences between the passages can be highlighted, enabling a user to quickly compare and contrast the passages.

What is claimed is: 
     
         1 . A method comprising:
 accessing a training set of information including training documents within a document execution environment, each training document including one or more passages, each passage associated with a passage type and a level of risk;   training a machine learned model based on the accessed training set of information, the machine learned model configured to determine a level of risk associated with a document passage based on least in part on the passage type of the document passage;   after the machine learned model is applied to a target document and determines a level of risk for a target passage within the target document, receiving feedback from a user indicating whether the determined level of risk for the target passage is accurate;   modifying the training set of information based on the received feedback; and   re-training the machine learned model based on the modified training set of information.   
     
     
         2 . The method of  claim 1 , wherein each training document is associated with a set of document characteristics comprising one or more of a document type, a region, a language, and an industry, and wherein the machine learned model is configured to determine a level of risk associated with a document passage based additionally on the set of document characteristics associated with a document in which the document passage appears. 
     
     
         3 . The method of  claim 1 , wherein the passage type for a passage comprises one or more of: a type of a legal clause, a type of business clause, a type of finance clause, and a type of content within the passage. 
     
     
         4 . The method of  claim 1 , wherein the feedback is received via an interface displaying the target document on a device of the user. 
     
     
         5 . The method of  claim 1 , wherein modifying the training set of information comprises including the target passage within the training set associated with a level of risk specified by the user. 
     
     
         6 . The method of  claim 1 , wherein one or more recommendations to mitigate the determined level of risk are presented to the user in conjunction with displaying the target passage and the determined level of risk. 
     
     
         7 . The method of  claim 6 , wherein the one or more recommendations include one or more of: a recommendation to provide the target document to additional users for review, a recommendation to have additional users digitally sign the target document, and a recommendation for one or more security measures to be implemented in association with the target document. 
     
     
         8 . A non-transitory computer-readable storage medium storing instructions that, when executed by a hardware processor, cause the hardware processor to perform steps comprising:
 accessing a training set of information including training documents within a document execution environment, each training document including one or more passages, each passage associated with a passage type and a level of risk;   training a machine learned model based on the accessed training set of information, the machine learned model configured to determine a level of risk associated with a document passage based on least in part on the passage type of the document passage;   after the machine learned model is applied to a target document and determines a level of risk for a target passage within the target document, receiving feedback from a user indicating whether the determined level of risk for the target passage is accurate;   modifying the training set of information based on the received feedback; and   re-training the machine learned model based on the modified training set of information.   
     
     
         9 . The non-transitory computer-readable storage medium of  claim 8 , wherein each training document is associated with a set of document characteristics comprising one or more of a document type, a region, a language, and an industry, and wherein the machine learned model is configured to determine a level of risk associated with a document passage based additionally on the set of document characteristics associated with a document in which the document passage appears. 
     
     
         10 . The non-transitory computer-readable storage medium of  claim 8 , wherein the passage type for a passage comprises one or more of: a type of a legal clause, a type of business clause, a type of finance clause, and a type of content within the passage. 
     
     
         11 . The non-transitory computer-readable storage medium of  claim 8 , wherein the feedback is received via an interface displaying the target document on a device of the user. 
     
     
         12 . The non-transitory computer-readable storage medium of  claim 8 , wherein modifying the training set of information comprises including the target passage within the training set associated with a level of risk specified by the user. 
     
     
         13 . The non-transitory computer-readable storage medium of  claim 8 , wherein one or more recommendations to mitigate the determined level of risk are presented to the user in conjunction with displaying the target passage and the determined level of risk. 
     
     
         14 . The non-transitory computer-readable storage medium of  claim 13 , wherein the one or more recommendations include one or more of: a recommendation to provide the target document to additional users for review, a recommendation to have additional users digitally sign the target document, and a recommendation for one or more security measures to be implemented in association with the target document. 
     
     
         15 . A system comprising:
 a hardware processor; and   a non-transitory computer-readable storage medium storing executable instructions that, when executed by the hardware processor, perform steps comprising:
 accessing a training set of information including training documents within a document execution environment, each training document including one or more passages, each passage associated with a passage type and a level of risk; 
 training a machine learned model based on the accessed training set of information, the machine learned model configured to determine a level of risk associated with a document passage based on least in part on the passage type of the document passage; 
 after the machine learned model is applied to a target document and determines a level of risk for a target passage within the target document, receiving feedback from a user indicating whether the determined level of risk for the target passage is accurate; 
 modifying the training set of information based on the received feedback; and 
 re-training the machine learned model based on the modified training set of information. 
   
     
     
         16 . The system of  claim 15 , wherein each training document is associated with a set of document characteristics comprising one or more of a document type, a region, a language, and an industry, and wherein the machine learned model is configured to determine a level of risk associated with a document passage based additionally on the set of document characteristics associated with a document in which the document passage appears. 
     
     
         17 . The system of  claim 15 , wherein the passage type for a passage comprises one or more of: a type of a legal clause, a type of business clause, a type of finance clause, and a type of content within the passage. 
     
     
         18 . The system of  claim 15 , wherein the feedback is received via an interface displaying the target document on a device of the user. 
     
     
         19 . The system of  claim 15 , wherein modifying the training set of information comprises including the target passage within the training set associated with a level of risk specified by the user. 
     
     
         20 . The system of  claim 15 , wherein one or more recommendations to mitigate the determined level of risk are presented to the user in conjunction with displaying the target passage and the determined level of risk.