Determining context using weighted parsing scoring

Abstract

According to one embodiment, a method, computer system, and computer program product for natural language processing is provided. The present invention may include detecting natural language entities, and running parsing algorithms on the natural language entities to determine the relationship between said natural language entities. The present invention may further comprise assigning, by the parsing algorithms, initial scores to detected natural language entities based on the relationship between said natural language entities; choosing a final score for plurality of natural language entities; and comparing the final score against a threshold to determine whether the natural language entities are within the same context.

1 . A processor-implemented method for determining a plurality of relationships between a plurality of natural language entities the method comprising:
 detecting a plurality of natural language entities within a plurality of natural language text, wherein each of the plurality of natural language entities comprises a semantic categorization of one or more tokens based on one or more requirements of a natural language context determination process;   running, by a processor, a plurality of parsing algorithms simultaneously on the detected plurality of natural language entities to determine a relationship between at least two natural language entities within the detected plurality of natural language entities;   assigning, by at least one of the plurality of parsing algorithms, a plurality of initial scores to a pair of the detected plurality of natural language entities based on the relationship, wherein at least one of the plurality of initial scores is a generic score, and wherein at least one of the plurality of initial scores is a fragment score, wherein the fragment score is an adjusted variant of the generic score incorporating one or more punctuation weights or one or more conjunction weights;   choosing a final score, wherein the final score is the highest score for the detected plurality of natural language entities from within the assigned one or more initial scores;   comparing the chosen final score against a threshold, wherein the threshold is determined by machine learning; and   where the chosen final score exceeds the threshold, transmitting the chosen final score to a natural language processing pipeline.