System and method of making unstructured data available to structured data analysis tools

Abstract

A system and method of making unstructured data available to structured data analysis tools. The system includes middleware software that can be used in combination with structured data tools to perform analysis on both structured and unstructured data. Data can be read from a wide variety of unstructured sources. The data may then be transformed with commercial data transformation products that may, for example, extract individual pieces of data and determine relationships between the extracted data. The transformed data and relationships may then be passed through an extraction/transform/load (ETL) layer and placed in a structured schema. The structured schema may then be made available to commercial or proprietary structured data analysis tools.

1 . A method of making unstructured data available to structured data tools comprising:
 accessing a source of unstructured data;   extracting the unstructured data;   writing the extracted unstructured data to a capture schema;   sending the unstructured data to a transformation tool;   transforming the unstructured data with the transformation tool;   writing the transformed parsed unstructured data in a structured analysis schema;   providing data connectors that allow structured data tools to access the structured analysis schema.   
     
     
         2 . The method of  claim 1 , wherein extracting includes parsing the unstructured data and associating the data source with the parsed unstructured data. 
     
     
         3 . The method of  claim 1 , wherein the transformation tool extracts individual pieces of data and performs text and/or data processing activities. 
     
     
         4 . The method of  claim 3 , wherein the text and data processing activities include: determining the topic of a section of text, extracting a section of text from a whole document, matching names, or matching addresses. 
     
     
         5 . The method of  claim 1 , wherein the structured data tools include business intelligence, statistical analysis, data visualization and mapping, or data mining. 
     
     
         6 . The method of  claim 1 , wherein the source of unstructured data includes email, word processing documents, spreadsheets, presentation materials, PDF files, web pages, news/media reports, case files, transcriptions, file servers, web servers, enterprise content, enterprise search tool repositories, intranets, knowledge management systems, or document management systems. 
     
     
         7 . The method of  claim 1 , wherein the transformation tool includes: (i) entity, concept and relationship tagging and extraction tools, (ii) categorization and topic extraction tools, (iii) data matching tools or, (iv) custom transformations. 
     
     
         8 . A system for making unstructured data available to structured data tools comprising:
 code to access a source of unstructured data;   code to extract the unstructured data;   code to write the extracted unstructured data to a capture schema;   code to send the unstructured data to a transformation tool;   code to transform the unstructured data with the transformation tool;   code to write the transformed parsed unstructured data in a structured analysis schema;   code to provide data connectors that allow structured data tools to access the structured analysis schema.   
     
     
         9 . The system of  claim 8 , wherein the code to extract includes code to parse the unstructured data and associate the data source with the parsed unstructured data. 
     
     
         10 . The system of  claim 8 , wherein the transformation tool extracts individual pieces of data and performs text and/or data processing activities. 
     
     
         11 . The system of  claim 10 , wherein the text and data processing activities include: determining the topic of a section of text, extracting a section of text from a whole document, matching names, or matching addresses. 
     
     
         12 . The system of  claim 8 , wherein the structured data tools include business intelligence, statistical analysis, data visualization and mapping, or data mining. 
     
     
         13 . The system of  claim 8 , wherein the source of unstructured data includes email, word processing documents, spreadsheets, presentation materials, PDF files, web pages, news/media reports, case files, transcriptions, file servers, web servers, enterprise content, enterprise search tool repositories, intranets, knowledge management systems, or document management systems. 
     
     
         14 . The system of  claim 8 , wherein the transformation tool includes: (i) entity, concept and relationship tagging and extraction tools, (ii) categorization and topic extraction tools, (iii) data matching tools or, (iv) custom transformations. 
     
     
         15 . An extraction service for extracting unstructured data from a plurality of unstructured data sources and a plurality of formats comprising:
 a plurality of APIs to interface with the plurality of unstructured data sources; and   a single internal API that interfaces with a plurality of software components that allow structured data tools to operate on unstructured data.   
     
     
         16 . The extraction service of  claim 15 , wherein the plurality of unstructured data sources includes email, word processing documents, spreadsheets, presentation materials, PDF files, web pages, news/media reports, case files, transcriptions, file servers, web servers, enterprise content, enterprise search tool repositories, intranets, knowledge management systems, and document management systems. 
     
     
         17 .- 27 . (canceled)