Visual data analysis method and device

Abstract

The present disclosure provides a visual data analysis method and device used for visual analysis for multiple types of data sources. By establishing connection relationships with various types of data sources, multiple types of data sources can be obtained in real time, and various types of data sources are combined and analyze in real time. The method includes: obtaining multiple types of data sources, and establishing a connection with each type of data source, wherein the type of data source is used to represent a source from which data is obtained; displaying, through a visual page, each piece of table information contained in each type of data source with which the connection is made.

1 . A visual data analysis method, comprising:
 obtaining multiple types of data sources, and establishing a connection with each type of data source, wherein the type of data source is used to represent a source from which data is obtained;   displaying, through a visual page, each piece of table information contained in each type of data source with which the connection is made;   in response to an association operation of a user on multiple tables that are displayed, generating a target dataset according to an association relationship between the multiple tables indicated by the association operation; and   displaying the target dataset on the visual page by means of a chart.   
     
     
         2 . The method according to  claim 1 , wherein obtaining multiple types of data sources through any one or more of following manners:
 receiving parameter information input by the user, and obtaining a data source of a corresponding type according to the parameter information;   obtaining a data source of a corresponding type through a file transfer protocol; or   using an executed structured query language (SQL) statement as an obtained data source of a corresponding type.   
     
     
         3 . The method according to  claim 2 , wherein obtaining a data source of a corresponding type according to the parameter information through any one or more of following manners:
 receiving a database parameter input by the user, and obtaining a data source of a database type according to the database parameter; or,   receiving an interface parameter input by the user, and obtaining a data source of an interface type according to the interface parameter; or,   obtaining text data uploaded by the user, and determining text data named by the user as a data source of a text type; or,   receiving a Redis parameter input by the user, and obtaining a data source of a Redis cache type according to the Redis parameter; or,   receiving a SQL statement input by the user, and determining the SQL statement input as a data source of a SQL statement type.   
     
     
         4 . The method according to  claim 2 , wherein the obtaining a data source of a corresponding type through a file transfer protocol, comprises:
 obtaining a file in a file transfer protocol (FTP) server by means of a secret file transfer protocol (SFTP), and determining the file obtained as a data source of a FTP type.   
     
     
         5 . The method according to  claim 2 , wherein the using an executed SQL statement as an obtained data source of a corresponding type, comprises:
 receiving a SQL statement executed by the user on a data source with which a connection is made, and determining the executed SQL statement as a data source of a SQL statement type.   
     
     
         6 . The method according to  claim 1 , wherein the establishing a connection with each type of data source, comprises:
 establishing a connection with each type of data source according to connection information of each type of data source.   
     
     
         7 . The method according to  claim 6 , wherein the establishing a connection with each type of data source according to connection information of each type of data source, comprises:
 writing the connection information of each type of data source into a configuration file of a distributed query engine; and   when starting the distributed query engine, establishing, according to the connection information of each type of data source in the configuration file, the connection with each type of data source.   
     
     
         8 . The method according to  claim 6 , wherein when the data source is a data source of a database type, the establishing a connection with each type of data source according to connection information of each type of data source, comprises:
 establishing a connection with the data source of the database type according to a database parameter, wherein the database parameter represents a parameter required to connect with a database.   
     
     
         9 . The method according to  claim 6 , wherein when the data source is a data source of an interface type, the establishing a connection with each type of data source according to connection information of each type of data source, comprises:
 running an interface according to an interface parameter to obtain Java script object notation (JSON) data, and parsing the JSON data to obtain a data source parameter; and   establishing a connection with the data source of the interface type according to the data source parameter parsed and the interface parameter; or   when the data source is a data source of a text type, the establishing a connection with each type of data source according to connection information of each type of data source, comprises:   determining a data source parameter according to a data source stored in a file storage server; and   establishing a connection with the data source of the interface type according to a server parameter of the file storage server and the data source parameter.   
     
     
         10 . (canceled) 
     
     
         11 . The method according to  claim 9 , wherein the data source parameter comprises at least one of a data source identifier, a type of data source, a library field, a table field, a column field, or a field type of a column field. 
     
     
         12 . The method according to  claim 6 , wherein when the data source is a data source of a SQL statement type, the establishing a connection with each type of data source according to connection information of each type of data source, comprises:
 performing a syntax verification on a SQL statement, and after determining that the syntax verification passes, parsing the SQL statement to obtain table information in the SQL statement; and   establishing a connection with the data source of the SQL statement type according to the SQL statement and the table information in the SQL statement.   
     
     
         13 . The method according to  claim 12 , wherein after parsing the SQL statement to obtain table information in the SQL statement, the method further comprises:
 storing the SQL statement and the table information in the SQL statement in a local database; and   generating a nested SQL statement using the stored SQL statement and a SQL statement input by the user, and determining the generated nested SQL statement as an obtained data source of the SQL statement type.   
     
     
         14 . The method according to  claim 1 , wherein the establishing a connection with each type of data source, comprises:
 building a shared data source application according to a connection pool of each data source contained in each type of data source; and   establishing a connection between each business system and each type of data source through the shared data source application, wherein the shared data source application provides a service for each business system to connect with each type of data source through an ability for integrating a connection with each type of data source.   
     
     
         15 . The method according to  claim 14 , wherein the establishing a connection between each business system and each type of data source through the shared data source application, comprises:
 establishing a connection between the shared data source application and each type of data source according to connection information of each data source described in a metadata; and   establishing a connection between each type of data source connected with the shared data source application and each business system through the shared data source application; or   receiving an access requirement of each business system through the shared data source application;   determining a connection pool of a target data source corresponding to each business system according to the access requirement of each business system and a number of connections in a connection pool of each data source; and   establishing a connection between each business system and a corresponding target data source through the connection pool of the target data source.   
     
     
         16 . (canceled) 
     
     
         17 . The method according to  claim 14 , wherein after the establishing a connection between each business system and each type of data source through the shared data source application, the method further comprises:
 receiving an operation instruction sent by the business system in a form of a metadata through the shared data source application; and   performing at least one operation of aggregation, filtering, or query on a data source corresponding to the operation instruction.   
     
     
         18 . The method according to  claim 1 , wherein in response to an association operation of a user on multiple tables that are displayed, the generating a target dataset according to an association relationship between the multiple tables indicated by the association operation, comprises:
 in response to a dragging instruction of the user for the multiple tables displayed, determining table information of each target table corresponding to the dragging instruction; and   receiving an association relationship between multiple target tables input by the user, and generating a target dataset according to the table information of each target table and the association relationship.   
     
     
         19 . The method according to  claim 18 , wherein the generating a target dataset according to the table information of each target table and the association relationship, comprises:
 determining first fields, that are the same, between the multiple target tables and second fields that are retained after the multiple target tables are associated according to the association relationship; and   generating a SQL statement according to the table information of each target table, the first fields and the second fields, and executing the SQL statement to obtain the target dataset.   
     
     
         20 . The method according to  claim 18 , wherein the generating a target dataset according to the table information of each target table and the association relationship, further comprises:
 receiving a filtering condition input by the user, wherein the filtering condition is used to filter data in multiple target tables; and   generating a target dataset according to the filtering condition, table information of the multiple target tables, and the association relationship between the multiple target tables.   
     
     
         21 . The method according to  claim 1 , wherein the displaying the target dataset on the visual page by means of a chart, comprises:
 determining a chart type specified by the user and a target data column in the target dataset;   using the target data column as chart data corresponding to the chart type, and using a chart component to draw a chart corresponding to the chart type; and   displaying the drawn chart on the visual page.   
     
     
         22 . (canceled) 
     
     
         23 . A visual data analysis device, comprising: a processor and a memory, wherein the memory is configured to store programs executable by the processor, and the processor is configured to read the programs in the memory and execute steps of the method according to  claim 1 . 
     
     
         24 . (canceled)