Information processing system and information processing method

Abstract

An information processing system selects a valid trained model candidate from trained models and presents the selected trained model candidate to a model user. A model user calculation device calculates a model application target data feature that is a feature of model application target data which is an application target of the trained model candidate. A model management server calculates a similarity between a model pre-evaluation data feature that is a feature of model pre-evaluation data for pre-evaluating performance of the trained model candidate and the model application target data feature. Then, the model management server calculates a score for predicting validity of the trained model candidate based on the similarity and model pre-evaluation accuracy that is model accuracy of a trained model pre-evaluated based on the model pre-evaluation data. Then, the model management server presents the score together with the trained model candidate to the model user.

What is claimed is: 
     
         1 . An information processing system that selects a valid trained model candidate from a plurality of trained models and presents the selected trained model candidate to a model user, the information processing system comprising:
 a model management server; and   a model user calculation device configured to be operated by the model user, wherein   the model management server stores, in a storage unit, the plurality of trained models, a model pre-evaluation data feature that is a feature of model pre-evaluation data for pre-evaluating performance of the plurality of trained models, and model pre-evaluation accuracy that is model accuracy of the plurality of trained models pre-evaluated based on the model pre-evaluation data,   the model user calculation device
 stores, in a storage unit, model application target data that is an application target of the trained model candidate selected from the plurality of trained models, 
 calculates a model application target data feature that is a feature of the model application target data, and 
 transmits the calculated model application target data feature to the model management server, and 
   the model management server
 calculates a similarity between the model pre-evaluation data feature and the model application target data feature received from the model user calculation device, 
 calculates a score for predicting validity of the trained model candidate based on the similarity and the model pre-evaluation accuracy, and 
 presents the score together with the trained model candidate to the model user via the model user calculation device. 
   
     
     
         2 . The information processing system according to  claim 1 , wherein
 the model user calculation device
 receives, as model user evaluation accuracy, a model evaluation result when the trained model candidate input by the model user is applied to the model application target data, and 
 transmits the received model user evaluation accuracy to the model management server, and 
   the model management server
 generates a prediction model for calculating prediction accuracy for predicting the validity of the trained model candidate based on the model user evaluation accuracy received from the model user calculation device, the similarity, and the model pre-evaluation accuracy, 
 calculates the prediction accuracy based on the model user evaluation accuracy, the similarity, the model pre-evaluation accuracy, and the prediction model, and 
 presents the score and the prediction accuracy together with the trained model candidate to the model user via the model user calculation device. 
   
     
     
         3 . The information processing system according to  claim 2 , wherein
 the model management server generates the prediction model by a regression analysis based on the model user evaluation accuracy, the similarity, and the model pre-evaluation accuracy.   
     
     
         4 . The information processing system according to  claim 2 , wherein
 the model management server generates the prediction model by a neural network based on the model user evaluation accuracy, the similarity, and the model pre-evaluation accuracy.   
     
     
         5 . The information processing system according to  claim 1 , wherein
 the model user calculation device
 acquires a data outline including a type and a data size of the model application target data, and 
 transmits the acquired data outline to the model management server, and 
   the model management server selects the trained model candidate from the plurality of trained models based on the data outline.   
     
     
         6 . The information processing system according to  claim 5 , wherein
 the model user calculation device
 receives task information related to use of the trained model candidate that is a search target input by the model user, and 
 transmits the received task information to the model management server, and 
   the model management server selects the trained model candidate from the plurality of trained models based on the task information and the data outline received from the model user calculation device.   
     
     
         7 . An information processing method for an information processing system to select a valid trained model candidate from a plurality of trained models and present the selected trained model candidate to a model user, wherein
 the information processing system includes a model management server, and a model user calculation device configured to be operated by the model user,   the information processing method comprising:   storing, by the model management server, in a storage unit, the plurality of trained models, a model pre-evaluation data feature that is a feature of model pre-evaluation data for pre-evaluating performance of the plurality of trained models, and model pre-evaluation accuracy that is model accuracy of the plurality of trained models pre-evaluated based on the model pre-evaluation data;   storing, by the model user calculation device, in a storage unit, model application target data that is an application target of the trained model candidate selected from the plurality of trained models;   calculating, by the model user calculation device, a model application target data feature that is a feature of the model application target data;   transmitting, by the model user calculation device, the calculated model application target data feature to the model management server;   calculating, by the model management server, similarity between the model pre-evaluation data feature and the model application target data feature received from the model user calculation device;   calculating, by the model management server, a score for predicting validity of the trained model candidate based on the similarity and the model pre-evaluation accuracy; and   presenting, by the model management server, the score together with the trained model candidate to the model user via the model user calculation device.