Combinatorial Generalization Ability Evaluation Method of SQL-to-text Model
Translating from Structured Query Language(SQL)to natural language can improve the usability of a database.Some progress is currently being made in this research,which mainly uses machine learning models.However,the capabilities of the existing translation models are still insufficient for practical applications.Because combinatorial generalization is a necessary ability for an SQL-to-text model to improve the translation effect in practical applications,and there is currently a lack of research on this ability for such models,a combination of SQL-to-text models is proposed as a generalization ability assessment method.This method generates a large amount of SQL and corresponding natural-language translations(referred to as SQL-natural language pairs)based on an existing SQL-to-text dataset.These SQL-natural language pairs are then divided into training and test data according to the number of SQL clauses they contain.Thus,the SQL clauses in the test data appear in the training data in different combinations,which produces a new data set that can be used to evaluate the generalization ability of the model combination.The evaluation results show that this method has a higher degree of query-knowledge use.It utilizes a more reasonable method to divide data,and the obtained data set meets the requirements for the evaluation of combinatorial generalization ability.It is close to the actual application scenario of the model,and is less restricted by the original data set.The combinatorial generalization ability of the existing models still needs to be further improved.Among them,the relationship-aware graph converter model designed for SQL-to-text tasks has the weakest combinatorial generalization ability,indicating that the original SQL-to-text data set is insufficient for the investigation of the combinatorial generalization ability.
国家科技期刊平台
NSTL
万方数据
