Generic Natural Language Generation Model Based on Skill Network
The natural language generation model trained with multi-task learning can complete various natural lan-guage generation tasks via one model.However,it is unclear what skills are learned in which part of the model pa-rameters due to all the model parameters being activated for all the tasks.To activate different parts of parameters according to the task,a generic natural language generation model with a sparsely activated approach(SkillNet-NLG)is proposed.A set of skills needed to accomplish the task are pre-defined before performing a task,and then the parameters relevant to these skills are activated while the other parameters are not.This approach can also learn new tasks efficiently by combining the task-related skills properly.The experimental results on natural language generation tasks demonstrate that the proposed model achieves comparable performance to task-specific models,and outperforms previous best performance methods on four of five tasks,including two multi-task learning baseline models(a dense model and a Mixture-of-Expert model).And when adapted to new tasks,it surpasses all baseline systems.
natural language generationmulti-task modelsparsely activated modelskill network
NETL
NSTL
万方数据
维普
