一种基于层级注意力机制的自然语言理解模型
首发时间:2018-12-06
摘要:任务型对话系统一般包括自然语言理解、对话状态追踪对话策略选择和自然语言生成等模块。其中,自然语言理解模块负责提取用户输入的自然语言句子中的关键信息,将自然语言转换为向量表示,屏蔽不同语言表达的差异性和同种语言表达的丰富性。自然语言理解模块是任务型对话系统的关键模块,而现有的面向任务型对话系统的自然语言理解模型往往仅考虑了词语级别的语义特征影响了任务型对话系统的性能。本文提出一种面向任务型对话系统的基于层级注意力机制的自然语言理解模型(Hierarchical Attention based Natural Language Understanding,HA-NLU)。综合考虑字、词、句子三个语义层级,HA-NLU使用基于循环神经网络和卷积神经网络构建的的语义解析网络提取每个层级的语义特征;同时,HA-NLU利用基于层级注意力机制构建的层间语义传递结构,逐层递进地挖掘自然语言文本的语义特征。实验结果表明,相比仅利用词语层级语义的模型,HA-NLU能够有效提升任务型对话系统的对话成功率,降低系统的平均对话轮次。
关键词: 人工智能 任务型对话系统 层级注意力机制 神经网络
For information in English, please click here
A Hierarchical Attention based Natural Language Understanding model
Abstract:The typical framework of task-oriented dialogue system consists of natural language understanding, dialog state tracking, dialog policy and natural language generation. Among the modules mentioned above, natural language understanding plays an important part because it transforms the natural language utterance users input into a vector representing the syntactic and semantic features and the following modules are based on the extracted feature. Current works focus on only semantic feature at the word level, while this paper introduces a natural language understanding model named HA-NLU based on hierarchical attention to leverage the semantic feature at three different levels exist in the utterance. HA-NLU parse semantic feature at different levels including the character level, the word level and the sentence level. For a particular level, HA-NLU uses a semantic parsing network based on RNN and CNN to extract feature from the text. The structure based on hierarchical attention in HA-NLU offers a tunnel transferring the semantic feature from lower levels to higher levels dynamically. To evaluate the model, this paper implements a task-oriented dialogue system. Experiments on the system show comparing with models focus on only semantic feature at word level, HA-NLU can raise the success rate of the system and decrease the average number of turns per dialogue of the system at the same time.
Keywords: artificial intelligence task-oriented dialogue system hierarchical attention Neural Network
基金:
引用
No.****
同行评议
勘误表
一种基于层级注意力机制的自然语言理解模型
评论
全部评论0/1000