融合先验知识的神经机器翻译模型研究
首发时间:2020-03-13
摘要:近些年随着深度学习的发展,通过结合大规模的语料和深层神经网络,神经机器翻译在很多方面已经超越了统计机器翻译,展现了愈加鲜活的生命力。神经机器翻译采用了编码器-解码器架构,通常由卷积神经网络或循环神经网络组成,由于其端到端的模型训练方式,没有充分利用句子中隐含的先验知识,如句子的短语结构、依存关系等信息。谷歌在2017年提出了Transformer模型,该模型摒弃了循环神经网络,由多个前馈神经网络及自注意机制构成,在许多自然语言处理任务中都达到了最好的性能指标。本文在Transformer翻译模型的基础上,提出一种利用词向量融合先验知识的方法,通过在原始文本中融合词性标注、短语结构等先验知识,提升了基于自注意机制的Transformer模型的翻译效果。本文针对中英翻译任务,通过对比基线Transformer模型,本文的方法可以提高1.19个BLEU值。
For information in English, please click here
Modeling prior knowledge for Neural Machine Translation System
Abstract:With the rapid development of deep learning in recent years, through the combination of large-scale corpora and deep neural networks, neural machine translation has surpassed statistical machine translation in many aspects, showing an increasingly vitality. Neural machine translation model adopts encoder-decoder architecture, and usually consists of convolutional neural networks or recurrent neural networks. The neural machine translation system adopts end-to-end training method, which makes no use of the implicit prior knowledge within the sentence, such as the phrase structure and dependency information. Transformer is proposed by Google in 2017 which is based solely on feed-forward networks and self-attention mechanisms, dispensing with recurrence entirely. This paper proposed a novel method that using word embedding to integrate prior knowledge into Transformer model. By adding part-of-speech tagging and phrase structure information of the text, the translation effect of Transformer model is improved. Compared to baseline Transformer model, experiments on Chinese-English translation show our method improves 1.19 BLEU scores.
Keywords: machine translation prior knowledge word embedding
基金:
引用
No.****
动态公开评议
共计0人参与
勘误表
融合先验知识的神经机器翻译模型研究
评论
全部评论0/1000