A Effective Bidirectional Mechanism with Pooling for Universial Sentence Representations
首发时间:2019-04-18
Abstract:BiLSTM with max pooling is adopted as a well-performed supervised universal sentence encoder. Max pooling is a common mechanism to get a fixed-size sentence representation. But we find that the max pooling for sentence encoder discards some useful backward and forward information at each time step and depends on a large number of parameters. In this paper, we propose an improved pooling mechanism based on max pooling for universal sentence encoder. The proposed model uses three kinds of methods to refine the backward and forward information at each time step, and then use a max-pooling layer or attention mechanism to obtain a fixed-size sentence representation from variable-length refined hidden states. Experiments conducted on Stanford Natural Language Inference (SNLI) Corpus, and we use it as a pretrained universal sentence encoder for transfer tasks. Experiments show that our model with less parameters performs better.
keywords: Computer Software and Theory Sentence Eembedding LSTM Transfer Tasks
点击查看论文中文信息
充分利用有效的双向网络信息表示句向量的机制
摘要:双向长短时记忆网络是一个有效的获取上下问信息的网络,后面加上一层最大池化层可以有效的用于表达一个句子,但是研究表明单纯的为了获取固定长度句子的最大池化操作会丢失一些有效的上下文信息,因为这个机制仅仅选择了当前时间的最大值,而且也需要很大的参数量,在本篇文章中提出了一种基于最大池化的充分利用上下文信息的机制,本文采用了三种方法去提炼上下文中的信息,最后采用最大池化层获取一个固定长度的表示结果,本文实验在斯坦福大学的自然语言推断数据集上进行,并采用得到的模型进行迁移学习,并且获得不错的效果。
基金:
引用
No.****
同行评议
勘误表
充分利用有效的双向网络信息表示句向量的机制
评论
全部评论0/1000