Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank¶
Why this mattered¶
This paper mattered because it made sentiment compositionality a concrete supervised learning problem rather than an illustrative weakness of bag-of-words models. The Stanford Sentiment Treebank labeled sentiment not only for whole movie-review sentences but for phrases inside their parse trees, creating a benchmark where systems had to learn how meaning changes under negation, intensification, contrast, and syntactic scope. The Recursive Neural Tensor Network was important less as the final architecture of sentiment analysis than as a demonstration that neural models could be trained to compute meaning over structure, not merely average or count words.
Its immediate effect was to make fine-grained sentiment analysis a standard testbed for neural NLP. After this paper, models could be evaluated on whether they understood that “not good” differs from “good,” or that a positive word embedded in a negative construction may not make the whole phrase positive. This shifted attention toward representation learning for phrases and sentences, and the Stanford Sentiment Treebank became a durable benchmark for recursive networks, Tree-LSTMs, CNNs, RNNs, attention models, and eventually transformer-based encoders.
In retrospect, the paper sits at an important transitional point: after distributed word representations had shown their value, but before large pretrained language models absorbed much of compositional semantics into scale and pretraining. It helped establish the expectation that neural NLP systems should learn semantic phenomena from annotated data and be tested at multiple levels of linguistic structure. Later breakthroughs moved away from explicit parse-tree recursion, but they inherited the central demand this paper sharpened: language models should represent how meanings combine, not just which words occur.
Abstract¶
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, Christopher Potts. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 2013.
Related¶
- cite ← Convolutional Neural Networks for Sentence Classification — Kim compares his CNN sentence classifier against Socher et al.'s recursive neural models on the Stanford Sentiment Treebank.