- 双塔模型 (Siamese Networks)
- 论文："Learning Deep Structured Semantic Models for Web Search using Clickthrough Data"
- 作者与时间：Huang et al., 2013
- 论文贡献：提出了DSSM（Deep Structured Semantic Model），利用Siamese结构结合点击数据训练深度语义模型，对文本进行高效编码和匹配。
-
论文链接：论文
- 论文："Learning Deep Structured Semantic Models for Web Search using Clickthrough Data"
- 预训练模型 (Pre-trained Language Models)
- 论文："BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"
- 作者: Devlin et al., 2018
- 贡献: BERT通过双向Transformer捕捉上下文语义，是目前许多语义匹配任务的基准模型。
- 论文链接：论文
- 论文："Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks"
- 作者: Reimers and Gurevych, 2019
- 贡献: 通过Siamese网络结构改进BERT在句子相似度任务上的性能，并提升推理效率。
- 论文链接：论文
- 论文："BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"
- 生成式模型在语义匹配中的应用
- 论文："T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"
- 作者：Raffel et al., 2020
- 贡献: T5模型将语义匹配任务转换为生成问题，展示了通用生成式模型在匹配任务上的优势。
- 论文链接：论文
- 论文："T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"
- 对比学习
- 论文："SimCSE: Simple Contrastive Learning of Sentence Embeddings"
- 作者：Tianyu Gao et al., 2021
- 贡献：
- Proposed a method for leveraging BERT in a contrastive learning framework to improve sentence embeddings.
- In the unsupervised setup, SimCSE generates positive pairs by using the same sentence twice with random dropout as augmentation.
- In the supervised setup, labeled sentence pairs are used as positives, while other pairs in the batch serve as negatives.
- SimCSE achieved state-of-the-art results on multiple semantic textual similarity benchmarks.
- 论文链接：论文
- 论文："ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation"
- 作者：Yue Zhang et al., 2021
- 贡献：
- Proposed using BERT in a contrastive learning framework with a focus on augmentation strategies to create diverse positive pairs.
- Strategies include token shuffling, cutoff, and back-translation to enrich positive samples.
- Demonstrated that augmentations can significantly improve sentence representation quality.
- 论文链接：论文
- 论文："SimCSE: Simple Contrastive Learning of Sentence Embeddings"
