Vo Van Phuc's 135M Vietnamese Encoder Hits 850K Monthly Downloads
Vietnamese sentence embedding model combining SimCSE contrastive learning with PhoBERT backbone, 135M parameters. Crossed 850K monthly downloads on Hugging Face , used by 22+ public Spaces. Scores 69.46 on ViRerank and 87.74 on MNLI-R, leading Vietnamese reranking benchmarks.
Takeaways
- Vietnamese sentence embedding model combining SimCSE contrastive learning with PhoBERT backbone, 135M parameters.
- Crossed 850K monthly downloads on Hugging Face , used by 22+ public Spaces.
- Scores 69.46 on ViRerank and 87.74 on MNLI-R, leading Vietnamese reranking benchmarks.
- Weak on open ended retrieval, significantly behind vietnamese-bi-encoder on that task.
- Requires pyvi word segmentation before tokenization, plugs into sentence-transformers directly.
- Best used as a second stage reranker in Vietnamese RAG and semantic search pipelines.
Vo Van Phuc’s Vietnamese encoder , sup-SimCSE-VietNamese-phobert-base , has crossed 850,000 monthly downloads on Hugging Face. The four-year-old checkpoint combines a compact 135M-parameter footprint with direct support in the popular sentence-transformers library.
For developers, its practical appeal comes from easy local deployment and Vietnamese-specific training. Reported evaluations show an uneven profile: competitive semantic similarity, strong candidate reranking, and weaker open-ended retrieval than newer alternatives.
PhoBERT supplies the Vietnamese backbone
PhoBERT is a RoBERTa-style encoder pretrained on Vietnamese text. Fine-tuning it with supervised SimCSE produces a 768-dimensional vector for each sentence, allowing applications to compare Vietnamese passages with cosine similarity.
During unsupervised SimCSE training, the encoder processes the same sentence twice with different dropout masks. The contrastive objective pulls those two representations together while separating other sentences in the minibatch. Supervised SimCSE adds natural-language inference labels, using entailment pairs as positives and contradiction pairs as hard negatives.
The resulting geometry supports semantic search, clustering, duplicate detection, FAQ matching, and retrieval-augmented generation. Cosine scores remain task-specific, so applications should tune similarity thresholds with representative data.
Segmentation is part of the input contract
PhoBERT’s pretraining pipeline used word-segmented Vietnamese, including underscores between syllables that form one word. A segmenter such as pyvi converts đại học to đại_học before PhoBERT applies its subword tokenizer. Queries and indexed documents need the same preprocessing, and already segmented text should not be segmented again.
Vietnamese diacritics should be preserved whenever the source provides them. Applications that receive unaccented text, misspellings, code-switching, or informal chat should include those patterns in evaluation because they differ from the model’s primary training distribution.
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
Read original report
