unilm

What it is

Large-scale self-supervised pre-training and foundation model research across tasks, languages, and modalities. The repository organizes multiple model families and toolkits aimed at vision, language, and multimodal understanding.

How it works

The collection covers architectures, pre-training strategies, and fine-tuning pipelines for diverse modalities (text, images, layout/format, and multimodal data). It includes toolkits such as s2s-ft for sequence-to-sequence fine-tuning and references to multiple model families (e.g., BEiT, LayoutLM, TrOCR, Kosmos series). The material points to publicly released models, code, and papers across iterations (e.g., BEiT, BEiT-3, LayoutLMv3, Kosmos-1/2/2.5).

Getting started

Install and usage instructions are provided in the repository README and related subprojects, but the truncated README content here does not include explicit commands. The latest releases section shows versioned artifacts such as yoco.v0 and s2s-ft v0.3/0.2/0.0 with notes on CPU fixes and feature support.

Recent releases

• yoco.v0 YOCO (2024-05-09): https://arxiv.org/abs/2405.05254
• s2s-ft.v0.3 s2s-ft version 0.3 (2020-04-02): fix an issue running on CPUs: https://github.com/microsoft/unilm/commit/13641268b59df5cf90d27b451d87ab58b6a07055
• s2s-ft.v0.2 s2s-ft version 0.2 (2020-03-13): - Support MiniLM in s2s-ft
• s2s-ft.v0.0 s2s-ft version 0.0 (2020-03-10): seq2seq fine-tuning with unilmv1/unilmv2

Traction

The repository lists 22178 stars and 2703 forks, indicating community engagement. (stars_7d or stars_1d not provided, so no time-based traction section beyond the raw counts.)

Behind the repo

Links reference broader Microsoft foundation-model and LMOps initiatives (e.g., LLMOps, RedStone) but this entry does not include a corporate/brand-backed startup link beyond the organization’s scope.

Caveats

• License: MIT
• Created: 2019-07-23
• Last push: 2026-01-23
• Open issues: 684
• Language: Python
• Topics include beit, layoutlm, trocr, xlm-e, multimodal, etc.