THUDM/LongWriter focuses on enabling long-form text generation from long-context LLMs, with deployment guidance and training data workflows. It provides code examples for tokenizers/models, vllm deployment, and evaluation setups; documented outputs include long-form case prompts and RL-based LongWriter-Zero variant.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
LongWriter is a Python-based project that aims to enable generation of 10,000+ words from long-context LLMs. It exposes deployment workflows, model variants (LongWriter-glm4-9b and LongWriter-llama3.1-8b), and an automated data construction pipeline called AgentWrite. It includes evaluation benchmarks LongBench-Write and LongWrite-Ruler, along with example prompts and outputs.
How it works
The repository provides:
- Deployment guidance for LongWriter models using transformers and optional vllm deployment to achieve long-form generation.
- A chat-style interface example using a tokenizer and model to produce long responses, with a code snippet showing a chat call and generation parameters.
- An automated data construction pipeline (AgentWrite) with plan.py and write.py scripts.
- Training setup references that reuse LongAlign code and adapt to new models, including FlashAttention 2.
Getting started
Install environment: (from README)
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained("THUDM/LongWriter-glm4-9b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("THUDM/LongWriter-glm4-9b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")
model = model.eval()
query = "Write a 10000-word China travel guide"
response, history = model.chat(tokenizer, query, history=[], max_new_tokens=32768, temperature=0.5)
print(response)
CUDA_VISIBLE_DEVICES=0 python trans_web_demo.py
Alternatively, deploy with vllm for fast generation as shown in vllm_inference.py.
Recent releases
- Latest releases: none
Traction
- Stars: 1871
- Forks: 182
- Open issues: 32
Behind the repo
- The project is associated with THUDM and references LongWriter-6k datasets and LongWriter-Zero papers.
Caveats
- Language: Python
- License: Apache-2.0
- Created: 2024-08-12
- Last push: 2025-06-24
- Environmental note: recommends transformers>=4.43.0 for deployment
- Training references require FlashAttention 2 according to the training code notes