OpenMythos is an open-source, theoretical implementation of the Claude Mythos-like model featuring a Recurrent-Depth Transformer with a three-stage architecture, switchable attention, and a sparse MoE feed-forward. It provides Python-based configuration and usage examples, plus variant configurations and training notes.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
OpenMythos is an open-source, theoretical implementation of the Claude Mythos model. It implements a Recurrent-Depth Transformer (RDT) with three stages: Prelude (transformer blocks), a looped Recurrent Block (up to max_loop_iters), and a final Coda. Attention is switchable between MLA and GQA, and the feed-forward uses a sparse MoE with routed and shared experts. The project includes pre-configured model variants and Python usage examples.
How it works
The architecture divides processing into three blocks:
- Prelude: transformer blocks run once
- Recurrent Block: looped over up to
max_loop_iters, updating hidden state h_t via h_{t+1} = A·h_t + B·e + Transformer(h_t, e) - Coda: transformer blocks run once
Attention is selected via cfg.attn_type with two options: "gqa" (Grouped Query Attention) and "mla" (Multi-Latent Attention). RoPE and caching strategies are described, including a MoE-based feed-forward with routing and shared experts. The recurrent loop uses an injection e at every step to maintain input signal across iterations.
Key design notes include stability constraints on the injection parameters to ensure the spectral radius of A remains below 1, and a three-stage flow (Input -> Prelude -> Recurrent Block -> Coda -> Output).
The implementation details are in open_mythos/main.py and the docs page for the OpenMythos class reference.
Getting started
Install:
pip install open-mythos
#uv pip install open-mythos
To enable Flash Attention 2 in GQAttention (requires CUDA and build tools):
pip install open-mythos[flash]
Usage example shows constructing a MythosConfig, selecting attn_type, creating an OpenMythos model, and running a forward pass and generation:
import torch
from open_mythos.main import OpenMythos, MythosConfig
attn_type = "mla" # or "gqa"
base = {
"vocab_size": 1000,
"dim": 256,
"n_heads": 8,
"max_seq_len": 128,
"max_loop_iters": 4,
"prelude_layers": 1,
"coda_layers": 1,
"n_experts": 8,
"n_shared_experts": 1,
"n_experts_per_tok": 2,
"expert_dim": 64,
"lora_rank": 8,
"attn_type": attn_type,
}
if attn_type == "gqa":
cfg = MythosConfig(**base, n_kv_heads=2)
else:
cfg = MythosConfig(
**base,
n_kv_heads=8,
kv_lora_rank=32,
q_lora_rank=64,
qk_rope_head_dim=16,
qk_nope_head_dim=16,
v_head_dim=16,
)
model = OpenMythos(cfg)
ids = torch.randint(0, cfg.vocab_size, (2, 16))
logits = model(ids, n_loops=4)
out = model.generate(ids, max_new_tokens=8, n_loops=8)
Getting started (continued)
Variant models can be loaded via pre-configured mythos_* functions, e.g. mythos_7b(), which return a MythosConfig instance suitable for OpenMythos. A quick summary of variants includes dimensions, number of experts, and context/output sizes for each scale.
Recent releases
Latest releases section shows no releases documented in this README excerpt (RELEASES (latest 0): - none).
Traction
The repository currently lists 14776 stars and 3296 forks. Open issues count is 60. Language is Python and license is MIT.
Behind the repo
No linked startup or company is provided in the provided facts.
Caveats
License: MIT. Created 2026-04-18, last_push 2026-05-23. No explicit open issues beyond the count provided. The README notes the project is a theoretical reconstruction based on public research and speculation and is not affiliated with Anthropic.