A Rust-based CLI tool named yek that serializes text-based files in a repo or directory for LLM consumption, leveraging Gitignore rules and Git history to prioritize files. It supports streaming output, multiple dirs, glob patterns, and a configurable yek.yaml.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
yek is a fast Rust-based command-line tool to serialize text-based files in a repository or directory for LLM consumption. It uses .gitignore rules by default, infers important files from Git history, and can stream output when output is piped. It supports processing multiple directories, glob patterns, and individual files, with configuration via a yek.yaml file.
How it works
The tool serializes files into a single output (written to a temp directory by default) and prints the path. It prioritizes more important files to appear later in the output to favor content that LLMs tend to focus on. It can detect piping and stream content accordingly. It can process multiple directories and accept glob patterns or explicit files. Configurables include max output size, token-based mode, and various ignore/include patterns via CLI or yek.yaml.
Getting started
Install options include running the curl/PowerShell install commands suitable for Unix-like systems and Windows PowerShell as shown in the README. You can also build from source using cargo install --path . after cloning.
Example usage:
yek
yek src/ | pbcopy
yek --tokens 128k
yek --max-size 100KB --output-dir /tmp/yek src/
yek src/ tests/
yek "src/**/*.ts"
yek "src/main.rs" "tests/*.rs" "docs/README.md"
CLI Reference (selected)
yek --help
Usage: yek [OPTIONS] [input-paths]...
Arguments:
[input-paths]... Input files and/or directories to process
Options:
--no-config Do not use a config file
--config-file <CONFIG_FILE> Path to the config file
-V, --version Print version of yek
--max-size <MAX_SIZE> Max size per chunk. e.g. "10MB" or "128K" or when using token counting mode, "100" or "128K" [default: 10MB]
--tokens <TOKENS> Use token mode instead of byte mode
--json Enable JSON output
--debug Enable debug output
--line-numbers Include line numbers in output
--output-dir [<OUTPUT_DIR>] Output directory. If none is provided & stdout is a TTY, we pick a temp dir
--output-name [<OUTPUT_NAME>] Output filename. If provided, write output to this file in current directory
--output-template [<OUTPUT_TEMPLATE>] Output template. Defaults to ">>>> FILE_PATH\nFILE_CONTENT"
--ignore-patterns <IGNORE_PATTERNS>... Ignore patterns
--unignore-patterns <UNIGNORE_PATTERNS>... Unignore patterns. Yek has some built-in ignore patterns, but you can override them here.
-t, --tree-header Include directory tree header in output (incompatible with JSON output)
--tree-only Show only the directory tree (no file contents, incompatible with JSON output)
-h, --help Print help
Configuration
A yek.yaml (or yek.toml / yek.json) config allows adjusting ignore patterns, priority rules, binary extensions, and output options. Configurable options include file processing (max_size, tokens, ignore/unignore patterns) and output configuration (json, debug, line_numbers, output_dir, output_name, output_template, tree_header, tree_only). Some options like --no-config, --config-file, and --version remain CLI-only. The example yek.yaml shows priority rules, Git-based boost, and various output settings.
Performance
yek is implemented in Rust and performs many tasks in parallel. A benchmark compares yek to repomix on a Next.js project, where yek completes in 5.19 seconds and is described as 230x faster than repomix.
Roadmap
See the issues for proposed features. Open to new feature proposals with detailed proposals.
License
MIT