About tokensift

tokensift turns raw terminal, VS Code, and CI failure output into conservative debugging briefs for Codex, Claude Code, ChatGPT, OpenCode, and other coding agents.

Raw failure logs often bury the root cause under progress bars, repeated warnings, dependency frames, cached layers, wandb/NCCL chatter, and CI noise. tokensift keeps the evidence a coding agent needs and folds repetitive junk.

Why tokensift exists

Copying an entire failure log into a coding agent is easy, but the longest log is not always the most useful prompt. Repeated progress lines and framework internals compete with the command, the first relevant user frame, the final exception, and the exact resource or build failure. tokensift was built to make that evidence boundary visible instead of silently summarizing everything into prose.

What conservative means

A conservative brief keeps concrete evidence before it removes noise. The parser retains type-specific signals, lists the categories it folded, and asks for a root cause, the smallest reasonable fix, and a verification command. It does not claim to diagnose the application, edit code, or guarantee that every relevant line was retained. The user reviews the brief before sharing it.

How the method is maintained

The cleaner uses deterministic JavaScript parsers rather than an LLM API. Parser behavior is checked against regression fixtures for Python, CUDA, Docker, pytest, TypeScript, test failures, generic terminal output, secret redaction, and prompt-injection-like text. New rules are expected to preserve the primary failure evidence before they fold additional lines.

Read the complete processing pipeline and the supported-log matrix for concrete behavior and limitations.

Operation and contact

tokensift is an independently operated developer utility. Product feedback, parser reports, privacy requests, and site questions can be sent to [email protected]. Reports should contain only the smallest redacted example needed to reproduce a parser issue.

Storage boundary

The backend-hosted deployment stores Sift submissions in a private first-party SQLite database for diagnostics, retains hosted raw submissions for up to 30 days by default, and does not require a login or LLM API call. Review the privacy policy before submitting a log.