rtk: cutting noise without cutting the signal

A few days ago rtk was #2 trending on GitHub. The pitch: a command-line proxy that cuts 60-90% of the tokens a coding assistant spends reading git log, ls, build output, test output. A dependency-free Rust binary that sits between the command and whatever reaches the model's context.

Before installing anything, something didn't add up: the exact same description text — word for word — was copied across half a dozen different repos, from different accounts. A typical pattern for spam inflating visibility, or worse, malicious binaries disguised as the trending tool.

The simple stuff works

Once installed, I tested it first against this very repo: real commands, not someone else's benchmarks. git log --stat shrinks 66.8%. A verbose ls -la on node_modules, 82.8%. A docker ps, 65%. Where there's real noise — headers, whitespace, repeated lines — it cuts well, and filtering costs milliseconds. Where there was no noise to begin with (a four-page Astro build, a grep with two hits), the saving is 0%. Nothing strange here: it does what it says, when the input is what it expects.

Raising the stakes

A four-page repo proves nothing at scale. So I cloned facebook/react (500 real commits) and ran two security tools rtk doesn't natively support: gitleaks and codeql. That's exactly what its documented generic commands — rtk err and rtk summary — are built for, meant to wrap "any command" — so this isn't misuse, it's the official path.

gitleaks found 5 possible secrets in the history (all five turned out to be real false positives — a GitHub token flagged in the code itself as "public on purpose", and a repeated test key). rtk err shrank that output to a comfortable size, but showed only 1 of the 5 findings — the other 4 sit in a log file on disk that nobody checks unless they know it's there.

With codeql the scale jumps an order of magnitude: 4,471 files analyzed, 1,505 real findings, 61 of them tagged as security (code injection, cross-window information leaks). The resulting SARIF weighs 5.65 MB.

{
  "runs": [
    {
      "results": [
        {
          "message": { "text": "[Sensitive data] is sent to another window without origin restriction." },
          "rule": { "id": "js/cross-window-information-leak" }
        }
        // ... +1504 more
      ]
    }
  ]
}

One example, and "+1504 more." Nobody would paste 5.65 MB of raw SARIF into an LLM — you have to compare against what's reasonable, not against the absurd. But "use rtk so you don't burn tokens on verbose output" is exactly what its own documentation recommends for any noisy command. An agent following that advice to the letter, on the output of a security scanner, sees 1 of 1,505 problems and calls the task done.

The same pattern, three times

I set up a Vitest suite with two deliberate failures. rtk vitest never got to run it: the project used pnpm, and a harmless pnpm warning (unapproved build scripts) made rtk abort before running anything. Once that warning was fixed, rtk vitest did run — but the result weighed more than the raw output (4,703 against 1,754 bytes): it expanded internal node_modules paths that Vitest's own reporter already trims by default.

In React I tried the same thing at real scale: a genuinely broken test, inside its own wrapper (yarn test → a script at scripts/jest/jest-cli.js). rtk jest didn't understand it — it tried to invoke Jest on its own, ignoring the project's wrapper, and failed outright without reaching the tests. The generic wrapper command (rtk test yarn test ...) did run the real command, but lost the why: it showed which two tests failed, not the error message or stack trace needed to fix them. I added real ESLint violations to the same file and tried rtk lint — again it ignored React's own wrapper (scripts/tasks/eslint.js), ran raw ESLint over the entire monorepo instead of the affected file, and parsing the resulting JSON failed.

Three different tools (test runner, linter, across two different projects), the same failure: when a project wraps its command with its own script — very common in large repos — rtk doesn't detect it.

What its own documentation says

Before writing any of this up as a finding, it was worth checking whether it was already flagged. rtk's site precisely lists which commands it optimizes — jest, vitest, eslint are all on that list — and explicitly says anything not listed "goes through passthrough, unchanged." But what broke here isn't the "unsupported" case: it's a failure in the analyzer on commands they themselves claim to support, when the project invokes them through its own wrapper. Their troubleshooting page doesn't mention wrappers, monorepos, or custom scripts of any kind.

Searching their Pull Requests turned up the explanation: there's "smart" support for yarn and for pnpm run <script> in active development, not yet merged to main. The yarn PR (#867) cites real telemetry: 126 unhandled yarn commands across 615 sessions over 30 days — yarn lint 19 times, yarn vitest 14. And in the review of that same PR, one of the maintainers found, testing against a real project (babel/babel), a failure from the same family as ours: a yarn install that genuinely failed (exit code 1, broken workspace build) collapsed to the word "ok". The exit code was preserved — but an agent seeing "ok" next to exit 1 has no way to know what broke without re-running the command raw.

The counter doesn't measure what you think

rtk ships its own savings dashboard (rtk gain ). Its usage guide publishes the exact formula: input tokens are len(raw_command_output) / 4 — but "raw output" means whatever rtk itself captured when it ran, not what the project's actual command would produce measured separately. In the rtk lint run over React, the dashboard logged ~244,000 input tokens — far more than the real yarn lint (1,155 bytes) — because what it actually ran was ESLint over the whole monorepo, due to the detection failure. The 99.8% "savings" it reports is real, mathematically, against that wrong command it generated itself. It can't be compared to what a developer would get by typing yarn lint, because that's never what actually ran.

The codeql SARIF line is different: there, the comparison is direct, same input file. The 100% savings the dashboard certifies is, literally, the same fact as "hides 1,504 of 1,505 findings" — not interpretation, the same byte reduction seen from two angles.

What others already knew

Two open issues in the repo, unresolved, confirm this isn't an isolated case from this test:

The hook, for real

The "correct" way to integrate isn't typing rtk by hand before every command: it's a hook (rtk init -g) that transparently rewrites Bash commands. I installed it in my real Claude Code config — it patched settings.json with an automatic backup, added a ten-line RTK.md, nothing invasive — and it worked exactly as documented, no restart needed. Given the shell injection finding in issue #640, which affects exactly this hook, I uninstalled it after testing.

Conclusion

None of this says rtk is useless. In the cases it supports well — git, ls, docker, directly invoked commands — it cuts real noise at negligible cost. But the "60-90%" headline doesn't survive looking underneath, and it's not the only thing that matters when deciding whether to install something:

The savings depend on which command, on whether the project wraps it with its own script, and the tool's own counter isn't built to warn you when something went wrong. The difference between believing a number and measuring it yourself — with the real command, at real scale, checked against the project's own docs, PRs, and issues — is exactly the work worth doing before writing about any tool.