ketch

What it is

Overview

Most research tooling for agents means wiring up several provider SDKs, each with its own auth and response shape. ketch collapses that into one binary with three research surfaces and two fetch surfaces.

ketch search  "query"   # web pages
ketch code    "query"   # real OSS source
ketch docs    "query"   # library documentation
ketch scrape  <url>     # HTML or PDF → markdown
ketch crawl   <url>     # BFS or sitemap walk

Output is YAML frontmatter followed by content, so it stays readable in a terminal and parseable by a program. Add --json to any command for a structured object instead.

An operator configures the backend once — ketch config set backend searxng — and every later ketch search call runs without knowing which provider serves it.

Install

Installation methods

# macOS / Linux, x86_64 or arm64
$ curl -fsSL https://ketch.run/install | sh

# Homebrew
$ brew install ketch

# Go
$ go install github.com/1broseidon/ketch@latest

The install script:

• Picks the build for your OS and architecture
• Verifies it against the release's checksums.txt
• Installs to /usr/local/bin if writable, otherwise ~/.local/bin
• Requires no sudo
• Never half-overwrites a running binary

To avoid piping a URL into a shell, read install.sh or download a prebuilt binary from the releases page. Pin a version or redirect the target with sh -s -- --version v0.17.0 --bin-dir ~/bin.

Quickstart

First commands

No API key needed. auto falls through the keyless providers until one answers, and reports which one served.

$ ketch search "golang error handling"
---
query: golang error handling
backend: parallel
result_count: 5
---
Error handling and Go - The Go Programming Language
  https://go.dev/blog/error-handling-and-go
  The language's design and conventions encourage you to explicitly check...

Best Practices for Error Handling in Go
  https://www.jetbrains.com/guide/go/tutorials/handle_errors_in_go/
  How can a reader see that any of these functions might observe an error?

$ ketch scrape https://go.dev/blog/error-handling-and-go
---
url: https://go.dev/blog/error-handling-and-go
title: Error handling and Go
words: 1693
---
## Introduction

If you have written any Go code you have probably encountered the built-in
`error` type...

Accepted input: one URL, several URLs, a JSON array, a file of URLs, or stdin. ketch detects the shape, so there's no batch flag. PDFs are detected by MIME type or signature and parsed to text. JS-shell pages are re-fetched through headless Chrome automatically, with the same output either way.

Choosing a surface

When to use each command

First match wins. The Not column names the most common mistake for each row.

In reverse: search finds URLs, scrape reads them, crawl reads a site, code reads public source, docs reads library docs. search --scrape fuses the two when you already want full content from every hit. Budget it like a scrape.

Commands

Command reference

Twelve commands. --json is the only flag global to all of them; everything else is per-command. Expand a row for its full flag list.

$ ketch search "query" --limit 10
$ ketch search "query" --scrape          # fetch full content per result
$ ketch search "query" -b brave
$ ketch search "query" --multi           # federate, rank-fuse
$ ketch search "query" --multi=brave,exa # a specific set

$ ketch code "http.NewRequestWithContext" --lang go --limit 2
---
query: http.NewRequestWithContext
lang: go
backend: grepapp
result_count: 2
---
harness/harness  registry/app/remote/clients/registry/client.go  (line 207)
  req, err := http.NewRequestWithContext(ctx, http.MethodGet, buildPingURL(c.url), nil)
  https://github.com/harness/harness/blob/main/...

$ ketch docs "routing" --library /vercel/next.js
$ ketch docs --resolve "next.js"     # name → Context7 IDs

$ ketch scrape <url>
$ ketch scrape <url1> <url2> <url3>      # concurrent batch
$ ketch scrape urls.txt                 # one URL per line
$ ketch scrape '["url1","url2"]'        # JSON array
$ echo "url1\nurl2" | ketch scrape      # stdin

$ curl -L https://example.com | ketch extract
$ cat page.html | ketch extract --select article --max-chars 4000

$ ketch crawl https://docs.example.com --depth 2
$ ketch crawl https://docs.example.com --sitemap
$ ketch crawl https://docs.example.com --background
$ ketch crawl status [id]
$ ketch crawl stop <id>

$ ketch browser status
$ ketch browser install    # download Chromium to ketch's cache dir

$ ketch config                  # effective config + active backends, as JSON
$ ketch config init
$ ketch config set backend searxng
$ ketch config path

$ ketch cache
$ ketch cache clear

$ ketch doctor

$ ketch mcp serve

$ ketch version
ketch v0.17.0
  commit: 8f41359
  built:  2026-09-16T22:51:07Z
  go:     go1.25.7 linux/amd64

Workflows

Common workflows

Bounded research query

Search, then fetch full content for every hit — bounded, because an unguarded page can cost ~25k tokens.

$ ketch search "raft consensus implementation tradeoffs" \
    --scrape --limit 5 --max-chars 6000 --trim

Crawl a docs site into the cache

Crawl once in the background; every page lands in the cache, so later scrapes are local reads.

$ ketch crawl https://docs.example.com --background
→ crawl_id: c_a1b2c3d4

$ ketch crawl status c_a1b2c3d4
→ {"status": "running", "pages": 847, ...}

# once complete, individual pages come from cache — no refetch
$ ketch scrape https://docs.example.com/guide/auth

Federated search across providers

Rank fusion surfaces what several engines agree on, and tags each result with who returned it.

$ ketch search "postgres connection pooling pgbouncer vs pgcat" --multi

Convert existing HTML

$ curl -L https://example.com/post | ketch extract --trim --max-chars 4000

Pages behind a session or consent wall

Export a Netscape cookies.txt from your browser, then attach it to any fetch.

$ ketch scrape <url> --cookie-file ~/cookies.txt
$ ketch config set cookie_file ~/cookies.txt   # persist as a default
$ ketch scrape <url> --cookie-file ""          # disable for one run

Only cookies whose domain, path, and secure scope match are sent, re-evaluated on every redirect. Values are never printed — not in output, JSON, errors, or doctor. Respecting a site's terms and using only your own session cookies is the operator's responsibility.

Exit-code branching in scripts

$ ketch search "$q" --json > out.json
$ case $? in
    0) jq -r '.results[].url' out.json ;;
    4) echo "upstream down, retry later" ;;
    5) echo "needs configuration — run ketch doctor"; exit 1 ;;
  esac

Research playbook

Usage guidelines

These are the disciplines the bundled skill enforces, for agents and terminal users alike.

• Bound every fetch. --max-chars 4000–8000 plus --trim on any page you haven't seen. Skipping the cap should come with a reason.
• Cite every claim. A synthesis without source URLs isn't a deliverable.
• Treat exit codes as control flow. Classify before reacting; never retry a 2 or 3 unchanged.
• Propose, then mutate. config set, browser install, and installs are operator actions — confirm the exact command first, and never touch a value that's already working.

Token budgets

Worked session

Two queries, three scrapes, one corroboration — including a rate limit and a source that never came back.

$ ketch search "Go iter.Seq real-world gotchas" --limit 5
→ exit 4: [upstream] ddg rate limited
  # an explicit backend failed — rotate, don't retry unchanged

$ ketch search "Go iter.Seq real-world gotchas" -b brave --limit 5
→ 5 results, 4 unique hosts → picked 3 primary sources

$ ketch scrape <u1> <u2> <u3> --max-chars 6000 --trim
→ u1, u2 ok; u3 failed (503) — named in the synthesis, not dropped silently

$ ketch code "iter.Seq" --lang go --limit 3
→ 3 repos with file and line URLs, to corroborate real usage

Five claims, each cited to its URL. The unretrieved source is listed as unretrieved. Where two sources conflict, the conflict is stated and attributed rather than averaged away.

Common mistakes

Backends

Backends by surface

$ ketch config set brave_api_key <key>
$ ketch config set tavily_api_key <key>
$ ketch config set serpbase_api_key <key>
$ ketch config set serply_api_key <key>

$ ketch config set searxng_url http://my-searxng:8080
$ ketch config set degoog_url http://my-degoog:8090

Configuration

Config file and environment variables

Defaults live in ~/.config/ketch/config.json.

$ ketch config init                    # write a default config file
$ ketch config set backend searxng
$ ketch config set browser chrome      # JS-rendered page fallback
$ ketch config                         # effective config + active backends

Every scalar key can also come from the environment as KETCH_ plus the upper-snake key name — useful in containers and CI where writing a file is awkward.

$ KETCH_BRAVE_API_KEY=<key> ketch search "query"
$ KETCH_BACKEND=ddg KETCH_LIMIT=10 ketch search "query"
$ KETCH_CONFIG=/etc/ketch/config.json ketch config

$ ketch cache          # stats
$ ketch cache clear
$ ketch scrape <url> --no-cache

$ ketch config set browser chrome       # from PATH
$ ketch config set browser /usr/bin/google-chrome-stable
$ ketch browser install                 # download Chromium
$ ketch browser status

Exit status

Exit code reference

Every failure mode has a number, so a script or an agent can branch on the outcome instead of pattern-matching an error string.

The MCP server carries the same taxonomy, as stable message prefixes: [validation], [not_found], [upstream], [precondition], [cancelled]. One asymmetry: a CLI crawl interrupted by SIGINT exits 0 with partial results, by design.

Pitfalls

Known pitfalls

• Scraping a bare domain auto-probes /llms.txt and may return that instead of the homepage. The title field reveals the swap; --no-llms-txt opts out.
• Batch scrape reports per-URL failures inside a successful call. The command exits 0 with per-result errors set — check every entry, don't just check the exit code.
• docs resolve never returns empty. Garbage in gets confident fuzzy matches out, so vet the name rather than trusting the score.
• Regex is per-backend. grepapp and sourcegraph accept it; github rejects it with a pointer to the other two.
• Background crawls are CLI-only. The MCP crawl tool is synchronous and capped at 30 pages by default, 100 hard, three minutes wall clock.
• The page cache is single-process. Running the MCP server long-term degrades concurrent CLI scrapes to uncached.

Agents

Using ketch with agents

Instead of teaching an agent a search API, a code API, and a docs API — each with its own auth and response shape — give it one binary and five surfaces.

Use `ketch` for external research — web pages, OSS code, library docs.
- `ketch search "query"` / `--scrape` for results with full content
- `ketch scrape <url> [url...]` for clean markdown from one or more URLs
- `ketch extract` for already-fetched HTML piped in
- `ketch code "query" --lang go` for real OSS code with line context
- `ketch docs "query" --library /org/repo` for version-aware docs
All commands support `--json`. `ketch config` reports active backends.
Bound every scrape with --max-chars and --trim. Cite every claim.

Bundled skill

The repo ships a fuller playbook as a skill, covering surface routing, token budgets, error-code control flow, a deep-research recipe, and guided backend setup. Any agent that loads SKILL.md-style files can use it.

Read it at skills/ketch/. Most of this page's playbook and gotchas sections come from it.

MCP server

For agents that speak MCP rather than shelling out, the same five surfaces run as tools over stdio, on the same config and backends as the CLI.

$ claude mcp add ketch -- ketch mcp serve

Claude Code plugin

ketch only needs the CLI on PATH. The repo also works as a plugin marketplace that installs the MCP server and skill together.

$ claude plugin marketplace add 1broseidon/ketch
$ claude plugin install ketch@ketch