Repository navigation
feat(cli): embeddable CLI library with CliHost - #73
Merged
Merged
Conversation
Move the markkit CLI into `markkit_cli::run(args, &impl CliHost)`; the binary is a thin wrapper. Hosts own stdio, file reads, URL extraction and MCP, and may refuse --ocr. URL support in NativeHost is the default native-http feature, so embedders link no reqwest. clap default features off to match bashkit. Lets bashkit ship a `markkit` builtin with the identical interface.
chaliy
force-pushed
the
claude/project-thread-solgcd
branch
from
November 1, 2026 02:37
bacdc2b to
2aa8b40
Compare
4 tasks done
Hosts with an async virtual filesystem (bashkit) must not block.
chaliy
added a commit
to everruns/fetchkit
that referenced
this pull request
Nov 1, 2026
<!-- claude-projects-attribution: {"github_login":"chaliy"} -->
_Requested by **Михайло** · [project
thread](https://claude.ai/code/project/chan_019YEkBJuZMHJdXrjom86cGX?thread=cmsg_019YEkBJuZMHJdXrjom86cGXFSAmywce8BSJgNQ4YNsrks)_
## What changed
Prep so bashkit can ship a `fetchkit` builtin with the exact same CLI as
the standalone binary.
- **CLI is now a library.** `fetchkit_cli::run(args, &dyn CliHost) ->
i32`. The `fetchkit` binary is a 6-line wrapper over it with
`NativeHost`. The host owns stdout/stderr, env lookups (`CLOUDFLARE_*`),
MCP stdio, and egress policy (`configure_tool`, e.g. a custom
`HttpTransport`). `run` never calls `process::exit`, prints directly, or
reads `std::env`.
- **Hosts can refuse host-only flags** (`--allow-env-proxy`,
`--render-rakers`, `--bot-auth-key/--bot-auth-agent`,
`--allow-remote-conversion`; `mcp` is refused by any host that doesn't
implement `serve_mcp`). Flags still parse and show in `--help`; the
command fails before any request with `Error: <flag> <reason>`, exit 1.
- **reqwest default features off.** New default `reqwest-tls` feature
carries TLS, HTTP/2, proxy discovery and gzip/brotli/deflate for
`ReqwestTransport`. With `default-features = false` fetchkit links no
TLS stack and no aws-lc-sys (bashkit is ring-only because aws-lc-sys
breaks its aarch64 manylinux cross builds).
- **clap without color/suggestions**, matching bashkit's clap set
(features unify; otherwise output would differ between the binary and
the builtin).
- **Markkit 0.2** (was 0.1.1): one markkit copy when bashkit links both.
Small API updates (`limits.*`, typed warning codes).
- MCP server reads stdin per line instead of holding `StdinLock` across
`.await` (needed for a `Send` future; same behavior).
## Why
Design: markkit + fetchkit "preinstalled" in bashkit sessions with an
identical CLI. Sharing the code (not copying clap structs) is the only
way to keep the interface from drifting. Companion: everruns/markkit#73.
## Before / After
Binary behavior unchanged. New `tests/host.rs` proves parity and the
host contract:
- `--help`, `fetch --help`, `mcp --help`, `--version`: `run()` output ==
binary stdout, byte for byte, exit 0
- usage error: same stderr as binary, exit 2
- `fetch` through a host transport: frontmatter markdown + JSON output,
every request (page and agent-discovery probes) went through the host
- each refused flag: exit 1, one-line error, zero transport calls
- broken stdout pipe: exit 0
```
cargo tree -p fetchkit --no-default-features -e normal -i aws-lc-sys
error: package ID specification `aws-lc-sys` did not match any packages
```
## Risk
- Low/Medium
- **Breaking** for `default-features = false` users who rely on HTTPS
through `ReqwestTransport`: enable `reqwest-tls`. CHANGELOG marks it.
- CLI help/errors lose ANSI color on terminals.
### Checklist
- [x] Unit tests are passed (`cargo test --workspace`, clippy `-D
warnings`, fmt, docs `-D warnings`, release build)
- [x] Smoke tests are passed
- [x] Documentation is updated (CHANGELOG)
- [x] Knowledge is up to date and not in conflict (tool-contract,
fetchers, log; OKF check passes)
chaliy
added a commit
that referenced
this pull request
Nov 1, 2026
<!-- claude-projects-attribution: {"github_login":"chaliy"} -->
_Requested by **Михайло** · [project
thread](https://claude.ai/code/project/chan_019YEkBJuZMHJdXrjom86cGX?thread=cmsg_019YEkBJuZMHJdXrjom86cGXFSAmywce8BSJgNQ4YNsrks)_
## What changed
Release v0.3.0: version bump (Cargo, markkit-js, markkit-wasm),
CHANGELOG Unreleased moved under `[0.3.0] - 2026-11-01`, knowledge log
entry.
Ships #73: the `markkit_cli` library (`run(args, &impl CliHost)`) that
bashkit's `markkit` builtin calls.
## Why
Minor bump: `markkit-cli` URL sources moved behind the default
`native-http` feature (breaking for `default-features = false` builds).
## Before / After
No behavior change in this PR beyond the version.
## Risk
- Low. Merging runs `release.yml` (tag, GitHub Release, crates.io
publish of `markkit` then `markkit-cli`).
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Wy7MByPXsW4jHxfLeTfGEc)_
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Михайло · project thread
What changed
Prep so bashkit can ship a
markkitbuiltin with the exact same CLI as the standalone binary.markkit_cli::run(args, &impl CliHost) -> i32; themarkkitbinary is a thin wrapper withNativeHost. The host owns stdout/stderr, stdin, file reads (--file,--rules), URL extraction and MCP stdio.runnever exits the process or touches the host filesystem.Markkit.NativeHostkeeps callingextract_url_document(markkit's SSRF policy). A sandbox fetches through its own egress, setsbase_urlto the final URL and callsextract_document.native-httpfeature (default) carriesmarkkit/http_clientforNativeHost. Without it the CLI library links no reqwest/TLS.--ocr(host Tesseract binary, host model files). It still parses and shows in help; exit 1 with a one-line error before any input is read.mcpis refused by any host that doesn't implementserve_mcp.Error: …/Caused by:layout but no longer append a backtrace whenRUST_BACKTRACEis set (output must not depend on env; embedders must not leak internals). A closed stdout pipe now exits 0 instead of panicking inprint!.Why
Design: markkit + fetchkit "preinstalled" in bashkit sessions with an identical CLI. Sharing the code (not copying clap structs) keeps the interface from drifting. Companion: everruns/fetchkit#189.
Before / After
Binary output unchanged apart from the backtrace/broken-pipe notes above. New
tests/host.rs:--help,extract --help,mcp --help,--version:run()stdout == binary stdout, byte for byte--layoutwithout--json): same stderr as binary, exit 2extractwith no source): same stderr as binary, exit 1--stdin,-),--file+--rulesread from an in-memory host fs--ocr: exit 1, no fetch;mcprefused by default host; broken pipe exits 0just pre-pr: fmt, clippy, test, check-okf, script tests, examples pass locally.cargo denyis not installed here, so CI runs it (no new dependencies).Risk
markkit-cliwithdefault-features = falsemust enablenative-httpfor URL sources (CHANGELOG).