feat(heap-dump): add --redact, --compress, and --open flags by parttimenerd · Pull Request #56 · SAP/cf-cli-java-plugin

parttimenerd · 2026-09-17T14:00:31Z

Summary

--redact: zeros primitive arrays (byte[], char[], etc.) in the heap dump before saving (lean redaction mode), removing sensitive data like passwords and tokens. Uses the bundled hprof-redact binary.
--redact-complete: zeros all primitive arrays and individual primitive fields (complete redaction, maximum privacy). Mutually exclusive with --redact.
--compress: saves the dump as .hprof.gz by transferring it gzip-compressed over SSH (requires JDK 17+ on the container; falls back to uncompressed otherwise). On JDK 17+ containers the plugin also transparently uses jmap gz=1 to reduce transfer size without --compress, decompressing on the fly.
--open: after downloading (and optionally redacting/compressing), spins up a temporary single-serve local HTTP server on 127.0.0.1 and opens hprof-analyzer in the browser with ?file=http://localhost:PORT/TOKEN.hprof. The server shuts down automatically after the file is fetched once.
--open-url <URL>: override the hprof-analyzer base URL (e.g. a locally running instance). Implies --open.
Windows ARM64 support added to CI and release builds.

Security notes (`--open`)

File is served under a random 16-hex-char token URL (e.g. /a3f9c2b1d4e56f78.hprof) — the real filename is never exposed
Server bound to 127.0.0.1 only (loopback, not reachable from the network)
Uses io.Copy directly (not http.ServeFile) to prevent path traversal
All other paths return 404; CORS OPTIONS preflight is handled without triggering server shutdown
macOS: the Application Firewall will prompt "Do you want the application 'cf-cli-java-plugin' to accept incoming network connections?" — click Allow (loopback only, safe)
Firefox: may show a dialog asking to allow the HTTPS page to access local services — click Allow

- Embeds hprof-redact binaries (linux/amd64, linux/arm64, darwin/arm64, windows/amd64) via //go:embed; extracted to ~/.cache/cf-java-plugin/ on first use (SHA8-keyed, reused on subsequent runs). - --compress: passes gz=1 to jmap (JDK 17+) to compress the dump on the remote container before transfer, saving bandwidth. jvmmon path falls back to post-creation gzip on the container. Output is .hprof.gz. - --redact: pipes the downloaded dump through hprof-redact (lean mode: zeros primitive arrays only). Output is -redacted.hprof or -redacted.hprof.gz when --compress is also set. - --redact-complete: like --redact but zeros all primitive values (instance scalar fields + CLASS_DUMP statics) via two-pass redaction. - --compress and --redact can be combined: compress on remote to reduce transfer, redact locally (hprof-redact reads .hprof.gz natively). - Adds FindHeapDumpGzFile to utils for locating *.hprof.gz on the container (jvmmon+compress path). - Extracts osWindows and cmdHeapDump constants to satisfy goconst.

- hprof-analyzer release.yml: add aarch64-pc-windows-msvc target on windows-11-arm runner; compile-all includes GOOS=windows GOARCH=arm64 - redact.go: embed dist/hprof-redact-windows-arm64.exe (stub until v0.3.1 ships); add windows/arm64 case; guard against 0-byte stubs - cfutils.go: add CopyOverCatGunzip (io.Pipe + compress/gzip for transparent streaming decompression) and ProbeRemoteFileGzip (checks gzip magic bytes 1f8b via SSH) - cf_cli_java_plugin.go: heap-dump SSHCommand now uses shell-level gz probe (jmap -h | grep gz) so jmap auto-uses gz=1 on JDK 17+; Go post-command probes the remote file magic bytes to decide: - remote gz + no --compress → CopyOverCatGunzip, save as .hprof - remote gz + --compress → CopyOverCat, save as .hprof.gz - not gz + --compress → warn JDK 17+ required, save uncompressed Removes @JMAP_GZ Go expansion (replaced by shell probe); keeps @COMPRESS_FLAG for jvmmon path

- appInstanceIndexSet: simonleung8/flags IsSet() returns true for any registered flag whose non-zero default was set at init time, making it impossible to distinguish user-provided from default. Compare against known default (-1) instead. - CheckRequiredTools wrapped in !options.DryRun guard so dry-run works without CF login or SSH access. - var err declaration hoisted before GenerateFiles block (needed after CheckRequiredTools was scoped inside the if block).

…-dump - Feature list in intro - Examples: --redact, --redact-complete, --compress, combined usage - New "Heap Dump Privacy" subsection: redaction modes table, what is preserved, file naming, supported platforms - New "Compressed Transfer" subsection: JDK 17+ requirement, fallback behaviour, transparent gz transfer without --compress - CHANGELOG [Unreleased]: all four new behaviours

- Use context.WithTimeout(context.Background(), 5s) for srv.Shutdown instead of the request context (which may already be canceled); suppress contextcheck lint with explanation since this is intentional - Add empty title arg to Windows `cmd /c start` to avoid URL being interpreted as the window title - Replace non-blocking done-channel check in test with a 2s timeout select to avoid a false-pass race condition

…-open

…traversal

…eakage and restrict access to single file

…en --dry-run

… in serveFileOnce

…s only affects local file format

golangci-lint typecheck fails when the dist/ hprof-redact binaries are missing (go:embed pattern not satisfied). Add a download step mirroring build.py, placed after the existing jstall download and before lint.

The previous attempt tried to curl the binaries directly, but they are packed inside tar.gz/zip archives. Add --deps-only flag to build.py to run only the download+extraction step (no go build), and use it in the PR validation workflow before golangci-lint.

- redact: delete partial output file on failure instead of leaving it behind; add --redact-keep-on-error flag to preserve it when needed - open: use sync.Once to guard close(doneCh)+srv.Shutdown so concurrent GETs cannot panic via double-close - open: print error and manual-open URL when browser launch fails

serveFileOnce: - add timeout parameter (10 min in production) — CLI no longer hangs forever if the browser never fetches the dump - add WriteTimeout (30s) to prevent stalled clients holding a goroutine - print stderr message when timeout expires so user knows why CLI exited tests: - TestServeFileOnce_TimeoutClosesServer: done closes, server stops after timeout - TestServeFileOnce_ConcurrentGETsNoPanic: regression for sync.Once fix - TestPipeHeapDumpThroughRedact_ErrorDeletesPartial: partial file removed on failure - TestPipeHeapDumpThroughRedact_KeepOnError: partial file kept with keepOnError=true - TestPipeHeapDumpThroughRedact_HappyPath: success deletes source, returns output path

dbriemann · 2026-09-21T12:38:42Z

+	wait := func() error {
+		return cat.Wait()
+	}
+
+	go func() {
+		_ = pw.CloseWithError(wait())
+	}()
+
+	return pr, wait, nil


Wait() will be called twice, which will produce an error. Only one goroutine should call cat.Wait().

I didn't know

Fixed: used sync.Once to cache the cat.Wait() result, making the closure idempotent. The goroutine and the caller both see the correct exit error regardless of which runs first.

The goroutine that closes pw already calls wait() (which calls cat.Wait()). The caller in cf_cli_java_plugin.go then calls waitRemote() — the same closure — a second time. The second cat.Wait() returns an error because the process was already reaped. Fix: cache the result with sync.Once so wait() is idempotent. Both the goroutine and the caller see the correct exit error regardless of order.

deps: - golang.org/x/crypto v0.52.0 → v0.56.0 (fixes GO-2026-6303/6354/6355) - golang.org/x/text v0.37.0 → v0.41.0 (fixes GO-2026-5970) - golang.org/x/sys/term bumped transitively ci (pr-validation.yml): - add explicit `go test -v -race ./...` step to the validate job - split off a `build` job (matrix: ubuntu/macos/windows) that runs after validate passes; each OS runs `go test -race`, builds the plugin with build.py, and uploads artifacts so colleagues can download and test the binaries directly from the Actions run page

…emove - Use .bat scripts on Windows in test helpers (makeFakeBin, makeCopyBin, makeWriteAndFailBin) so test binaries are executable without needing shell support; use osWindows constant instead of string literal - Close *os.File before os.Remove in pipeHeapDumpThroughRedact pre-check to avoid Windows file-locking errors ("being used by another process") - Strip trailing \r\n in HappyPath test to account for Windows `more` appending CRLF

… corrupting gzip data in tests

dbriemann

lgtm

jstall never had a --ssh-prefix flag — the code was passing an unknown option that caused "Unknown option: --ssh-prefix" for every status/jstall invocation. Fix: - Use --cf <appName> (jstall's built-in CF shortcut) for the normal case - Fall back to --ssh with the full cf-ssh command only when --app-instance-index is needed (jstall's --cf doesn't support instance selection) - Remove the --ssh-prefix argument entirely - Update embedded jstall-minimal.jar from 0.6.0 to 0.7.1 (latest) Also add missing Description fields and fix empty ShortName prefix in generateOptionsMapFromFlags so --compress/--open/--redact show proper help text instead of "-,".

- Replace --cf with --ssh "cf ssh <app> -c" so jstall doesn't internally wrap the command in `sh -c`, which fails on Windows (no sh.exe) - Fix continuation-line indentation in OPTIONS help: compute miscLineIndent so all flags align at the same column regardless of ShortName length

…ng error) jstall v0.7.2 fixed --cf to use ProcessBuilder instead of sh -c, so it now works on Windows. Switch back to --cf for the normal case (cleaner, no quoting issues). Remove shellQuote which is no longer needed. For --app-instance-index, fall back to --ssh without shell quoting since jstall also uses ProcessBuilder for --ssh in v0.7.2.

parttimenerd added 27 commits September 9, 2026 15:16

docs: design spec for --open flag on heap-dump

1020396

docs: implementation plan for --open heap-dump flag

a40dc1c

feat: add serveFileOnce and openBrowser helpers for --open

8c3b131

feat: add --open and --open-url flags to Options and flagDefinitions

82d626d

feat: invoke serveFileOnce+openBrowser after heap-dump download for -…

101d658

…-open

fix: stream file directly via io.Copy to prevent http.ServeFile path …

964e2ea

…traversal

fix: use random token URL path in serveFileOnce to prevent filename l…

a79a3da

…eakage and restrict access to single file

fix: print Would-open line before dry-run SSH command output for --op…

43d2260

…en --dry-run

fix: handle CORS OPTIONS preflight without triggering server shutdown…

22fc173

… in serveFileOnce

docs: document --open and --open-url flags in README and CHANGELOG

91d8a55

docs: document macOS firewall permission dialog for --open flag

b46bb1f

docs: add .tool.yaml with --open, --redact, --compress how-to entries

0791be0

chore: exclude docs/superpowers planning artifacts from repo

812fe9b

docs: fix supported platforms list for --redact in CHANGELOG

7f9847b

Merge branch 'master' into heap-dump-compress-redact

c8202bf

docs: clarify that gzip transfer is always used on JDK 17+, --compres…

160e510

…s only affects local file format

ci: download hprof-redact binaries before golangci-lint in PR validation

189fe24

golangci-lint typecheck fails when the dist/ hprof-redact binaries are missing (go:embed pattern not satisfied). Add a download step mirroring build.py, placed after the existing jstall download and before lint.

style: add trailing newline to .gitignore

fa8c71c

style: fix markdownlint MD013/MD060 violations in README

332018e

style: fix MD060 aligned table style in README redact section

b99e31e

style: fix MD060 table column alignment accounting for multi-byte chars

6dcbc21

parttimenerd requested a review from dbriemann September 18, 2026 10:16

dbriemann reviewed Sep 18, 2026

View reviewed changes

Comment thread cf_cli_java_plugin.go Outdated

Comment thread open.go Outdated

ansteiner requested changes Sep 18, 2026

View reviewed changes

Comment thread open.go Outdated

parttimenerd added 2 commits September 18, 2026 16:20

ansteiner previously approved these changes Sep 18, 2026

View reviewed changes

dbriemann reviewed Sep 21, 2026

View reviewed changes

Comment thread cf_cli_java_plugin.go Outdated

Stream heap dumps through hprof-redact

fbe08ec

parttimenerd dismissed ansteiner’s stale review via fbe08ec September 21, 2026 08:24

parttimenerd added 2 commits September 21, 2026 10:36

Fix changelog heading duplication

b92623b

Improve heap-dump stream error handling

f166766

parttimenerd requested review from ansteiner and dbriemann September 21, 2026 08:46

dbriemann reviewed Sep 21, 2026

View reviewed changes

parttimenerd added 4 commits September 21, 2026 16:22

fix: use PowerShell binary copy on Windows instead of more to avoid…

f1e68ab

… corrupting gzip data in tests

dbriemann previously approved these changes Sep 21, 2026

View reviewed changes

parttimenerd dismissed dbriemann’s stale review via b93362b September 22, 2026 07:43

parttimenerd added 3 commits September 22, 2026 10:00

ci: trigger CI to pick up jstall v0.7.2 and latest hprof-redact nightly

171bb8a

Navigation Menu

feat(heap-dump): add --redact, --compress, and --open flags - #56

feat(heap-dump): add --redact, --compress, and --open flags#56
parttimenerd wants to merge 40 commits into
SAP:masterfrom
parttimenerd:heap-dump-compress-redact

parttimenerd commented Sep 17, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dbriemann Sep 21, 2026

Uh oh!

parttimenerd Sep 21, 2026

Uh oh!

parttimenerd Sep 21, 2026

Uh oh!

dbriemann left a comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

parttimenerd commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Security notes (--open)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dbriemann Sep 21, 2026

Choose a reason for hiding this comment

Uh oh!

parttimenerd Sep 21, 2026

Choose a reason for hiding this comment

Uh oh!

parttimenerd Sep 21, 2026

Choose a reason for hiding this comment

Uh oh!

dbriemann left a comment

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

parttimenerd commented Sep 17, 2026 •

edited

Loading

Security notes (`--open`)