Repository navigation
secrets: detect credential fields in structured config (JSON) - #43
Merged
Merged
Conversation
The generic-secret content rule anchors on password/passwd/secret/api_key but match_generic floors the value at 16 chars and the engine gates on >=3.5 bits/byte of entropy, so a short low-entropy credential assigned to a credential-named config field (a device PIN, a base64-wrapped passcode) is a deliberate false-negative. Lowering those floors on the flat byte stream would reintroduce a false-positive storm, because nothing there separates a real passwd field holding a short value from passwd=%s or an empty password. Add a structure-aware pass (scan_config_credentials) that parses the file as JSON first, so the key, the value, and their relationship are exact; the parse is itself a strong gate, since binary, code, and malformed text never enter. It fires only on a curated strong credential key (pin-like: passwd/pin/psk/ wpakey...; token-like: secret/apikey/token... which also require length >= 8 or a decode), rejects empty / placeholder / enum / single-character values, auto-decodes base64/hex and re-judges the decoded form, and defers a long high-entropy value under a key the flat-stream rule already anchors to that rule so a credential is reported once. Findings use category "config-credential"; confidence is Pattern for a raw value and Structural when it decoded; values are redacted. JSON only for now; ini/env/xml are follow-ons behind the same key/value judgment. Adds tests/test_config_cred.py (positives, placeholder/enum/non-JSON negatives, no-double-report, redaction).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
generic-secretcontent rule anchors onpassword/passwd/secret/api_key, butmatch_genericfloors the value at 16 characters and the engineapplies a >=3.5 bits/byte entropy gate. A short, low-entropy credential assigned
to a credential-named field in a config file (a device PIN, a base64-wrapped
passcode) therefore falls through by design. The rule's own comment says as
much: it is precision-first on a flat byte stream, and a short value is the
accepted false-negative.
Lowering those floors is not an option. On a flat stream nothing separates a
real
"passwd":"<value>"frompasswd=%s,oldpasswddir=..., or"password":"", so it would bring back a false-positive storm.Change
A structure-aware pass,
scan_config_credentials(src/configcred.cpp), wiredinto the secrets pass in
walk.cpp. It parses the file as JSON first, so thekey name, the value, and the key/value relationship are exact. The parse is
itself a strong gate: binary, code, and malformed text never enter.
It fires only when:
passwd/pin/psk/wpakey/...; token-like:secret/apikey/token/...,which additionally require length >= 8 or a successful decode),
repeated character,
A long, high-entropy value under a key the flat-stream rule already anchors is
deferred to that rule, so a credential is reported once, not twice. Findings use
a new category
config-credential; confidence is Pattern for a raw value andStructural when the value decoded; values are redacted.
JSON only in this change; ini/env/xml are natural follow-ons behind the same
key/value judgment.
Tests
tests/test_config_cred.pyasserts the positives (base64-encoded passcodes, aWi-Fi PSK, a numeric PIN, an array-inherited key), the negatives that must stay
silent (empty / placeholder / enum / numeric token-key / non-credential key, a
non-JSON document, a plaintext file), no double-report of a high-entropy value,
and value redaction. Full suite green (ctest 15/15). Also validated across a
test set of extracted firmware: the targeted config is caught and config-heavy
images produce no false positives.