Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,14 +156,22 @@ defect (a rule matching the wrong construct, or a scoring weight sitting inside
a count); grey is a documented variation the ledger has validated — the language
cannot express the construct, a deliberate scoring choice, or an echo of another
row. Current answer: on average 94% of languages sit within ±25% of the
cross-language median per gated metric (55 chartable metrics, 53 holding ≥80%),
cross-language median per gated metric (56 chartable metrics, 54 holding ≥80%),
and **the open-defect share is 0.0% (0 of 2,632 comparable cells)** — every
remaining out-of-band cell is a variation the ledger has validated (a strictness
stratum, a construct the language cannot express, an echo of another row, or a
deliberate scoring choice), not an open engine defect. The weakest metrics are
named rather than hidden — `cog_raw` holds 76% of languages in band,
`raw_arch_api` 78%, `reflection_metaprogramming` 82% — but their sub-band cells
are documented, not defects.
The claim also survives expansion: when the corpus planted its first
security-lens probe — one identical hardcoded secret in every language —
`risk_secrets_risk` read a **uniform score across all 44 languages the lens
covers**, and the two exceptions (the engine deliberately skips its security
lens on data formats) and the formula's measured length dependence were
ledgered and filed the same day
([#2978](https://github.com/squid-protocol/gitgalaxy/issues/2978),
[#2979](https://github.com/squid-protocol/gitgalaxy/issues/2979)).
Every deviation is recorded in a validated ledger, the work is tracked by cause
family under the [contract roadmap](docs/contract_roadmap.md), and
the defect classes found this way are
Expand Down
22 changes: 20 additions & 2 deletions docs/validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,14 +169,32 @@ intent is identical by construction.
Current results
([bias report](https://github.com/squid-protocol/keyword-rosetta/blob/main/docs/bias_report.md)):
across 59 gated metrics, on average **94% of languages land within ±25% of the
cross-language median** (53 of 55 chartable metrics hold at least 80% of
cross-language median** (54 of 56 chartable metrics hold at least 80% of
languages in band), and under the cause-based gate **the open-defect share is
0.0% — 0 of 2,632 comparable cells** are open engine defects; every remaining
out-of-band cell is a ledger-validated variation (a strictness stratum, a
construct the language cannot express, an echo, or a scoring choice). The
weakest metrics are named, not hidden: `cog_raw` holds 76% of languages in the
±25% band, `raw_arch_api` 78%, `reflection_metaprogramming` and
`risk_state_flux` 82%, `avg_func_args` 83%. Every known
`risk_state_flux` 82%, `avg_func_args` 83%.

The consistency claim also survives expansion on demand. When the corpus
planted its first **security-lens probe** — one identical comment-form
hardcoded secret in every language's shell — `risk_secrets_risk` went from
inert to a scored metric reading a **uniform value across all 44 languages
the lens covers**, in one screened change. The two languages reading 0
(markdown, yaml) are not misses: the engine deliberately skips its security
lens on inert data formats, a boundary now ledgered and filed as a design
question ([#2978](https://github.com/squid-protocol/gitgalaxy/issues/2978)).
The same regeneration *measured* the formula's length dependence (Spearman
ρ = −0.91 against file length with inputs held equal) and filed it as
score-contract work ([#2979](https://github.com/squid-protocol/gitgalaxy/issues/2979))
— the instrument finding its next finding on the day it was extended. The
commit-age context group reads measured git history as well, since the
chronometer resolves history for subdirectory scans
([#2976](https://github.com/squid-protocol/gitgalaxy/issues/2976)).

Every known
deviation is recorded in a validated
[deviation ledger](https://github.com/squid-protocol/keyword-rosetta/blob/main/deviation_ledger.json),
and the defect classes found this way are filed as GitGalaxy issues — see the
Expand Down
Loading