Conversation
chore: back-merge main into develop
A template could see only what its own mock declared, so the things every mock
wants -- which proxy answered, which version answered, who called -- had to be
extracted by hand in each one, and two of them could not be extracted at all.
Nine system variables now, bound into every render: `proxy_name`, `mock_name`,
`doppel_version`, `request_id`, `method`, `path`, `host`, `peer_ip`, `real_ip`.
`real_ip` is the chain a proxy in front writes: `X-Real-IP`, then the *leftmost*
`X-Forwarded-For` entry -- the original client rather than the hop next to us,
across field lines, because a chain split over several is legal and some proxies
emit it -- then the connection's own address, so the variable always has a value
and no template needs `| default('')`. `peer_ip` is that address on its own, and
the pair is deliberate: one is a claim anybody can make and the other is a fact,
and a mock that decides something needs to know which it is reading.
They are reserved. Bound *after* the operator's extractions, so a mock that
extracts into `proxy_name` finds Doppel's value rather than its own -- the
alternative is a template whose meaning depends on which mock rendered it. The
extraction still costs a header read, so `startup_advisories` names the mock and
the names it shadowed. The list of names lives in `doppel-core`, which is the
crate both the binding and the validation can see.
`server.external_url` may now be a template over them, rendered per request:
`http://{{ host }}/` answers each client with the address it asked for, and
`https://{{ proxy_name }}.gw.example.com/` gives each proxy its own name behind a
wildcard. Opt-in for a reason -- `Host` is the caller's claim, so a deployment
that builds a redirect out of it is choosing to, and the documentation says so in
a warning rather than a footnote. A template that fails to render, or renders to
something that is not a url, means no rewriting for that request rather than a
failed one: a cosmetic improvement to a `Location` is not worth a 500.
Every variable name in `main.example.yaml` and the documentation is `snake_case`
now, matching the system ones. Jinja accepts either, and an operator's own names
are untouched -- but two conventions in one context read as two sources.
The reference configuration also stopped extracting `X-Request-ID` into
`request_id`: it is a system variable, always present and minted when the client
sends none, so the extraction was exactly the wasted work the new advisory warns
about. `mock4` keeps the strict-rendering lesson on `X-Trace-Id`, which is
genuinely the operator's, and renders `proxy_name` and `doppel_version` beside it.
Seventeen unit tests, three on the templated address, one end-to-end through the
binary over a real socket -- the `real_ip` chain needs a peer to fall back to, and
only a connection has one. Mutation-checked twice: nothing binding the system
variables fails the end-to-end test with `undefined variable proxy_name`, and
ignoring `X-Real-IP` fails the chain test.
feat: bind Doppel's own variables into every template
A DSN carries the key that authorises sending events, which makes it the one
Sentry setting that is a credential -- and until now the only way to give Doppel
one was to write it into a document the admin API returns and the store keeps.
`DOPPEL_SENTRY_DSN` provides it instead, or replaces what `sentry.dsn` says.
The environment wins, the same way `DOPPEL_ADMIN_TOKENS` wins over `admin.tokens`:
that is how a deployment overrides a document it may not be able to edit. Which
source won goes in the startup line beside the redacted DSN, because "sentry
reporting enabled" does not answer the question an operator has when it is the
wrong DSN.
An empty variable counts as unset and leaves `sentry.dsn` in force. Deliberately
that direction: `DOPPEL_SENTRY_DSN=${SENTRY_DSN}` with nothing behind `SENTRY_DSN`
is a compose file that means nothing by it, and silently switching error reporting
off is the worse reading of it. Turning it off is still `dsn: ""`, or no section.
Doppel reads its own name rather than the conventional `SENTRY_DSN`: a variable
sitting in the environment for the service beside this one should not make this one
start reporting to it.
Read at startup and not merged into `Config`, like the other two environment
values -- the revision is a hash of the document, so folding the environment into
it would make two instances reading one stored document disagree about the
revision. Validation stays where it was, at the client: the crate that knows what
a DSN must parse as is also the one that can redact it in the error.
Five unit tests on the precedence and two through the built binary, which is the
only place the variable is actually read from a process. Mutation-checked by
ignoring the environment: three tests fail, including the end-to-end one.
feat: take the Sentry DSN from the environment
…ds report `DOCKERHUB.md` knew about `DOPPEL_ADMIN_TOKENS` and not about `DOPPEL_SENTRY_DSN`, which is the file's own failure mode: it is published somewhere this repository cannot see, and nothing fails when it falls behind. Found by the step added to `pre-release-check` for exactly that, on its first release. The section is about both credentials now, with the empty-variable rule stated -- a variable that failed to interpolate leaves a configured DSN in force rather than silently turning error reporting off. Writing that turned up a second thing, and this one was nearly a false claim I made myself: nothing that ships enables the `sentry` feature. Not the image, not the released binaries. A DSN given to either warns at startup and is not reported, so a reader following the new section would have set a variable that does nothing. Both pages say so now, next to the flag that changes it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Doppel 1.1.0. Templates can see what Doppel knows, and the two credentials can
come from the environment.
System variables
Nine, bound into every template Doppel renders, whether or not the mock asked for
anything:
proxy_name,mock_name,doppel_version,request_id,method,path,host,peer_ip,real_ip. Before this, the things every mock wants --which proxy answered, which version, who called -- had to be extracted by hand in
each one, and two of them could not be extracted at all.
real_ipis the chain a proxy in front writes:X-Real-IP, then the leftmostX-Forwarded-Forentry, then the connection's own address, so it always has avalue.
peer_ipbeside it is deliberate: one is a claim anybody can make and theother is a fact.
They are reserved -- bound after the operator's extractions, so a template always
means what the documentation says -- and a mock that extracts into one of those
names is called out at startup, because the extraction still costs a header read
and its result is then thrown away.
An address that can depend on the request
server.external_urlmay be a template over those variables:Opt-in for a reason, and the documentation says so in a warning:
Hostarrivesfrom the client, so a deployment that builds a redirect out of it is choosing to
let a caller decide where its own redirects point. A template that renders to
something unusable means no rewriting for that request rather than a failed one.
Credentials from the environment
DOPPEL_SENTRY_DSNprovides the Sentry DSN or overridessentry.dsn, the wayDOPPEL_ADMIN_TOKENSalready does for tokens: a DSN carries the key thatauthorises sending events, and a deployment that keeps credentials in the
environment should not have to write one into a document the admin API returns.
Startup logs which source won. An empty variable counts as unset, deliberately --
a failed interpolation leaves a configured DSN in force rather than silently
switching error reporting off.
Not breaking
Variable names in
main.example.yamland the documentation aresnake_casenow,and an operator's own names are untouched: Jinja accepts either. Nothing else
changed shape, which is why this is a minor bump.
Verification
make gateclean with the PostgreSQL suites included -- 817 Rust tests, 127 jest,eslint, both TypeScript projects, the size budgets, a strict docs build, the
schema, the parameter reference, the documentation links and the licence check --
and
make e2eclean, 81 Playwright tests.release_notes.py 1.1.0composes a bodyfrom the section this branch writes.
The pre-release pass found one thing, fixed in
52a008ebefore the release commit:DOCKERHUB.mdhad fallen behind on the environment variables, and writing thatsection turned up a claim of mine that was nearly false -- nothing that ships
enables the
sentryfeature, so a reader following it would have set a variablethat does nothing. Both pages say so now.
After the merge