fix: wire the documented HOTDATA_* env vars into the destination

[eddietejeda]

Fixes #95.

The README documents plain HOTDATA_* env vars for every destination setting, but only HOTDATA_API_KEY was actually bridged — the rest were read solely by the diagnostic CLI's from_env(). In a pipeline, HOTDATA_DATABASE_ID et al. silently did nothing, including the pin-the-database advice the first-run log message itself gives.

Fix: parsing moves into a shared config.plain_env_overrides(); the factory applies it for each keyword not passed explicitly, mirroring the existing HOTDATA_API_KEY bridge. Precedence (now also stated in the README): explicit hotdata(...) param > plain HOTDATA_* > DESTINATION__HOTDATA__* / config.toml > default. The diagnostic CLI's retry defaults are aligned with the destination's (8 / 1.5s, were 5 / 1.0s), and from_env() is rebuilt on the shared helper so the two paths can't drift again.

Note: the issue suggested keeping DESTINATION__HOTDATA__* precedence over the plain names; this PR gives the plain names precedence instead, because that's the behavior the HOTDATA_API_KEY bridge has always had and two precedence rules in one table would be worse than one.

Tests: 5 new cases in test_factory.py (env reaches config — the issue's repro; param > plain; plain > dlt-style; dlt-style still works alone; defaults when unset). Full suite: 284 passed, 1 skipped. Ruff clean on changed files (the 7 pre-existing merge_demo.py E501s are untouched).

[claude]

Review

Blocking Issues

• src/hotdata_dlt_destination/config.py:52-60 — a set-but-empty HOTDATA_MAX_RETRIES, HOTDATA_RETRY_BACKOFF_SECONDS, or HOTDATA_CREATE_DATABASE_IF_MISSING is no longer harmless. The first two raise ValueError out of the hotdata(...) constructor, before any load runs. The third resolves to False and silently disables database creation. These three checks use is not None; the string loop above them uses truthiness. See the inline comment for the fix.

Action Required

1. Treat a blank value as unset for the three non-string settings in plain_env_overrides().
2. Add a test that sets those three names to the empty string and asserts the defaults survive.

Nit (not blocking)

.env.example:12-13 still ships HOTDATA_MAX_RETRIES=5 and HOTDATA_RETRY_BACKOFF_SECONDS=1.0. docs/runbook.md:17 tells the reader to copy that file to .env. Those two names now reach the destination, so a copied template lowers the retry budget to 5 x 1.0s. configuration.py:50-54 states the budget must outlast a catalog-level 409 hold lasting tens of seconds. Update both lines to 8 and 1.5.

Context disclosure

CI was queued at review time, so no check result is available here.

[claude]

A set-but-empty HOTDATA_MAX_RETRIES now aborts every hotdata(...) construction.

_parse_max_retries("") raises ValueError: HOTDATA_MAX_RETRIES must be an integer, got ''. factory.py:132 calls plain_env_overrides() inside __init__, so the error escapes the destination constructor and kills the pipeline before any load. Before this PR the destination never read this name, so a blank value was harmless.

HOTDATA_RETRY_BACKOFF_SECONDS= fails the same way. HOTDATA_CREATE_DATABASE_IF_MISSING= resolves to False and silently disables database creation.

The string loop above already uses truthiness, so a blank HOTDATA_SCHEMA correctly falls back. Blank values are a convention in this repo: .env.example:4 ships HOTDATA_DATABASE_ID=.

Fix: treat blank as unset for these three settings too.

Suggested change

	flag = os.environ.get("HOTDATA_CREATE_DATABASE_IF_MISSING")
	if flag is not None:
	overrides["create_database_if_missing"] = flag.lower() in {"1", "true", "yes"}
	retries = os.environ.get("HOTDATA_MAX_RETRIES")
	if retries is not None:
	overrides["max_retries"] = _parse_max_retries(retries)
	backoff = os.environ.get("HOTDATA_RETRY_BACKOFF_SECONDS")
	if backoff is not None:
	overrides["retry_backoff_seconds"] = _parse_backoff(backoff)
	flag = os.environ.get("HOTDATA_CREATE_DATABASE_IF_MISSING")
	if flag:
	overrides["create_database_if_missing"] = flag.lower() in {"1", "true", "yes"}
	retries = os.environ.get("HOTDATA_MAX_RETRIES")
	if retries:
	overrides["max_retries"] = _parse_max_retries(retries)
	backoff = os.environ.get("HOTDATA_RETRY_BACKOFF_SECONDS")
	if backoff:
	overrides["retry_backoff_seconds"] = _parse_backoff(backoff)

Please also add a test that sets these three names to the empty string and asserts the defaults survive.
true

[eddietejeda]

Fixed in 9b0f728 — the three non-string settings now use truthiness like the string loop, with a comment naming the .env.example convention; added test_blank_plain_env_vars_are_treated_as_unset covering all three. Also took the nit: .env.example retry values now match the destination defaults (8 / 1.5).

[claude]

nit: blank the illustrative values in .env.example, or comment them out. (not blocking)

docs/runbook.md:19 tells a user to copy .env.example to .env for a shell loader. After this PR that copy declares customers and orders on every pipeline, and initialize_storage creates a managed table per declared name. The remaining sample values match the destination defaults, but now outrank config.toml, so a config.toml schema is silently replaced by public.

[claude]

nit: add a test for HotdataDestinationConfig.from_env(). (not blocking)

from_env() now splats plain_env_overrides() into the constructor. The override keys name HotdataClientConfiguration fields, and HotdataDestinationConfig is a separate dataclass with its own field set. Adding a plain env var for a destination-only setting — max_state_files, say — makes from_env() raise TypeError and breaks the hotdata-dlt-destination console script. No test file covers config.py or cli.py today.

[claude]

super nit: move ### Changed above ### Fixed. (not blocking)

The style note at CHANGELOG.md:20 gives the group order as Added / Changed / Fixed / Removed.

[claude]

Prior blocking issue is resolved: blank HOTDATA_MAX_RETRIES, HOTDATA_RETRY_BACKOFF_SECONDS and HOTDATA_CREATE_DATABASE_IF_MISSING now fall back to the defaults, and test_blank_plain_env_vars_are_treated_as_unset covers all three. Three non-blocking comments left inline. CI "CI / Test (Python 3.12)" had not reported at review time.