Skip to content

fix(serialize): a stacked entity is described as of its newest filing - #89

Merged
jfrench9 merged 1 commit into
mainfrom
bugfix/stack-entity-newest
Sep 30, 2026
Merged

jfrench9 merged 1 commit into
mainfrom
bugfix/stack-entity-newest

Conversation

@jfrench9

Copy link
Copy Markdown
Member

Summary

Stacking a filer's 10-Ks and 10-Qs across a change in how it describes itself was refused. An entity's node is shared by every filing it made, since its id comes from the CIK, but some of its properties describe the filer as of each filing. When a filer's category moves from "Accelerated Filer" to "Non-accelerated Filer" between filings, merge_graph_tables saw one id with two values and raised. Name, ticker or exchange can change the same way.

The fix: an entity is described as of the newest filing in the stack, whatever order the filings are stacked in. Each filing's own value is untouched in its dei facts, so the history stays queryable.

Changes

  • serialize/lpg.py, merge_graph_tables:
    • each part is ranked by its newest report: filing date, then period end, then stacking order;
    • on an Entity conflict, the row from the higher-ranked part wins;
    • a conflict in any other table is still an error, apart from the existing dimensional-role exception for elements;
    • the input tables aren't modified.
  • serialize/README.md: "Stacking filings" lists both exceptions.

For comparison, the platform's DuckDB staging keeps an arbitrary row per id within a batch and the first one staged across batches, so it doesn't pick a description deliberately.

Output Impact

BROADER COVERAGE: stacks that were refused on an entity conflict now build, and describe the entity as of its newest filing. A stack that already built is unchanged.

Testing

  • just test-all: 668 passed, 2 skipped; ruff check, ruff format --check and basedpyright clean.
  • New tests/test_lpg.py::TestMerge::test_an_entity_is_described_as_of_the_newest_filing: the newest description wins in both stacking orders, the entity is linked to both reports, and the inputs are untouched. It fails against main's lpg.py.
  • test_an_id_with_two_meanings_is_refused now uses a Unit conflict, since an Entity difference is legitimate.
  • End to end: a small-cap filer's last ten 10-K and 10-Q filings, loaded from the CDN and stacked into a .lbdb.
    • Before, the merge was refused, with category the only differing field.
    • Now there's one entity with the newest category, linked to all ten reports.
    • The ten filings' dei:EntityFilerCategory facts still read three and seven for the two categories.

An entity's node is shared by every filing it made, but some of its
properties describe it as of each filing: a filer's category, name or exchange
can change between them. Stacking a filer's 10-Ks and 10-Qs across such a
change was refused as a conflict.

merge_graph_tables now keeps the entity row from the part with the newest
report (by filing date, then period end, then stacking order), whatever order
the filings are stacked in. Each filing's own value stays in its dei facts.
Any other table's conflict is still an error.
@jfrench9
jfrench9 merged commit 3b3a622 into main Sep 30, 2026
4 checks passed
@jfrench9
jfrench9 deleted the bugfix/stack-entity-newest branch September 30, 2026 07:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant