fix(serialize): a stacked entity is described as of its newest filing - #89
Merged
Merged
Conversation
An entity's node is shared by every filing it made, but some of its properties describe it as of each filing: a filer's category, name or exchange can change between them. Stacking a filer's 10-Ks and 10-Qs across such a change was refused as a conflict. merge_graph_tables now keeps the entity row from the part with the newest report (by filing date, then period end, then stacking order), whatever order the filings are stacked in. Each filing's own value stays in its dei facts. Any other table's conflict is still an error.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacking a filer's 10-Ks and 10-Qs across a change in how it describes itself was refused. An entity's node is shared by every filing it made, since its id comes from the CIK, but some of its properties describe the filer as of each filing. When a filer's category moves from "Accelerated Filer" to "Non-accelerated Filer" between filings,
merge_graph_tablessaw one id with two values and raised. Name, ticker or exchange can change the same way.The fix: an entity is described as of the newest filing in the stack, whatever order the filings are stacked in. Each filing's own value is untouched in its
deifacts, so the history stays queryable.Changes
serialize/lpg.py,merge_graph_tables:serialize/README.md: "Stacking filings" lists both exceptions.For comparison, the platform's DuckDB staging keeps an arbitrary row per id within a batch and the first one staged across batches, so it doesn't pick a description deliberately.
Output Impact
BROADER COVERAGE: stacks that were refused on an entity conflict now build, and describe the entity as of its newest filing. A stack that already built is unchanged.
Testing
just test-all: 668 passed, 2 skipped; ruff check, ruff format --check and basedpyright clean.tests/test_lpg.py::TestMerge::test_an_entity_is_described_as_of_the_newest_filing: the newest description wins in both stacking orders, the entity is linked to both reports, and the inputs are untouched. It fails againstmain'slpg.py.test_an_id_with_two_meanings_is_refusednow uses aUnitconflict, since an Entity difference is legitimate..lbdb.dei:EntityFilerCategoryfacts still read three and seven for the two categories.