Skip to content

Extended RAG, CVL context - #115

Open
jtoman wants to merge 11 commits into
masterfrom
jtoman/extended-rag
Open

Extended RAG, CVL context#115
jtoman wants to merge 11 commits into
masterfrom
jtoman/extended-rag

Conversation

@jtoman

@jtoman jtoman commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Our attempts to make all CVL knowledge RAG driven has been yielding mixed results. This attempts to condense some of the most common and important information about CVL into "context documents" which are inlined into every CVL agent's task. Explicit cache markers are added to make this avoid prompt blow ups.

The context documents incorporate (and moot) several existing artifacts:

  1. The store-based KB. All of the advice in the KB has now been migrated to the recipes index (with some extra information as well beyond what was in kb_populate)
  2. cvl_guidance.j2 has been expanded and updated and converted to cvl_baseline.md.
  3. "summarization" rag and "invariants & quantifiers" rag attempt to distill the core knowledge behind these key techniques.

In addition, we've tried our hand at CoNsTiTuTiOnAl Ai: we have pretty consistently viewed agents treating "get the spec all verified" as the end goal. The "verification_charter.j2" is our first attempt at making the goal of the agent explicit: verified software is the goal, not a verified spec. The feedback judge's criteria are now AlIgNeD with this constitutional principle; most of the evaluations we ask of it are grounded in this principle.

The feedback prompt has also been expanded to close one common failure mode; a property which ends up passing but skips all of the actually important methods.

@jtoman
jtoman requested a review from shellygr July 31, 2026 17:33
Comment thread composer/kb/resources/recipes/mocks-and-edits.md
Comment thread composer/kb/resources/recipes/r06-immutables.md Outdated
Comment thread composer/kb/resources/recipes/r07-contract-extensions.md Outdated
Comment thread composer/kb/resources/recipes/r10-nativecodesize.md
Comment thread composer/kb/resources/recipes/r11-ecrecover-axioms.md Outdated
Comment thread composer/kb/resources/cvl_summarization_rag_draft.md
Comment thread composer/kb/resources/cvl_summarization_rag_draft.md
Comment thread composer/kb/resources/cvl_summarization_rag_draft.md Outdated
catch-all entry whose receiver the Prover could not pin to the named contract;
entry pattern mismatch (locations on reference-type parameters, exact
signature); an entry naming the inheriting contract instead of the defining
contract for an inherited method (A2.1); the call originating from CVL (A2.5).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we cannot just assume sighash is conveniently always resolved and not to put specific instructions in such cases

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

? I'm not sure what in the text you're referring to here.

Comment thread composer/templates/property_judge_prompt.j2 Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants