Skip to content

fix(markdown): recognize ~~~ fenced code blocks, not just backticks#2022

Open
kimdzhekhon wants to merge 1 commit into
Graphify-Labs:v8from
kimdzhekhon:fix/markdown-tilde-fence
Open

fix(markdown): recognize ~~~ fenced code blocks, not just backticks#2022
kimdzhekhon wants to merge 1 commit into
Graphify-Labs:v8from
kimdzhekhon:fix/markdown-tilde-fence

Conversation

@kimdzhekhon

Copy link
Copy Markdown

Summary

Found via code review, not tied to an existing issue.

  • The markdown extractor's fence-toggle check only matched lines starting with ```, so a ~~~-fenced block (CommonMark's other valid fence syntax) fell through as regular prose.
  • Any # ...-shaped line inside a ~~~ block became a spurious heading node, and any markdown link inside it became a spurious document-reference edge.
  • Extended the existing simple toggle (no fence-char/length tracking — matches the existing implementation's level of rigor, not full CommonMark fence-matching) to also recognize ~~~.

Test plan

  • Added test_markdown_tilde_fenced_heading_not_parsed and test_markdown_tilde_fenced_link_not_parsed in tests/test_languages.py, mirroring the existing backtick-fence regression tests
  • uv run pytest tests/test_languages.py -q -k markdown — 12 passed
  • uv run pytest tests/ -q — full suite: 3414 passed (4 pre-existing failures unrelated to this change, missing optional openai dependency in this environment)

The fence-toggle check only matched lines starting with ``` , so a
~~~-fenced block (CommonMark's other valid fence syntax) was parsed as
regular prose: any "# ..." line inside it became a spurious heading
node, and any markdown link inside it became a spurious document
reference edge.

Extend the same simple toggle (no fence-length/char tracking, matching
the existing implementation's level of rigor) to also match ~~~.

Found via codex-assisted code review, verified against the actual code
before delegating the fix.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant