Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/DEMO_LIBRARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ shipped types, including Scratchpad, PRD, Test Plan, and Handoff.
| image-resizing | Drop or paste files to upload and embed; duplicate drops reuse files; resize images, wrap prose, Shift+Enter beside, Enter below |
| file-uploads | Drop the same image or file again: one attachment and stored copy, multiple embeds; renamed copies, Dual, and Undo |
| callout-gallery | `> [!SUMMARY]`, `[!NOTE]`, `[!TIP]`, `[!IMPORTANT]`, `[!WARNING]`, and `[!CAUTION]` callouts, a custom label, nested list and code, plain and escaped quotes, and both deck themes; comment inside a callout, or edit one in Editor and check Raw |
| markdown-blocks | `markdown` and `md` fences shown as rendered drafts with a Rendered / Source switch; a discussion on the draft shows in both views; select rendered text across a link, press C, then switch to Source |
| math-gallery | KaTeX math: inline, display, and fenced formulas, math in lists and tables, a six-row slide equation, dollar amounts that stay text, and invalid math shown as source; comment on the text around a formula |

Flowcharts currently support source-backed node and edge comments. Other
Expand Down
43 changes: 43 additions & 0 deletions docs/SOURCE_COMMENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,49 @@ Their source context uses the last successfully tracked revision. A stale
browser selection is rejected; the form retains its draft and asks the
reader to reopen the latest plan before posting.

## Markdown blocks

A `markdown` or `md` fence renders in the reading view as a Markdown block:
the rendered draft, a Rendered / Source switch, and the literal source,
hidden while Rendered shows. Each page load starts on Rendered. A live update
keeps the view chosen on that page, including when the draft is edited.
The existing live-update swap maps the block through the shared text diff
before restoring the viewport.

Text selections in Markdown blocks create ordinary comments with no `anchor_kind`.
Table-cell comments use the existing signed `table_cell` targets, translated
from the draft body into document source offsets. They work inline and in the
expanded table; Source shows the same cell discussion on its literal text.
The hidden source stays in the browser's anchor text model in both views, and
the rendered pane stays out of it. This keeps occurrence counting the same as
for any code block, on the client and in `MarkdownTextExtractor`.

- The server marks each rendered element with the body range it came from
(`data-markdown-from` and `data-markdown-to`, UTF-16 offsets into the
fence body). The block also carries the parser-backed visible text and
complete source spans from `MarkdownTextExtractor`. `coplan/markdown_blocks.js`
matches each text node within its element's range and advances parent
cursors after nested formatting. Link URLs cannot match visible labels.
Entities and escapes map to their whole source spelling. Footnote
numbers and other generated text do not map. Authored HTML without element
positions uses the body range and the same forward-only parser spans.
Authored IDs and source-position attributes are stripped before rendering;
they cannot claim generated coordinates or footnote targets.
- A selection in the rendered draft becomes the source range from its first
to its last mapped character. A selection across `**bold**` text or a link
therefore stores the Markdown source of that passage.
- When Rendered shows, a highlight that falls in the hidden source is drawn
over the rendered characters that came from that range. A comment made in
either view shows in both.
- A comment anchored only to text with no rendered form, such as part of a
URL, is visible in Source only.
- Only a fence the parser produced becomes a block. Authored HTML imitating
one stays ordinary HTML. Fences inside slide decks, editor previews, and
comments stay code blocks. Structural discussions become outdated if an edit
moves their draft into a literal slide fence.
- Writing inspection keeps attribution on canonical source. The Rendered/Source
buttons remain available during inspection so that source attribution is reachable.

## Supported syntax

- Markdown table header and body cells, including repeated values, Unicode,
Expand Down
97 changes: 97 additions & 0 deletions engine/app/assets/stylesheets/coplan/markdown.css
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,103 @@
user-select: none;
}

/* A `markdown` or `md` fence renders as a Markdown block: the same
terminal-style header as a code block, with a Rendered / Source switch,
over either the rendered body or the literal source. The header is real
DOM (it holds buttons), kept out of the anchor text model by
data-markdown-block-chrome. */
.markdown-rendered .markdown-block {
border: var(--code-block-border, 1px solid var(--color-border));
border-radius: var(--code-block-radius, var(--radius));
margin-bottom: var(--space-md);
min-width: 0;
}

.markdown-rendered .markdown-block [hidden] {
display: none !important;
}

.markdown-block__bar {
display: flex;
align-items: center;
justify-content: space-between;
gap: var(--space-sm);
padding: 0.3em var(--space-sm) 0.3em 4.6em;
background-color: var(--code-label-bg, var(--code-header-bg));
background-image: var(--code-label-dots,
radial-gradient(circle at 1.1em 50%, #ff5f57 0.3em, transparent 0.36em),
radial-gradient(circle at 2.2em 50%, #febc2e 0.3em, transparent 0.36em),
radial-gradient(circle at 3.3em 50%, #28c840 0.3em, transparent 0.36em));
border-bottom: var(--code-label-border, 1px solid var(--color-border));
border-radius: var(--code-label-radius, calc(var(--radius) - 1px) calc(var(--radius) - 1px) 0 0);
color: var(--code-header-text);
font-family: var(--code-label-font, var(--font-mono));
font-size: 0.75rem;
line-height: var(--code-label-leading, 1.2);
letter-spacing: var(--code-label-tracking, 0.04em);
user-select: none;
}

.markdown-block__views {
display: inline-flex;
gap: 2px;
padding: 2px;
border: 1px solid var(--color-border);
border-radius: var(--radius);
background: var(--code-block-bg);
}

.markdown-block__view {
padding: 0.2em 0.65em;
border: 0;
border-radius: calc(var(--radius) - 2px);
background: transparent;
color: inherit;
font: inherit;
letter-spacing: inherit;
cursor: pointer;
}

.markdown-block__view:hover {
background: var(--color-interaction-hover-bg);
color: var(--color-text);
}

.markdown-block__view[aria-pressed="true"] {
background: var(--color-surface, var(--color-bg));
color: var(--color-text);
box-shadow: 0 0 0 1px var(--color-border);
}

.markdown-block__view:focus-visible {
outline: 2px solid var(--color-primary);
outline-offset: 1px;
}

.markdown-block__rendered {
padding: var(--space-md) var(--space-lg);
min-width: 0;
overflow-x: auto;
}

.markdown-block__rendered > :first-child {
margin-top: 0;
}

.markdown-block__rendered > :last-child {
margin-bottom: 0;
}

.markdown-rendered .markdown-block > pre {
margin: 0;
border: 0;
border-radius: 0 0 var(--code-block-radius, var(--radius)) var(--code-block-radius, var(--radius));
}

.markdown-rendered .markdown-block > pre[lang]::before {
content: none;
}

.markdown-rendered code {
font-family: var(--font-mono);
font-size: 0.875em;
Expand Down
148 changes: 139 additions & 9 deletions engine/app/helpers/coplan/markdown_helper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ module MarkdownHelper

CALLOUT_TYPES = %w[note tip important warning caution summary].freeze

# Fence languages the reading view shows rendered, with a Source toggle.
MARKDOWN_BLOCK_LANGUAGES = %w[markdown md].freeze

# The only elements and values comrak emits for math. Anything else
# carrying the attribute (authored HTML) loses it during rendering.
MATH_ELEMENTS = %w[span code pre].freeze
Expand Down Expand Up @@ -66,14 +69,14 @@ module MarkdownHelper
# expandable_images: frame each image with the same full-window expander
# diagrams and tables use. Follows data_tables by default, so decks opt
# out of both together.
def render_markdown(content, interactive: true, footnote_prefix: nil, footnotes: :inline, line_offset: 0, data_tables: true, expandable_images: data_tables, source_comments: false, retain_sourcepos: false)
def render_markdown(content, interactive: true, footnote_prefix: nil, footnotes: :inline, line_offset: 0, data_tables: true, expandable_images: data_tables, source_comments: false, retain_sourcepos: false, strip_html_attributes: [])
render_options = { unsafe: true }
# Sourcepos wires checkboxes and structural comment targets to source,
# and tells parser-generated math from authored HTML that imitates it;
# it is stripped after generating trusted interaction metadata.
source = content.to_s.encode("UTF-8")
render_options[:sourcepos] = true if interactive || source.include?(ContentRegions::Iframe::PREFIX) || source.include?("$") || source.include?("math")
html = CoPlan::Markdown.to_html(source, options: { extension: EXTENSION_OPTIONS, render: render_options }, plugins: { syntax_highlighter: nil })
render_options[:sourcepos] = true if interactive || retain_sourcepos || source.include?(ContentRegions::Iframe::PREFIX) || source.include?("$") || source.include?("math")
html = CoPlan::Markdown.to_html(source, strip_html_attributes: strip_html_attributes, options: { extension: EXTENSION_OPTIONS, render: render_options }, plugins: { syntax_highlighter: nil })
with_chips = transform_mention_anchors(html)
with_references = transform_reference_anchors(with_chips, numbered_sections: footnote_prefix.nil?)
scrubber = LinkScrubber.new
Expand All @@ -84,7 +87,7 @@ def render_markdown(content, interactive: true, footnote_prefix: nil, footnotes:
sanitized = render_iframe_blocks(sanitized, content.to_s)
sanitized = render_callouts(sanitized)
result = interactive ? make_checkboxes_interactive(sanitized, content, line_offset: line_offset, source_comments: source_comments, retain_sourcepos: retain_sourcepos) : sanitized
unless interactive
unless interactive || retain_sourcepos
fragment = Nokogiri::HTML::DocumentFragment.parse(result)
fragment.css("[data-sourcepos]").each { |node| node.remove_attribute("data-sourcepos") }
result = fragment.to_html
Expand Down Expand Up @@ -216,11 +219,12 @@ def select_footnotes(html, mode)
return html if mode == :inline

doc = Nokogiri::HTML::DocumentFragment.parse(html)
sections = doc.css("section[data-footnotes]")
# A rendered Markdown block keeps its own footnotes next to its body.
sections = doc.css("section[data-footnotes]").reject { |section| section.ancestors("[data-markdown-block-rendered]").any? }
if mode == :only
sections.map(&:to_html).join
elsif mode == :exclude
sections.remove
sections.each(&:remove)
doc.to_html
else
raise ArgumentError, "unknown footnote mode: #{mode.inspect}"
Expand Down Expand Up @@ -284,7 +288,10 @@ def math_positions(source)
def make_checkboxes_interactive(html, content, line_offset: 0, source_comments: false, retain_sourcepos: false)
doc = Nokogiri::HTML::DocumentFragment.parse(html)
source_lines = content.to_s.each_line.map(&:rstrip)
Plans::SourceTargets.new(content).annotate(doc) if source_comments && line_offset.zero?
if source_comments && line_offset.zero?
Plans::SourceTargets.new(content).annotate(doc)
render_markdown_blocks(doc, content)
end

doc.css('input[type="checkbox"]').each do |cb|
li = cb.ancestors("li").first
Expand Down Expand Up @@ -415,7 +422,8 @@ def render_callouts(html)
# rendered text — see exactly what they saw before.
def wrap_data_tables(html)
doc = Nokogiri::HTML::DocumentFragment.parse(html)
tables = doc.css("table").reject { |table| table.ancestors("table").any? }
# A Markdown block's rendered pane already wrapped its own tables.
tables = doc.css("table").reject { |table| table.ancestors("table, [data-markdown-block-rendered]").any? }
return html if tables.empty?

tables.each do |table|
Expand All @@ -442,7 +450,7 @@ def wrap_data_tables(html)
# sits inside the link.
def wrap_expandable_images(html)
doc = Nokogiri::HTML::DocumentFragment.parse(html)
images = doc.css("img")
images = doc.css("img").reject { |image| image.ancestors("[data-markdown-block-rendered]").any? }
return html if images.empty?

images.each do |image|
Expand All @@ -464,6 +472,128 @@ def wrap_expandable_images(html)
doc.to_html
end

# In the reading view, a `markdown` or `md` fence shows its body
# rendered, with a toggle back to the literal source. The source <pre>
# stays in the DOM, hidden, and remains the text that comment anchors
# count and match, so the document's anchor text model (and the
# server's, MarkdownTextExtractor) is the same whichever view shows.
# The rendered pane is kept out of that model; its block elements carry
# data-markdown-from/to, body-relative character offsets that let the
# browser map a rendered selection or highlight to the source.
#
# Like math, only a fence the parser produced qualifies: authored HTML
# can imitate the lang and sourcepos, so the <pre> must sit at the
# unique position of a markdown code block in the AST.
def render_markdown_blocks(doc, content)
pres = doc.css("pre[lang][data-sourcepos]").select do |pre|
MARKDOWN_BLOCK_LANGUAGES.include?(pre["lang"].downcase) && pre.ancestors(".deck-region, [data-markdown-block]").none?
end
return doc if pres.empty?

positions = markdown_block_positions(content)
source_targets = Plans::SourceTargets.new(content)
# Count excluded deck fences too: authored HTML must not borrow a
# real fence's position merely because that fence stays literal.
counts = doc.css("pre[data-sourcepos]").map { |pre| pre["data-sourcepos"] }.tally
pres.each_with_index do |pre, index|
next unless positions.include?(pre["data-sourcepos"]) && counts[pre["data-sourcepos"]] == 1

code = pre.element_children.find { |child| child.name == "code" }
next unless code

block = doc.document.create_element("div", class: "markdown-block", "data-markdown-block" => "",
"data-controller" => "coplan--markdown-block")
bar = doc.document.create_element("div", class: "markdown-block__bar", "data-markdown-block-chrome" => "")
bar.add_child(doc.document.create_element("span", pre["lang"], class: "markdown-block__label"))
views = doc.document.create_element("div", class: "markdown-block__views", role: "group", "aria-label" => "Markdown view")
{ "rendered" => "Rendered", "source" => "Source" }.each do |view, label|
views.add_child(doc.document.create_element("button", label, type: "button", class: "markdown-block__view",
"aria-pressed" => (view == "rendered").to_s, "data-view" => view,
"data-coplan--markdown-block-target" => "button", "data-action" => "coplan--markdown-block#show"))
end
bar.add_child(views)
rendered = doc.document.create_element("div", class: "markdown-block__rendered", "data-markdown-block-rendered" => "",
"data-coplan--markdown-block-target" => "rendered")
rendered.inner_html = render_markdown_block_body(code.text, index + 1, source_targets: source_targets, position: pre["data-sourcepos"])
rendered["data-markdown-map"] = markdown_block_text_map(code.text).to_json

pre.add_previous_sibling(block)
pre["hidden"] = ""
pre["data-markdown-block-source"] = ""
pre["data-coplan--markdown-block-target"] = "source"
block.add_child(bar)
block.add_child(rendered)
block.add_child(pre)
end
doc
end

# Source positions of the fences that render as Markdown blocks.
def markdown_block_positions(content)
positions = Set.new
CoPlan::Markdown.parse(content.to_s.encode("UTF-8"), options: { extension: EXTENSION_OPTIONS }).walk do |node|
next unless node.type == :code_block && MARKDOWN_BLOCK_LANGUAGES.include?(node.fence_info.to_s.split.first&.downcase)

p = node.source_position
positions << "#{p[:start_line]}:#{p[:start_column]}-#{p[:end_line]}:#{p[:end_column]}"
end
positions
end

# A block's body renders read-only: no task toggles (their lines are
# not document lines), no outline headings or heading ids (they would
# collide with the document's), and footnotes scoped to the block.
def render_markdown_block_body(body, number, source_targets:, position:)
html = render_markdown(body, interactive: false, retain_sourcepos: true, footnote_prefix: "markdown-block-#{number}", strip_html_attributes: %w[id data-sourcepos])
wrapper = Nokogiri::HTML::DocumentFragment.parse(html).element_children.find { |node| node.name == "div" && node.classes.include?("markdown-rendered") }
inner = Nokogiri::HTML::DocumentFragment.parse(wrapper ? wrapper.inner_html : html)
source_targets.annotate_markdown_block(inner, body, position)
inner.css(%w[h1 h2 h3 h4 h5 h6].map { |heading| "#{heading} a.anchor[id]" }.join(", ")).each do |anchor|
anchor.remove if anchor.text.strip.empty?
end
# Keep generated, block-scoped footnote targets. Other authored ids
# could steal the enclosing document's outline or fragment targets.
inner.css("[id]").each do |element|
next if element["id"].start_with?("markdown-block-#{number}-fn")

element.remove_attribute("id")
end
lines = body.lines
offsets = [ 0 ]
lines.each { |line| offsets << offsets.last + line.length }
# Browsers index strings in UTF-16 code units, Ruby in characters.
# They differ after a character outside the BMP, such as an emoji.
astral = body.match?(/[^\u0000-\uFFFF]/)
utf16 = ->(offset) { astral ? body[0, offset].encode("UTF-16LE").bytesize / 2 : offset }
inner.css("[data-sourcepos]").each do |element|
range = Plans::SourceTargets.char_range(element["data-sourcepos"], lines, offsets)
element.remove_attribute("data-sourcepos")
next unless range
next if element["data-markdown-from"]

element["data-markdown-from"], element["data-markdown-to"] = range.map { |offset| utf16.call(offset).to_s }
end
inner.to_html
end

# Reuse the parser-backed source spans used by writing provenance. A
# visible entity or escape can cover several source characters; searching
# raw source for visible letters loses those boundaries (and can match a
# link URL or an earlier formatted copy instead). Image alt text has no
# DOM text node; footnote numbers are generated UI. Neither is a candidate.
def markdown_block_text_map(body)
text, _, spans = Plans::MarkdownTextExtractor.call(body, source_spans: true, exclude_nodes: %i[image footnote_reference])
utf16_offsets = [ 0 ]
body.each_char { |char| utf16_offsets << utf16_offsets.last + (char.ord > 0xffff ? 2 : 1) }
browser_spans = []
text.each_char.with_index do |char, index|
from, to = spans.fetch(index)
span = from.negative? ? nil : [ utf16_offsets.fetch(from), utf16_offsets.fetch(to) ]
(char.ord > 0xffff ? 2 : 1).times { browser_spans << span }
end
{ text: text, spans: browser_spans }
end

# Rewrites commonmarker's per-document footnote ids (#fn-N / #fnref-N)
# and the hrefs that point at them so multiple fragments can coexist in
# one DOM. Only ids/fragments with the fn-/fnref- shape are touched —
Expand Down
Loading
Loading