Skip to content

fix(english): load Royal Road chapter list through parsePage - #2665

Open
RibatTRW wants to merge 2 commits into
lnreader:masterfrom
RibatTRW:fm/lnreader-plugins-royalroad-2659
Open

RibatTRW wants to merge 2 commits into
lnreader:masterfrom
RibatTRW:fm/lnreader-plugins-royalroad-2659

Conversation

@RibatTRW

@RibatTRW RibatTRW commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Intent

Work on #2659 and include screenshots of the working fix in the PR.

Issue 2659, "[RoyalRoad] Some(stubbed?) novels don't load." (plugin: English, Royal Road; plugin version 2.3.1; LNReader 2.1.4, Android 14): searching for and opening "A Practical Guide to Sorcery" (not Evil) or "Mother of Learning", the chapter list does not load at all; expected the list of chapters to load. The reporter suspects these are stubbed novels.

What Changed

  • Moved the window.chapters / window.volumes extraction out of the parseNovel HTML parser callback into a reusable parseChapters(scriptContent) method on the Royal Road plugin. parseNovel now calls it, and chapter output (names, paths, release times, order, volume titles) is unchanged.
  • Added a parsePage(novelPath, page) method that fetches the fiction page and returns its chapters. Novels saved without a chapter list (e.g. "A Practical Guide to Sorcery", "Mother of Learning") can now get their list from the app. Page "1", which the app requests when it has no stored pages, returns the full list. Any other page returns only the chapters whose volume title matches.
  • Bumped the plugin version from 2.3.1 to 2.3.2.

Fixes #2659

🤖 Generated with Claude Code

Root cause

The current plugin (2.3.1) parses both reported novels correctly the first time they are opened. Fetched through curl from the live site, Mother of Learning gives 109 chapters and A Practical Guide to Sorcery gives 18. The same counts come out under Node, under Hermes 0.12 with the plugin compiled to ES5, and in this repo's playground. A Practical Guide to Sorcery is a real stub (status label STUB, 18 chapters still published). Mother of Learning is not a stub; all 109 chapters are published.

The failure is in how LNReader 2.1.4 opens a novel it has already stored (src/hooks/persisted/useNovel/store-helper/bootstrapService.ts):

  1. If the novel is in the database, the app uses that row and does not parse the novel again.
  2. If no chapters are stored for the current page (page "1" for Royal Road), the app calls plugin.parsePage(path, "1").
  3. Royal Road had no parsePage, so fetchPage threw Could not fetch chapters for page 1.
  4. The novel screen showed a load error with no chapters. The store's novel is never set on that failure, so pull-to-refresh (useNovelRefresh) returns early and can't repair it.

Once a novel is stored without its chapters, that novel stays broken while every other novel keeps working, which matches "some novels don't load".

Limit: I couldn't observe which event left the reporter's two novels stored without chapters. That would need their device or database. One way it can happen: insertNovelAndChapters saves the novel row, then waits for the cover download, and only then inserts the chapters.

Ruled out:

  • These two pages break the parser (for example, a chapter title that cuts off the lazy window.chapters regex). Every runtime above parses both pages completely. Pages served to an Android WebView or Android Chrome User-Agent contain the same chapter data.
  • Hermes fails on the regex. Hermes 0.12 runs the chapter regex over each page's script text without error.
  • Leftover volume-titled pages from before 2.3.0 get collapsed to page "1". Ruled out for Mother of Learning: Wayback Machine snapshots from March 2025 to October 2026 show window.volumes = [] and every chapter with volumeId: null.
  • Search returns the wrong path. The plugin returns fiction/21220 and fiction/34009.

Risk Assessment

⚠️ Medium: The refactor of parseNovel and the new parsePage are behavior-preserving and read correctly, but the root cause of #2659 still isn't confirmed against the live site. The user has already declined that concern, so I'm not raising it again.

Testing

This was not tested in the LNReader app. The Android toolchain isn't installed on the machine that made this change.

All live-site evidence below comes from Royal Road pages fetched through curl. Node's own fetch, and the no-mistakes validation pipeline, get a Cloudflare 403 from royalroad.com. The screenshots are from this repo's plugin playground, running in headless Chromium with the playground proxy in curl mode. Everything was run on head 02a493d.

Scenario 1: Mother of Learning and A Practical Guide to Sorcery show their chapter list

  • Playground, end to end: the Latest listing, search (both reported novels appear), novel detail, chapter list and chapter text. Mother of Learning lists 109 chapters, A Practical Guide to Sorcery lists 18, and Super Supportive (a normal novel) lists 312.

  • App path that failed (regression check): a harness runs LNReader v2.1.4's real bootstrapService.ts against the real plugin, with an in-memory stand-in for the app database. "Stored, no chapters" means the novel row exists with no chapters, which is the state that sends the app to parsePage. Results with volume view off (the default):

    Novel Not stored yet: 2.3.1 / 2.3.2 Stored, no chapters: 2.3.1 Stored, no chapters: 2.3.2
    Mother of Learning (fiction/21220) 109 / 109 ❌ Could not fetch chapters for page 1 ✅ 109
    A Practical Guide to Sorcery (fiction/34009) 18 / 18 ❌ same error ✅ 18
    Super Supportive (fiction/63759, normal) 312 / 312 ❌ same error ✅ 312
    Pale Lights (fiction/65058) 197 / 197 ❌ same error ✅ 197
    The Elf Who Would Become A Dragon (fiction/104434) 160 / 160 ❌ same error ✅ 160

    The same matrix with volume view on gives the same results: 2.3.1 fails every "stored, no chapters" case, and 2.3.2 loads the full list for each novel.

Scenario 2: with volume view on, parsePage returns each volume's chapters for its own page

Mother of Learning has no volumes, so this used two Royal Road fictions that do:

  • Pale Lights: 3 volumes plus 1 chapter outside any volume.
  • The Elf Who Would Become A Dragon: 4 volumes.

With the plugin's "Enable Pagination / Volume view" setting on:

Call Chapters returned
Pale Lights parsePage("Lost Things") / ("Good Treasons") / ("Borrowed Trouble") 46 / 80 / 70, all from that volume
Pale Lights parsePage("1") 197, the whole list
The Elf parsePage("Spring") / ("Summer") / ("Autumn") / ("Winter") 38 / 40 / 39 / 43, all from that volume
The Elf parsePage("1") 160, the whole list
A Practical Guide to Sorcery parsePage("A Conjuring of Ravens") / ("A Pact of Stars") / ("1") 2 / 16 / 18

Each volume page matched the chapters parseNovel labels with that volume.

This testing found a bug, which commit 02a493d fixes. In the first version, parsePage("1") returned only the chapters labelled "1" when any existed. For Pale Lights that was a single chapter ("Maps") out of 197. Page "1" now always returns the whole list.

Other checks

  • Hermes 0.12 with the plugin compiled to ES5 (as in the production build): parseNovel and parsePage("1") return 109 / 18 / 197 with volume view off and on.
  • npm run build:compile passes, and Prettier and ESLint are clean for plugins/english/royalroad.ts.
  • Plugin Live Check (CI) is red because Cloudflare blocks the GitHub runner, not because of this change. In run 38056493906 and the earlier run 38055684264, popularNovels returned no novels:
    • This diff doesn't touch popularNovels or parseNovels.
    • With the checker's default filters, master (2.3.1) and this branch both return 20 novels through curl. Both return 0 through Node's fetch, which gets HTTP 403 with cf-mitigated: challenge.
    • fetchApi doesn't throw on a status code, so a challenge page parses to an empty list. The checker then reports FAIL rather than INCONCLUSIVE, because its root-URL probe passed on the runner.
    • This is the first live-check run on royalroad.ts; the workflow was added after the plugin's last change.
    • Locally, npm run check:plugin -- plugins/english/royalroad.ts is INCONCLUSIVE (HTTP 403 from Cloudflare).

Screenshots

These come from this repo's plugin playground, not from the LNReader app. Cover images that go through the playground proxy in curl mode don't display (a playground limitation); covers in the listing and search load directly.

The playground only calls parsePage for plugins that report totalPages, so the last two screenshots call it from DevTools on the playground page, with the output printed on the page.

Latest listing

Latest listing

Search for "A Practical Guide to Sorcery" (both reported novels in the results)

Search results

Mother of Learning: details and chapter list (109)

Mother of Learning details
Mother of Learning chapter list

Mother of Learning: chapter 1 text

Mother of Learning chapter 1 text

A Practical Guide to Sorcery: chapter list (18, a stub)

A Practical Guide to Sorcery chapter list

Super Supportive, a normal novel: chapter list (312)

Super Supportive chapter list

parsePage(path, "1"), the call LNReader makes for a novel stored with no chapters (volume view off)

parsePage page 1 for the reported novels and a normal novel

parsePage per volume with volume view on (Pale Lights and The Elf)

parsePage per volume with volume view on


This PR was written by an AI agent (Claude Code), and no human has reviewed it. Please review it with that in mind.

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - medium risk

✅ No issues found.

⚠️ **Test** - 2 warnings
  • ⚠️ Live check for plugins/english/royalroad.ts is INCONCLUSIVE: royalroad.com returns HTTP 403 (Cloudflare) in this environment. The reported novels could not be opened, and no screenshots could be taken. This matches earlier declined rounds.
  • ⚠️ live validation verdict: inconclusive (0 of 1 scenarios were driven live against the product); untested: Open 'Mother of Learning' and 'A Practical Guide to Sorcery' and see the chapter list load
  • Live validation: ⚠️ inconclusive - 0 of 1 scenarios driven live against the product
Scenario Result Live Evidence
Open 'Mother of Learning' and 'A Practical Guide to Sorcery' and see the chapter list load ⏸️ untested no I ran the live checker, which got a Cloudflare 403 from royalroad.com. A browser, token or proxy that Cloudflare accepts would let this be re-run.
  • npm ci
  • npm run check:plugin -- plugins/english/royalroad.ts (INCONCLUSIVE, HTTP 403)
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

LNReader 2.1.4 loads a stored novel's chapters from its database and,
when the current page has none, asks the plugin's parsePage for them.
Royal Road had no parsePage, so a novel saved without its chapters failed
with "Could not fetch chapters for page 1" every time it was opened.
parsePage now returns the chapters from the fiction page, and the chapter
JSON parsing is shared with parseNovel.

Fixes lnreader#2659
@greptile-apps

greptile-apps Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable regression was established.

Summary

royalroad.ts moves chapter extraction into parseChapters, adds parsePage for chapter-list recovery, and raises the plugin version to 2.3.2.

  • Royal Road can rebuild missing chapter lists page by page.

Reviews (1) · Last reviewed commit: "fix(english/royalroad): serve chapter li..." · Reviewed by Greptile

LNReader asks parsePage for page "1" when a stored novel has no chapter
pages at all. With volume view on, a novel that mixes chapters outside any
volume with volume chapters (e.g. Pale Lights, 1 + 196) got only the one
unvolumed chapter. Page "1" now returns every chapter; any other page
returns that volume's chapters.
@RibatTRW

Copy link
Copy Markdown
Contributor Author

Plugin Check cant pass CF

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RoyalRoad] Some(stubbed?) novels don't load.

1 participant