Skip to content

feat(browser): render every device of a URL in one job, one result (v1.23.0) - #155

Open
harper-joseph wants to merge 2 commits into
mainfrom
feat/url-jobs-browser
Open

feat(browser): render every device of a URL in one job, one result (v1.23.0)#155
harper-joseph wants to merge 2 commits into
mainfrom
feat/url-jobs-browser

Conversation

@harper-joseph

Copy link
Copy Markdown
Contributor

What

A job is one URL. When the plugin (>= 0.66.0, follow-up PR) claims a job carrying deviceTypes, the worker renders every device in turn on the job's one concurrency slot — a fresh page per device, through the same renderer — and posts one result:

{ id, url, deviceTypes, variants: [ { deviceType, outcome, statusCode, headers, renderTime, isIndexable, structuredOffers, reason, error, contentLength }, … ] }

followed by the variants' encoded bodies concatenated in order (each variant's contentLength says how many of those bytes are its own). The README gains a Queue protocol section with the full shape.

Why

The plugin's schedule is per device today, so a URL's desktop and mobile renders are two independent rows that drift apart: render-now and revalidate write one device key on purpose, reconcile repairs a missing row with fresh jitter, each retry lane delays only its own device, and completion-relative rescheduling re-anchors each device to its own finish time. The codebase calls a split pair "a normal production state", and it costs real complexity (PageVerification.basisAt, the per-row verdicts in the reenqueue path) and some correctness (the per-URL strikes counter is fed per device, so both devices failing burns the fast-retry lane in one cycle, while one device succeeding resets it under the other). Rendering every device of a URL in one pass and returning one result is what lets the plugin make one scheduling decision per URL and keep the pair aligned by construction. Measured on the customer corpus, desktop and mobile structured data agreed byte-for-byte in 39/40 samples; the one exception was a pair rendered 41h apart.

How

  • RenderJob.variants() fans a multi-device job out to one per-device RenderJob sharing the claim (id, lease, callback); a legacy job is its own single variant, so the Renderer contract is unchanged and renderOnce is untouched.
  • sendResult (legacy flat shape) and the new RenderJob.sendVariantsResult share one postResult with the existing retry policy (retriable statuses, host health, lease-bounded), so the two shapes cannot drift.
  • Variants run sequentially, deliberately: parallel variants would need page-level accounting against concurrency and double the job's burst on the origin; sequential keeps every capacity number true (renders per slot unchanged; a job just holds its slot for N renders). rps therefore paces job starts — noted in the options table.
  • A variant is skipped and the result posted partial when the lease has under 30s left (the same floor the run loop applies to a job as claimed, now one named constant) or the worker began draining between variants. The plugin stores what rendered and retries the URL for the rest; a result that never arrived would cost the whole lease first.
  • Per-window stats gain jobs (results posted) beside completed (renders) and variantsSkipped.

Compatibility / rollout order

  • A job without deviceTypes (any released plugin) is rendered and posted exactly as before — the existing jobResult tests pin the legacy envelope unchanged. This can be deployed ahead of the plugin.
  • The reverse is degraded, not broken: an older renderer handed a multi-device job renders only deviceType (the plugin sends the first device there for exactly this reason) and posts it flat; the plugin stores that one device and the others go unrendered until the fleet is upgraded. So: render fleet first, then plugin 0.66.0.

Tests

  • test/jobResult.test.ts: variants() fan-out; the multi-variant envelope and body framing decoded exactly as the plugin will (offsets walk contentLength, sum to the body); a no-content variant consumes zero bytes; a partial result echoes deviceTypes while listing only attempted variants; the legacy envelope is unchanged (no variants/deviceTypes keys).
  • test/variantRender.test.ts (new): the worker loop over a stub browser — sequential order, a page per variant, every page closed, job refs never overlap, one POST; a throwing variant is error and does not stop the rest; a lease that runs short between variants posts partial; a legacy job renders once and posts flat.
  • 172/172 browser tests, npm run lint, npm run format:check locally (this repo has no PR CI).

Version

@harperfast/prerender-browser 1.21.0 → 1.23.0 (1.22.0 is reserved by #154). Plugin follow-up: prerender-v0.66.0.

🤖 Generated with Claude Code

…esult; v1.23.0

A job is now one URL. When the plugin (>= 0.66.0) claims a job carrying
`deviceTypes`, the worker renders each device in turn on the job's single
concurrency slot — a fresh page per device, the same renderer — and posts ONE
result: `{ id, url, deviceTypes, variants: [...] }` followed by the variants'
encoded bodies concatenated in order, each variant declaring its own
`contentLength`. That is what lets the plugin keep a URL's device variants
aligned (same render pass, seconds apart, one scheduling decision) instead of
the split pairs every per-device retry lane, render-now and reconcile repair
produce today.

- `RenderJob.variants()` fans a multi-device job out to one per-device job
  sharing the claim; a legacy job is its own single variant, so the renderer
  contract is unchanged.
- `sendResult` (legacy, flat shape) and the new `sendVariantsResult` share one
  `postResult` with the existing retry policy, so the two cannot drift.
- A variant is skipped and the result posted PARTIAL when the lease has under
  30s left or the worker began draining between variants; the plugin retries
  the URL for the devices it did not get back.
- Stats gain `jobs` (results posted) beside `completed` (renders) and
  `variantsSkipped`.

Compatibility: a job without `deviceTypes` is rendered and posted exactly as
before, so this deploys ahead of the plugin. An older renderer handed a
multi-device job renders only the first device — degraded, not broken — so the
render fleet rolls out first.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements support for multi-device render jobs, allowing a single job to handle multiple device variants sequentially within a single concurrency slot. Key changes include updating the RenderJob and RenderWorker classes to manage variant fanning and result aggregation, introducing a minimum lease time check to prevent late posting, and adding comprehensive tests for the new queue protocol. I have reviewed the changes and the provided feedback regarding potential runtime errors in the render loop, which should be addressed to ensure robustness.

Comment thread packages/browser/src/Worker.ts Outdated
Comment on lines +499 to +504
const posted = await (
job.deviceTypes ? RenderJob.sendVariantsResult(job, attempted) : attempted[0].sendResult()
).catch((err) => {
logger.error({ id: job.id, err }, 'failed to send job result');
return false;
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

If attempted is empty, attempted[0] will be undefined. Calling attempted[0].sendResult() will throw a synchronous TypeError which bypasses the .catch() block. Guard against an empty attempted array to ensure synchronous errors are avoided before the promise is returned. Additionally, ensure error serialization is robust by using optional chaining.

const posted = await (
			attempted.length === 0
				? Promise.resolve(false)
				: job.deviceTypes
				? RenderJob.sendVariantsResult(job, attempted)
				: attempted[0].sendResult()
		).catch((err) => {
			logger.error({ id: job.id, err: err?.message ?? String(err) }, 'failed to send job result');
			return false;
		});
References
  1. When calling an async function, using .catch() is sufficient unless the target object might be null/undefined or the method might be missing, in which case synchronous errors must be handled.
  2. When handling or serializing caught exceptions, do not assume the error is a standard Error object. Use error?.message ?? String(error) to ensure robust serialization.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7acc296: the post now starts inside a Promise.resolve().then(...) chain, so a synchronous throw lands in the same .catch and is counted as a post failure rather than escaping render(). For the record attempted cannot be empty — the skip check only runs after one variant has completed, and variants() always yields at least one — which the comment now states; the chain form keeps that from having to be trusted.

…chronous throw is caught too (review)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant