Skip to content

ci: run the Obsidian E2E suite in CI - #227

Merged
chhoumann merged 2 commits into
masterfrom
ci/e2e-suite
Oct 24, 2026
Merged

chhoumann merged 2 commits into
masterfrom
ci/e2e-suite

Conversation

@chhoumann

@chhoumann chhoumann commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

Summary

The live E2E suite (tests/e2e/, 110 tests) only ran on maintainers' machines. CI now runs it on every PR and push to master.

New E2E (Obsidian) job in ci.yml:

  • Downloads the Obsidian 1.14.4 Linux tarball from obsidianmd/obsidian-releases and checks its sha256 (from the release asset digest). 1.14.4 is pinned rather than the latest 1.14.5, which is two days old, to keep the 7-day release-age rule.
  • Installs xvfb/xauth, builds, then starts the obsidian-e2e instance headless (--obsidian-app / --obsidian-bin point at the tarball). The runner already wraps Xvfb and passes --no-sandbox, and it seeds obsidian.json with the CLI enabled.
  • Runs vitest --config vitest.e2e.config.mts with HOME at the instance's private home, uploads .obsidian-e2e-artifacts/ and the app log on failure, and stops the instance.

obsidian-e2e 0.14.0 to 0.15.0. 0.15.0 needs Vitest ^5 and Node >=22.12. The repo is already on Vitest 5.0.5, engines is ^22.13.0 || ^24.0.0 || >=26, and CI uses Node 24, so this is a version bump only. obsidian-e2e is on the minimumReleaseAgeExclude list. Its new yaml@2.9.1 floor was already in the lockfile.

Flaky RUN-03. In a full-suite run, RUN-03: duplicate-key Notice is well-formed read the first .notice on screen. That was sometimes a leftover from an earlier file (MetaEdit could not update 'count'…). It now waits for the duplicate-key notice. Alone, the file always passed.

Rune guard test. rejectSvelteRuneModules moves from esbuild.config.mjs to scripts/svelte-rune-guard.mjs, unchanged. The new scripts/svelte-rune-guard.test.ts bundles .svelte.ts/.js/.mts imports through esbuild and expects the build to fail, and checks that a plain .ts import builds. With the guard's filter disabled, 3 of the 4 cases fail.

Skipped wikilink test. It was skipped because of a real bug, not the transport. #226 fixes the bug and re-enables the test, so this PR leaves it alone.

Verification

  • Local headless run mirroring the job: Obsidian 1.14.4 tarball, DISPLAY unset, obsidian-e2e 0.15.0. 16 files passed, 109 passed, 1 skipped, in 1m57s.
  • Before the RUN-03 fix, the same full run on 1.13.7 failed RUN-03 once. The file alone passed 3 of 3.
  • pnpm run lint, pnpm run build, pnpm run test (363 passed).
  • This PR's own E2E (Obsidian) check is the CI proof.

No runtime or release impact.


Note

Low Risk
Changes are CI, test harness, and build-plugin organization; no plugin runtime or release behavior.

Overview
CI now runs the live Obsidian E2E suite on every PR/push via a new E2E (Obsidian) job: pinned Obsidian 1.14.4 (SHA-256 verified), Xvfb, build, obsidian-e2e start/stop, Vitest with the instance’s private HOME, and failure artifact upload. AGENTS.md notes this mirrors local pnpm run test:e2e.

obsidian-e2e is bumped 0.14.0 → 0.15.0 (lockfile updated). The Svelte rune rejection esbuild plugin is extracted to scripts/svelte-rune-guard.mjs (behavior unchanged) with new unit tests that assert .svelte.ts/.js/.mts imports fail the build.

E2E flake fix: RUN-03 waits for a notice whose text includes already has property instead of the first .notice on screen.

Reviewed by Cursor Bugbot for commit 68dbccc. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add Obsidian E2E suite to CI with pinned Obsidian 1.14.4

  • New ci.e2e GitHub Actions job runs the Vitest E2E suite headlessly under Xvfb on Ubuntu, against a SHA-256-verified Obsidian 1.14.4 download
  • Failure artifacts (E2E artifact directory and Obsidian logs) are uploaded, and Obsidian is stopped even when the job fails
  • Bumps obsidian-e2e dependency range from ^0.14.0 to ^0.15.0 in package.json
  • Moves the rejectSvelteRuneModules esbuild guard from esbuild.config.mjs into svelte-rune-guard.mjs and adds a Vitest suite covering .svelte.ts/.svelte.js/.svelte.mts rejection
  • Fixes flaky duplicate-property E2E notice matching in audit-suggester.test.ts by waiting for a .notice containing already has property
  • Risk: CI now depends on the pinned Obsidian 1.14.4 release in .github/workflows/ci.yml; verify the SHA-256 stays current on Obsidian updates

Macroscope summarized 68dbccc.

Add an E2E (Obsidian) job that installs a pinned, checksum-verified
Obsidian 1.14.4 tarball, starts the obsidian-e2e instance headless under
Xvfb, and runs tests/e2e. Failure artifacts and the app log are uploaded.

- Bump obsidian-e2e to 0.15.0 (Vitest 5, Node 22.12+; both already met).
- RUN-03 waited for the first .notice on screen, which could be a
  leftover from an earlier suite. It now waits for the duplicate-key one.
- Move the Svelte rune-module guard out of esbuild.config.mjs into
  scripts/svelte-rune-guard.mjs and cover it with a unit test.
Comment thread .github/workflows/ci.yml Resolved
@chhoumann
chhoumann merged commit f6b46b5 into master Oct 24, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant