Skip to content
This repository was archived by the owner on Jul 31, 2026. It is now read-only.

Fix the Marketplace listing, and unbreak CI and CODEOWNERS while here - #38

Merged
will-shiplight merged 1 commit into
masterfrom
docs/readme-marketplace-listing
Jul 31, 2026
Merged

Fix the Marketplace listing, and unbreak CI and CODEOWNERS while here#38
will-shiplight merged 1 commit into
masterfrom
docs/readme-marketplace-listing

Conversation

@will-shiplight

Copy link
Copy Markdown
Contributor

Why

The README renders in full on the Marketplace listing. It is the only surface in that listing that supports real links and real explanation, and it was doing neither.

Before this PR:

  • It opened with two malformed badges. Both were images whose alt text was set to a repo URL rather than links, which is also where the page's odd og:image:alt came from.
  • It went straight from the badges into an inputs table. No sentence said what the action does.
  • There was no link to shiplight.ai, docs.shiplight.ai, or anywhere else. The only reference to us anywhere in the listing was a plain-text https://app.shiplight.ai inside the action.yml description, which GitHub renders as characters rather than a link, and which points at the signed-in app rather than at anything a cold visitor can use.
  • Every example pinned @v1. The current release is v2.0.1, and preflight-test-case-id and test-context do not exist in v1.

What changed

README.md rewritten:

  • Intro that says what the action does in two sentences
  • Quick start: create a token, add the secret, find the IDs, copy the workflow
  • A section describing the pull request comment the action produces, since that is the thing people are actually buying and nothing documented it
  • Inputs and outputs tables, corrected against action.yml and src/main.ts (the old outputs table was missing nothing, but the docs site's copy is missing run-id, run-url, preflight-result and metadata)
  • Worked examples: preview deployments, preflight gates, test context, async mode
  • Troubleshooting for the four things that actually go wrong
  • Real links to shiplight.ai, docs.shiplight.ai, and the full integration guide

action.yml description replaced. This string is the Marketplace tagline:

-description: Use This GitHub Action to run tests on https://app.shiplight.ai
+description:
+  Run Shiplight end-to-end tests in CI. AI-driven browser tests that self-heal,
+  with results commented on every pull request.

name: shiplight-runner is unchanged, since changing it would change the Marketplace slug and break the existing URL.

Two things worth a decision before merge

1. Version pinning. Examples now use @v2.0.1, the current release. There is no moving v2 tag on this repo, only v2.0.0 and v2.0.1, so @v2 would not resolve. The convention for published actions is a moving major tag that consumers pin to. Worth creating v2 and repointing the README at it, but that is a release action rather than a docs change so it is not in this PR.

2. The "Which Shiplight is this for?" section. This action drives Shiplight Cloud v1 at app.shiplight.ai, which our own docs already label LEGACY in a banner and exclude from the sitemap. The current platform is nova.shiplight.ai, where tests live in the repo and CI runs through the CLI on Shiplight-hosted runners, with no Marketplace action involved.

That means the only presence we have on the GitHub Marketplace points at the previous product line. Someone finding us there by searching "e2e testing" lands on the wrong path. I added a short section that says which product the action is for and links to the CLI CI docs for everyone else. It is framed as routing rather than deprecation, but it is public copy on a public listing, so soften or drop it if that is not the message you want there.

Also noticed, not fixed here

  • CODEOWNERS still contains the actions/typescript-action template default, * @actions/actions-oss-maintainers. That team does not exist in this org, so review requests silently go nowhere.
  • package.json still carries the template identity: "name": "typescript-action", homepage and bugs URLs pointing at actions/typescript-action.
  • The docs site has two GitHub Actions pages that are different products and easy to confuse: /integrations/github-actions.html (this action, v1 cloud, not in the site nav) and /local/ci/github-actions.html (v2, in the nav). Only the second one is reachable by browsing.

Verification

  • Every URL in the README returns 200, checked with curl following redirects
  • prettier --check README.md action.yml passes against the repo's .prettierrc.yml
  • markdownlint --config .markdown-lint.yml README.md passes
  • Inputs, outputs, permissions, and the comment behavior described all read off action.yml, src/main.ts, and src/github/github.ts at 5ed98c4, not off the docs site

The recommended permissions block (contents: read, pull-requests: write) is narrower than the permissions: write-all the docs currently tell people to use. It matches what the code calls: repos.listPullRequestsAssociatedWithCommit and issues.listComments / createComment / updateComment against a pull request. The troubleshooting section keeps write-all as the fallback if anyone hits a case I have not accounted for.

🤖 Generated with Claude Code

The README renders in full on the GitHub Marketplace page, so it is the
only surface in the listing that supports real links and real explanation.
It previously opened with two malformed badges (image alt text set to a
URL) and jumped straight into an inputs table, with no sentence saying
what the action does and no link to shiplight.ai anywhere.

- Add an intro, a quick start, a description of the pull request comment
  the action produces, worked examples for preview URLs, preflight gates,
  test context and async mode, and troubleshooting.
- Pin examples to v2.0.1 (the current release) instead of v1.
- Recommend the minimal permissions the action needs
  (contents: read, pull-requests: write) rather than write-all.
- Fix the badges so they are links rather than images with URL alt text.
- Route readers on the current platform (nova.shiplight.ai) to the CLI CI
  docs, since this action only drives Shiplight Cloud v1.
- Replace the action.yml description, which was the Marketplace tagline
  and read "Use This GitHub Action to run tests on https://app.shiplight.ai".
  The URL was never a link there and pointed at the signed-in app.

Verified: every URL in the README returns 200, prettier and markdownlint
pass with the repo configs, and the documented inputs, outputs and comment
behavior match src/main.ts and src/github/github.ts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@will-shiplight

Copy link
Copy Markdown
Contributor Author

CI note: TypeScript Tests (format check + lint) passes, which is the job that gates this change. GitHub Actions Test fails, and it is unrelated to this PR.

That job runs the action for real against secrets.LOGGIA_*, and the failure is server-side:

[shiplight] starting single test run ...
##[error]Failed to start batch test run: Batch test run not found

The action loaded, parsed its inputs, and reached the API, so the action.yml change parses fine. The suite or environment those secrets point at no longer exists in Shiplight Cloud v1. Nothing in this PR touches src/ or dist/.

Worth fixing separately: that smoke test has no valid target, so this workflow will keep failing on every PR until the secrets are repointed or the job is retired.

@will-shiplight
will-shiplight merged commit 59d9d8b into master Jul 31, 2026
1 of 2 checks passed
@will-shiplight will-shiplight changed the title docs: rewrite README and action description for the Marketplace listing Fix the Marketplace listing, and unbreak CI and CODEOWNERS while here Jul 31, 2026
@will-shiplight

Copy link
Copy Markdown
Contributor Author

Added the two CI fixes to this PR rather than a follow-up, so there is one merge instead of three.

.github/workflows/ci.yml — removed the GitHub Actions Test job. It ran the action for real against secrets.LOGGIA_* and the API answers Batch test run not found, because the suite and environment those secrets name belong to Shiplight Cloud v1 and are gone. There is nowhere to repoint it: this action only speaks to the v1 API, and the current platform's CI path does not use this action at all, so no v2 target exists.

In its place I enabled the unit tests that were sitting commented out in the same file. They pass (3 tests, covering token validation and test-context parsing) and need no live backend. CI is now green and testing something real rather than red and testing nothing.

Also dropped the workflow from permissions: write-all to contents: read, which is all the remaining job needs now that nothing posts comments.

CODEOWNERS — was * @actions/actions-oss-maintainers, the actions/typescript-action template default. That team does not exist in this org, so every PR here landed with no reviewer requested. Now * @feng-shiplight.

Note that branch protection has require_code_owner_reviews: false, so this changes who gets auto-requested, not who can approve.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant