Skip to content

Commit b33ac20

Browse files
authored
v0.9.44: go/ links and the Links page, extension voice input and page access, Mothership workspace file writes, trace content kept on large runs
2 parents dd06b90 + 9f42bd3 commit b33ac20

119 files changed

Lines changed: 4238 additions & 2297 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/CI.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,8 @@ Memory-heavy builds and browser jobs retain appropriately sized Blacksmith runne
3838

3939
WebKit acceptance on Blacksmith uses the matching, digest-pinned Playwright browser image for mobile and docs navigation. The app, database fixtures, native modules, and harness stay on the host and retain loopback URLs. Chromium and GitHub fallback keep native browser installation and launch. Update both workspace Playwright dependencies, the image index digest, and the expected client version/revision checks together; verify the installed release's native launch timeout and real WebKit acceptance before changing the pin.
4040

41+
HTTP and docs browser installation first asks the pinned Playwright driver to simulate its OS dependency installation. When the simulation requires no package changes, the runner avoids an apt update; missing dependencies, proposed upgrades, or an unsuccessful simulation use the normal dependency installer. Browser selection and downloads still use the pinned driver, including full native Chromium for extension acceptance.
42+
4143
Every HTTP and browser group consumes the same production build in an isolated job/database. Electron also uses that build and skips its development-route warm-up. Local HTTP and Electron development runs retain their dev-server behavior. Build inputs exclude test-only files so a harness edit can reuse an unchanged production build.
4244

4345
Desktop inbox protocol, assistant actions UI, and organization search UI run as three parallel HTTP jobs with isolated databases, apps, and reports. Each harness owns its acceptance task, so an edit selects its own job. The inbox protocol task depends on server and realtime code; the browser tasks also depend on workspace UI. Desktop inbox needs no browser. The UI jobs and browser extension install Chromium while their app starts, waiting for both before executing assertions; browser-install failures still fail the job and preserve the install log. No acceptance assertions are removed by the split.
@@ -71,6 +73,8 @@ The current weights are median test durations in seconds from runs [41496623284]
7173

7274
Desktop native outputs are keyed by source, build recipes, compiler, SDK, OS, architecture, and Node header bytes. Every build still validates restored checksums, executable modes, universal architectures, and the helper signature; invalid or incomplete outputs rebuild. Native-input pushes to main/staging seed trusted Node 22 and 24 caches without repeating the PR smoke suite. PR cache entries never populate trusted branch caches.
7375

76+
Desktop releases resolve and validate their source once, then check out that immutable commit for packaging and four isolated smoke partitions. The release smoke partitions reuse the existing whole-file duration sharder on the release's Node 22 toolchain and run alongside signing and notarization. Historical tags without the sharder retain one complete smoke run, and native compilation caching is optional when the checked-out revision predates it. Packaging retains source checks, package validation, and packaged Electron smoke. Publication waits for packaging and every smoke partition, downloads the packaging job's immutable artifact ID from the same run, verifies any existing release assets, and uploads the updater manifest last. Artifact-only runs never publish.
77+
7478
Each shard uploads its coverage/timing report, the complete discovery report, and its deterministic shard plan. `bun run test:scripts scripts/duration-shards.test.ts scripts/desktop-e2e.test.ts` checks partition coverage, unknown files, deterministic ordering, and balancing. The CLI regression executes tiny browser-free Playwright fixtures to prove that imported test factories and paths containing regex characters are selected correctly; it launches no Electron or browser.
7579

7680
Local pushes run `lint:check` and `type-check` through Husky before Git sends commits. These gates reuse Turborepo's task cache; a failure stops the push. The pre-commit hook still formats staged files, and hosted CI owns tests, audits, builds, and acceptance checks.

‎.github/scripts/http-e2e.sh‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ start_browser_install() {
7575
local name=$1
7676
shift
7777
browsers_log="$report_dir/$name-playwright-install.log"
78-
setsid --wait bunx --no-install playwright install --with-deps "$@" > "$browsers_log" 2>&1 &
78+
setsid --wait bash "$GITHUB_WORKSPACE/.github/scripts/install-playwright.sh" "$@" > "$browsers_log" 2>&1 &
7979
browsers_pid=$!
8080
}
8181

@@ -279,6 +279,7 @@ case "$group" in
279279
export REDIS_URL=redis://127.0.0.1:6379
280280
export NEXT_PUBLIC_FORCE_HOSTED=false
281281
export KNOWLEDGE_MEMBER_ACCESS=true CREDENTIAL_GROUPS=true ACCESS_CONTROL_ENABLED=true
282+
export SIM_LINKS=true
282283
export NEXT_PUBLIC_ACCESS_CONTROL_ENABLED=true
283284
export API_ENCRYPTION_KEY="$ENCRYPTION_KEY"
284285
start_browser_install "$group" --no-shell chromium
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
#!/usr/bin/env bash
2+
# The dependency dry run uses the pinned driver's package list without updating apt.
3+
set -euo pipefail
4+
5+
browsers=()
6+
for argument in "$@"; do
7+
case "$argument" in
8+
--only-shell|--no-shell) ;;
9+
*) browsers+=("$argument") ;;
10+
esac
11+
done
12+
13+
if ! bunx --no-install playwright install-deps --dry-run "${browsers[@]}"; then
14+
bunx --no-install playwright install-deps "${browsers[@]}"
15+
fi
16+
exec bunx --no-install playwright install "$@"

‎.github/workflows/desktop-release.yml‎

Lines changed: 145 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -57,20 +57,24 @@ on:
5757
default: true
5858

5959
permissions:
60-
contents: write
60+
contents: read
6161

6262
jobs:
63-
build:
64-
name: build
65-
runs-on: macos-26
66-
timeout-minutes: 60
63+
source:
64+
name: source
65+
runs-on: ubuntu-24.04
66+
timeout-minutes: 5
67+
outputs:
68+
name: ${{ steps.channel.outputs.name }}
69+
app_id: ${{ steps.channel.outputs.app_id }}
70+
origin: ${{ steps.channel.outputs.origin }}
71+
release_repository: ${{ steps.channel.outputs.release_repository }}
72+
token_kind: ${{ steps.channel.outputs.token_kind }}
6773
steps:
6874
- name: Checkout code
6975
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
7076
with:
71-
fetch-depth: 0
7277
persist-credentials: false
73-
ref: ${{ github.event_name == 'workflow_dispatch' && inputs.version || github.sha }}
7478

7579
# Prerelease versions carry their environment in the tag: -dev.N is a
7680
# dev build, -staging.N a staging build. Legacy -alpha/-beta tags remain
@@ -115,11 +119,16 @@ jobs:
115119
echo "::error::Manual desktop releases must use a stable source-repository tag."
116120
exit 1
117121
fi
122+
if [ "$GITHUB_EVENT_NAME" = workflow_dispatch ] && [ "$GITHUB_REF" != "refs/tags/$VERSION" ]; then
123+
echo "::error::Run this workflow from the $VERSION tag, so it executes only that tag's code and its caches stay scoped to that tag."
124+
exit 1
125+
fi
118126
if [ "$PUBLISH" = true ] && [ "$SIGN" != true ]; then
119127
echo "::error::Desktop releases must be signed before publication."
120128
exit 1
121129
fi
122130
if [ "$TOKEN_KIND" = stable ]; then
131+
git fetch --depth=1 origin "refs/tags/${VERSION}:refs/tags/${VERSION}"
123132
TAG_COMMIT="$(git rev-parse "refs/tags/${VERSION}^{commit}")"
124133
HEAD_COMMIT="$(git rev-parse HEAD)"
125134
if [ "$TAG_COMMIT" != "$HEAD_COMMIT" ]; then
@@ -140,28 +149,49 @@ jobs:
140149
exit 1
141150
fi
142151
152+
smoke:
153+
name: smoke (${{ matrix.shard }}/${{ strategy.job-total }})
154+
needs: source
155+
runs-on: macos-26
156+
timeout-minutes: 30
157+
strategy:
158+
fail-fast: false
159+
matrix:
160+
shard: [1, 2, 3, 4]
161+
steps:
162+
- &checkout-source
163+
name: Checkout verified source
164+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
165+
with:
166+
persist-credentials: false
167+
143168
- name: Setup Bun
144169
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
145170
with:
146171
bun-version: 1.4.4
147172

148-
- name: Setup Node
173+
- &setup-node
174+
name: Setup Node
149175
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v6
150176
with:
151177
node-version: 22
152178

153-
- name: Cache Electron binaries
179+
- name: Cache Electron and Playwright downloads
154180
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
155181
with:
156182
path: |
157183
~/Library/Caches/electron
158184
~/Library/Caches/electron-builder
159-
key: electron-cache-${{ runner.os }}-${{ hashFiles('apps/desktop/package.json') }}
185+
~/Library/Caches/ms-playwright
186+
key: desktop-downloads-${{ runner.os }}-${{ hashFiles('bun.lock') }}
187+
restore-keys: desktop-downloads-${{ runner.os }}-
160188

161-
- name: Install dependencies
189+
- &install-dependencies
190+
name: Install dependencies
162191
run: bun install --frozen-lockfile
163192

164-
- name: Inject release version
193+
- &inject-version
194+
name: Inject release version
165195
env:
166196
VERSION: ${{ inputs.version }}
167197
run: |
@@ -175,27 +205,83 @@ jobs:
175205
exit 1
176206
fi
177207
178-
- name: Verify desktop source
179-
run: |
180-
bun run --cwd apps/desktop lint:check
181-
bun run --cwd apps/desktop type-check
182-
bun run --cwd apps/desktop test
208+
- name: Restore native compilation
209+
uses: ./.github/actions/desktop-native-cache
183210

184-
- name: Bundle main and preload
211+
- &bundle-desktop
212+
name: Bundle main and preload
185213
working-directory: apps/desktop
186214
env:
187-
SIM_DESKTOP_DEFAULT_ORIGIN: ${{ steps.channel.outputs.origin }}
215+
SIM_DESKTOP_DEFAULT_ORIGIN: ${{ needs.source.outputs.origin }}
188216
run: bun run build
189217

190218
- name: Install system-browser fixture
191219
working-directory: apps/desktop
192220
run: bunx playwright install chromium
193221

194-
- name: Run Electron smoke tests
222+
- name: Install tmux
223+
run: brew install tmux
224+
225+
- name: Run Electron smoke partition
195226
working-directory: apps/desktop
196227
env:
197-
SIM_DESKTOP_DEFAULT_ORIGIN: ${{ steps.channel.outputs.origin }}
198-
run: bun run test:e2e
228+
SIM_DESKTOP_DEFAULT_ORIGIN: ${{ needs.source.outputs.origin }}
229+
PLAYWRIGHT_JSON_OUTPUT_FILE: test-results/desktop-smoke.json
230+
BACKGROUND_EXECUTOR_REPORT_PATH: test-results/background-executor-report.json
231+
TERMINAL_CANCEL_REPORT_PATH: test-results/terminal-cancel-report.json
232+
CONTINUE_SIGN_IN_REPORT_PATH: test-results/continue-sign-in-report.json
233+
run: bun run ../../scripts/desktop-e2e.ts ${{ matrix.shard }}/${{ strategy.job-total }}
234+
235+
- name: Upload smoke coverage and timings
236+
if: always()
237+
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
238+
with:
239+
name: desktop-release-smoke-${{ matrix.shard }}
240+
path: |
241+
apps/desktop/test-results/*.json
242+
if-no-files-found: ignore
243+
retention-days: 7
244+
245+
- name: Upload Electron failure artifacts
246+
if: failure()
247+
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
248+
with:
249+
name: desktop-release-smoke-failure-${{ matrix.shard }}
250+
path: apps/desktop/test-results/
251+
if-no-files-found: ignore
252+
retention-days: 7
253+
254+
# Restores no caches: every byte that gets signed comes from this checkout or
255+
# a fresh download, never from a cache another run could have written.
256+
build:
257+
name: build
258+
needs: source
259+
outputs:
260+
artifact_id: ${{ steps.artifacts.outputs.artifact-id }}
261+
runs-on: macos-26
262+
timeout-minutes: 60
263+
steps:
264+
- *checkout-source
265+
266+
- name: Setup Bun
267+
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
268+
with:
269+
bun-version: 1.4.4
270+
no-cache: true
271+
272+
- *setup-node
273+
274+
- *install-dependencies
275+
276+
- *inject-version
277+
278+
- name: Verify desktop source
279+
run: |
280+
bun run --cwd apps/desktop lint:check
281+
bun run --cwd apps/desktop type-check
282+
bun run --cwd apps/desktop test
283+
284+
- *bundle-desktop
199285

200286
- name: Write App Store Connect API key
201287
if: ${{ inputs.sign }}
@@ -253,8 +339,8 @@ jobs:
253339
APPLE_API_KEY_ID: ${{ secrets.APPLE_API_KEY_ID }}
254340
APPLE_API_ISSUER: ${{ secrets.APPLE_API_ISSUER }}
255341
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
256-
PRODUCT_NAME: ${{ steps.channel.outputs.name }}
257-
APP_ID: ${{ steps.channel.outputs.app_id }}
342+
PRODUCT_NAME: ${{ needs.source.outputs.name }}
343+
APP_ID: ${{ needs.source.outputs.app_id }}
258344
run: >
259345
bunx electron-builder --mac --publish never
260346
-c.productName="$PRODUCT_NAME" -c.appId="$APP_ID"
@@ -274,8 +360,8 @@ jobs:
274360
working-directory: apps/desktop
275361
env:
276362
CSC_IDENTITY_AUTO_DISCOVERY: 'false'
277-
PRODUCT_NAME: ${{ steps.channel.outputs.name }}
278-
APP_ID: ${{ steps.channel.outputs.app_id }}
363+
PRODUCT_NAME: ${{ needs.source.outputs.name }}
364+
APP_ID: ${{ needs.source.outputs.app_id }}
279365
run: >
280366
bunx electron-builder --mac --publish never -c.mac.notarize=false
281367
-c.mac.identity=- -c.mac.hardenedRuntime=false
@@ -333,9 +419,9 @@ jobs:
333419
- name: Validate signature and notarization
334420
if: ${{ inputs.sign }}
335421
env:
336-
APP_ID: ${{ steps.channel.outputs.app_id }}
422+
APP_ID: ${{ needs.source.outputs.app_id }}
337423
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
338-
PRODUCT_NAME: ${{ steps.channel.outputs.name }}
424+
PRODUCT_NAME: ${{ needs.source.outputs.name }}
339425
VERSION: ${{ inputs.version }}
340426
run: |
341427
SEMVER="${VERSION#v}"
@@ -403,13 +489,40 @@ jobs:
403489
fi
404490
SIM_DESKTOP_EXECUTABLE="$EXECUTABLE" bunx playwright test e2e/packaged-smoke.spec.ts
405491
492+
- name: Upload artifacts to the workflow run
493+
id: artifacts
494+
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
495+
with:
496+
name: sim-desktop-${{ inputs.version }}
497+
compression-level: 0
498+
path: |
499+
apps/desktop/release/*.dmg
500+
apps/desktop/release/*.zip
501+
apps/desktop/release/*.blockmap
502+
apps/desktop/release/latest-mac.yml
503+
retention-days: 7
504+
505+
publish:
506+
name: publish
507+
needs: [source, build, smoke]
508+
if: inputs.publish
509+
runs-on: ubuntu-24.04
510+
timeout-minutes: 10
511+
permissions:
512+
contents: write
513+
steps:
514+
- name: Download verified desktop artifacts
515+
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
516+
with:
517+
artifact-ids: ${{ needs.build.outputs.artifact_id }}
518+
path: apps/desktop/release
519+
406520
- name: Upload artifacts to the release
407-
if: ${{ inputs.publish }}
408521
env:
409522
DESKTOP_RELEASE_TOKEN: ${{ secrets.DESKTOP_RELEASE_TOKEN }}
410-
RELEASE_REPOSITORY: ${{ steps.channel.outputs.release_repository }}
523+
RELEASE_REPOSITORY: ${{ needs.source.outputs.release_repository }}
411524
SOURCE_RELEASE_TOKEN: ${{ github.token }}
412-
TOKEN_KIND: ${{ steps.channel.outputs.token_kind }}
525+
TOKEN_KIND: ${{ needs.source.outputs.token_kind }}
413526
VERSION: ${{ inputs.version }}
414527
run: |
415528
case "$TOKEN_KIND" in
@@ -448,7 +561,7 @@ jobs:
448561
local ARTIFACT="$1"
449562
local NAME SIZE DIGEST REMOTE REMOTE_SIZE REMOTE_DIGEST
450563
NAME="$(basename "$ARTIFACT")"
451-
SIZE="$(stat -f%z "$ARTIFACT")"
564+
SIZE="$(stat -c%s "$ARTIFACT")"
452565
DIGEST="sha256:$(shasum -a 256 "$ARTIFACT" | awk '{print $1}')"
453566
REMOTE="$(jq -c --arg name "$NAME" '.assets[] | select(.name == $name)' <<< "$RELEASE_JSON")"
454567
if [ -n "$REMOTE" ]; then
@@ -467,15 +580,3 @@ jobs:
467580
upload_or_verify "$ARTIFACT"
468581
done
469582
upload_or_verify "${ARTIFACTS[4]}"
470-
471-
- name: Upload artifacts to the workflow run
472-
if: ${{ !inputs.publish }}
473-
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
474-
with:
475-
name: sim-desktop-${{ inputs.version }}
476-
path: |
477-
apps/desktop/release/*.dmg
478-
apps/desktop/release/*.zip
479-
apps/desktop/release/*.blockmap
480-
apps/desktop/release/latest-mac.yml
481-
retention-days: 7

‎.github/workflows/docs-negotiation.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ on:
2020
- '.github/actions/setup/**'
2121
- '.github/scripts/stop-session.sh'
2222
- '.github/scripts/playwright-server.sh'
23+
- '.github/scripts/install-playwright.sh'
2324
push:
2425
branches: [staging]
2526
paths:
@@ -40,6 +41,7 @@ on:
4041
- '.github/actions/setup/**'
4142
- '.github/scripts/stop-session.sh'
4243
- '.github/scripts/playwright-server.sh'
44+
- '.github/scripts/install-playwright.sh'
4345
workflow_dispatch:
4446
inputs:
4547
benchmark:
@@ -375,7 +377,7 @@ jobs:
375377
if: matrix.group == 'chromium' || (matrix.group == 'webkit' && vars.CI_PROVIDER != '' && vars.CI_PROVIDER != 'blacksmith')
376378
env:
377379
BROWSER: ${{ matrix.group }}
378-
run: bunx --no-install playwright install --with-deps --only-shell "$BROWSER"
380+
run: bash .github/scripts/install-playwright.sh --only-shell "$BROWSER"
379381
- name: Verify production documentation
380382
id: acceptance
381383
env:

‎apps/desktop/README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,8 @@ The build also derives the app icon from `SIM_DESKTOP_DEFAULT_ORIGIN`. Every cha
108108

109109
CI (`.github/workflows/desktop-release.yml`, wired into `ci.yml`):
110110
- Stable builds run only after `release` on a `vX.Y.Z:` commit to main — **never before**: `scripts/create-single-release.ts` skips creation if the tag exists, so a desktop job publishing first would eat the changelog. Stable assets remain on `simstudioai/sim`; dev/staging assets publish to the public `simstudioai/sim-desktop-releases` repository so source-repository followers are not notified for internal shell builds. The job builds `--publish never`; reruns verify the size and SHA-256 digest of existing release assets instead of overwriting them.
111-
- **Secrets gate**: `desktop-signing` in `ci.yml` probes the six Apple secrets and skips the desktop job with a warning until they exist — releases never fail on a missing Apple account, and the first release after the secrets land ships desktop artifacts automatically. Manual/one-off builds: Actions → "desktop-release" → Run workflow with a `vX.Y.Z` version (`publish: false` uploads artifacts to the run instead of the release).
111+
- **Secrets gate**: `desktop-signing` in `ci.yml` probes the six Apple secrets and skips the desktop job with a warning until they exist — releases never fail on a missing Apple account, and the first release after the secrets land ships desktop artifacts automatically. Manual/one-off builds: Actions → "desktop-release" → Run workflow, select the `vX.Y.Z` tag under "Use workflow from", and enter the same version (`publish: false` uploads artifacts to the run instead of the release). Any other ref is refused, so a run only executes the tag's own code and workflow, and its caches stay scoped to that tag.
112+
- **Cache trust**: the signed `build` job restores no caches, so nothing it signs can come from a cache another run wrote. Only the unsigned smoke jobs use caches.
112113
- The product semver is **injected** from the release tag into `apps/desktop/package.json` at build time (repo package versions are placeholders). A mismatch guard fails the build.
113114
- Fuses are flipped at package time (`electronFuses` in `electron-builder.yml`): runAsNode off, NODE_OPTIONS off, inspect args off, ASAR-only + integrity validation, and cookie encryption on. The packaged smoke test asserts every fuse byte so Electron upgrades fail until new fuses receive an explicit policy.
114115
- **Cookie-encryption go/no-go**: on every Electron bump, verify a packaged build keeps its session across relaunch (there are historical cookie-persistence bugs with the `EnableCookieEncryption` fuse). If it reproduces, set `enableCookieEncryption: false` and record it here.

0 commit comments

Comments
 (0)