Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ Full guide in `CONTRIBUTING.md`; the operationally important parts (the ones tha

- **Branch off `dev`** (the default branch) and open PRs **against `dev`**. `production` is the maintainer-only stable track — never target it directly.
- PRs are **squash-merged**, so the **PR title becomes the commit** and must be a [Conventional Commit](https://www.conventionalcommits.org). The title — not the branch commits — is what release-please reads to compute the next version, so it matters even though individual commits are squashed away. A `PR Title` CI check enforces it.
- Type → bump (pre-1.0, so `feat` and breaking `!` both bump **minor**): `feat` minor; `fix`/`perf`/`deps`/`revert`/`refactor` patch; `docs`/`chore`/`test`/`ci`/`build`/`style` are hidden and bump nothing. Allowed scopes are free-form.
- Type → bump: `feat` **minor**; `fix`/`perf`/`deps`/`revert`/`refactor` **patch**; `docs`/`chore`/`test`/`ci`/`build`/`style` are hidden and bump nothing. Allowed scopes are free-form.
- ⚠️ **A `!` (or `BREAKING CHANGE:` footer) bumps the MAJOR — do not use it casually.** The configs set `bump-minor-pre-major: true`, but that only applies **below 1.0.0**; we are on 5.x, so it is inert and a breaking marker means exactly what semver says. A `refactor(cloud)!:` PR title once produced a `6.0.0-beta.1` release PR for what was only a flag rename in an unconsumed beta. Because PRs are squash-merged, **the PR title IS the commit** — the `!` lands even if no branch commit carried it.
- **Never hand-edit `package.json` version, `CHANGELOG.md`, or the `.release-please-manifest*.json` files** — release-please owns all of them. `src/types/generated/schema.types.ts` is likewise generated (openapi-typescript).
- A first-time contributor must sign the CLA (the CLA Assistant bot comments on the first PR); the CLA check must be green to merge.
- **CI (`.github/workflows/cli-ci.yml`) runs on every PR** including forks: gitleaks secret scan, `pnpm lint`, `pnpm typecheck`, `pnpm build`, `pnpm audit --audit-level moderate`. The **integration tests need the private `devicecloud-dev/dcd` mock-api** (cloned via the `DCD_SSH_DEPLOY_KEY` secret), and GitHub withholds secrets from fork and Dependabot PRs — so `pnpm test` is **skipped there** and a maintainer runs the full suite before merge. gitleaks also runs as a pre-commit hook (allowlist in `.gitleaks.toml`); without the binary installed the hook self-skips and CI is the backstop.
Expand Down
84 changes: 84 additions & 0 deletions src/commands/cloud.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import {
import { MoropoService } from '../services/moropo.service.js';
import { ReportDownloadService } from '../services/report-download.service.js';
import {
deviceFromResultRow,
ResultsPollingService,
RunFailedError,
} from '../services/results-polling.service.js';
Expand All @@ -31,8 +32,14 @@ import {
EAndroidDevices,
EiOSDevices,
EiOSVersions,
isIosMatrixConfig,
} from '../types/domain/device.types.js';
import { resolveAuth } from '../utils/auth.js';
import {
assertMatrixSupported,
matrixIsIos,
parseDeviceMatrix,
} from '../utils/device-matrix.js';
import { detectCiContext, isCI } from '../utils/ci.js';
import {
CliError,
Expand Down Expand Up @@ -211,6 +218,10 @@ export const cloudCommand = defineCommand({
'android-device',
);
const androidNoSnapshot = Boolean(args['android-no-snapshot']);
// Repeatable device-matrix flags: one validated cell each, no cross-product.
const iosMatrixFlags = collectRepeatedFlag(rawArgs, ['--ios-device-matrix']);
const androidMatrixFlags = collectRepeatedFlag(rawArgs, ['--android-device-matrix']);
const deviceMatrix = parseDeviceMatrix(iosMatrixFlags, androidMatrixFlags);
const json = Boolean(args.json);
const jsonFileFlag = Boolean(args['json-file']);
const jsonFileName = args['json-file-name'] as string | undefined;
Expand Down Expand Up @@ -502,6 +513,28 @@ export const cloudCommand = defineCommand({
{ debug, logger: (m: string) => out(m) },
);

// Validate every device-matrix cell up front — each on its own, against
// the same compatibility matrix — so an unsupported cell fails fast,
// naming it, before anything is uploaded.
for (const cfg of deviceMatrix) {
if (isIosMatrixConfig(cfg)) {
deviceValidationService.validateiOSDevice(
cfg.iOSVersion,
cfg.iOSDevice,
compatibilityData,
{ debug, logger: (m: string) => out(m) },
);
} else {
deviceValidationService.validateAndroidDevice(
cfg.androidApiLevel,
cfg.androidDevice,
cfg.googlePlay ?? googlePlay,
compatibilityData,
{ debug, logger: (m: string) => out(m) },
);
}
}

if (maestroChromeOnboarding && !androidApiLevel && !androidDevice) {
warnOut(
'The --maestro-chrome-onboarding flag only applies to Android tests and will be ignored for iOS tests.',
Expand Down Expand Up @@ -639,6 +672,8 @@ export const cloudCommand = defineCommand({
'include-tags': includeTags,
'exclude-tags': excludeTags,
'exclude-flows': excludeFlows,
'ios-device-matrix': iosMatrixFlags,
'android-device-matrix': androidMatrixFlags,
};
for (const [k, v] of Object.entries(args)) {
if (!canonicalFlagKeys.has(k)) continue;
Expand Down Expand Up @@ -762,6 +797,7 @@ export const cloudCommand = defineCommand({
continueOnFailure,
debug,
deviceLocale,
deviceMatrix,
env,
executionPlan,
flowFile,
Expand All @@ -784,6 +820,53 @@ export const cloudCommand = defineCommand({
disableAnimations,
});

// Device-matrix cost preview: the server prices the exact fan-out (quote
// == charge) so the user sees the cell count and estimated cost before the
// flow zip is uploaded. An unsupported cell fails fast here. Skipped when
// there is no matrix, and tolerant of older APIs that lack the endpoint.
if (deviceMatrix.length > 0) {
const estimate = await ApiGateway.estimateMatrix(apiUrl, auth, fields);
// A null estimate means the API predates the matrix (404/405). It would
// silently strip deviceMatrix and run one device — refuse rather than
// hand back a green single-device run the user reads as a matrix.
assertMatrixSupported(deviceMatrix, estimate);
if (estimate) {
const osPrefix = matrixIsIos(deviceMatrix) ? 'iOS' : 'API';
const rows = ui.fields([
['cells', colors.highlight(String(estimate.cellCount))],
['est. cost', colors.highlight(`$${estimate.totalCost.toFixed(2)}`)],
]);
for (const col of estimate.columns) {
const label = [
col.deviceName,
col.osVersion && `${osPrefix} ${col.osVersion}`,
col.googlePlay && 'Play',
]
.filter(Boolean)
.join(' · ');
rows.push(
...ui.fields([
[
label,
colors.dim(
`${col.flowCount} flow${col.flowCount === 1 ? '' : 's'} · $${col.cost.toFixed(2)}`,
),
],
]),
);
}
if (estimate.excludedFlows.length > 0) {
rows.push(
colors.dim(
`${estimate.excludedFlows.length} flow${estimate.excludedFlows.length === 1 ? '' : 's'} target their own device (excluded from the matrix)`,
),
);
}
out(ui.section('Device matrix'));
out(ui.branch(rows));
}
}

// New path: upload the zip directly to storage, then submit a JSON test
// referencing it. Older API deployments lack these endpoints — a real API
// 404s (route undefined), some proxies 405 (path/method not allowed); in
Expand Down Expand Up @@ -865,6 +948,7 @@ export const cloudCommand = defineCommand({
consoleUrl: url,
status: 'PENDING',
tests: results.map((r) => ({
device: deviceFromResultRow(r),
fileName: r.test_file_name,
flowName:
testMetadataMap[r.test_file_name]?.flowName ||
Expand Down
8 changes: 8 additions & 0 deletions src/config/flags/device.flags.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,14 @@ export const deviceFlags = {
type: 'string',
description: `[iOS only] iOS version to run your flow against (options: ${iosVersions})`,
},
'ios-device-matrix': {
type: 'string',
description: `[iOS only] Device-matrix cell as <device>:<version>, e.g. iphone-16:18. Repeatable — every flow runs once per cell (no cross-product). Cannot be combined with --android-device-matrix.`,
},
'android-device-matrix': {
type: 'string',
description: `[Android only] Device-matrix cell as <device>:<apiLevel> (append :play for Google Play), e.g. pixel-7:34 or pixel-7:34:play. Repeatable — every flow runs once per cell (no cross-product). Cannot be combined with --ios-device-matrix.`,
},
orientation: {
type: 'string',
description:
Expand Down
46 changes: 46 additions & 0 deletions src/gateways/api-gateway.ts
Original file line number Diff line number Diff line change
Expand Up @@ -588,6 +588,52 @@ export const ApiGateway = {
}
},

/**
* Dry-run cost + cell-count estimate for a (possibly device-matrix)
* submission. Runs the same resolve → validate → fan-out → price core as the
* submit path server-side, without persisting. The preview is best-effort:
* an API that predates this endpoint 404s (route undefined) or 405s (route
* matched a sibling path, no POST) — in either case return null so the caller
* proceeds without a preview. A 400 (an invalid config) still surfaces as a
* normal API error so the CLI can fail fast before uploading the flow zip.
*/
async estimateMatrix(baseUrl: string, auth: AuthContext, body: Record<string, unknown>) {
try {
const res = await fetch(`${baseUrl}/uploads/estimateMatrix`, {
body: JSON.stringify(body),
headers: {
'content-type': 'application/json',
...auth.headers,
},
method: 'POST',
});
if (res.status === 404 || res.status === 405) {
return null;
}
if (!res.ok) {
await this.handleApiError(res, 'Failed to estimate device matrix');
}
return await parseJsonResponse<{
cellCount: number;
totalCost: number;
excludedFlows: string[];
columns: Array<{
deviceName: string;
osVersion: string;
googlePlay: boolean;
flowCount: number;
cost: number;
}>;
}>(res, 'Failed to estimate device matrix');
} catch (error) {
if (error instanceof TypeError && error.message === 'fetch failed') {
throw this.enhanceFetchError(error, `${baseUrl}/uploads/estimateMatrix`);
}

throw error;
}
},


/**
* Generic report download method that handles both junit and allure reports
Expand Down
42 changes: 42 additions & 0 deletions src/services/results-polling.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,46 @@ export interface TestMetadata {
tags: string[];
}

/**
* The device a result ran on. Additive: single-device runs are unchanged, and
* a device matrix disambiguates two `tests[]` entries that share a `name` (the
* same flow on two devices) by their device.
*/
export interface TestDevice {
googlePlay?: boolean;
name?: string;
osVersion?: string;
}

/**
* Structured device for a result row. Prefers the friendly deviceName/osVersion
* the per-flow targeting / matrix fan-out stamps onto each result's config
* (#1097), falling back to the raw simulator_name for older rows. Returns
* undefined when neither is present, so single-device runs that predate the
* field simply omit `device`. Shared by the sync polling path and the async
* (--async --json) path so both emit an identical device shape.
*/
export function deviceFromResultRow(r: {
config?: unknown;
simulator_name?: string | null;
}): TestDevice | undefined {
const config = (r.config ?? {}) as { deviceName?: string; osVersion?: string };
const sim = r.simulator_name ?? undefined;
const name = config.deviceName ?? sim;
if (!name && !config.osVersion) return undefined;
return {
name,
osVersion: config.osVersion,
googlePlay: sim ? /(_PLAY|-play)$/.test(sim) : undefined,
};
}

export interface PollingResult {
consoleUrl: string;
status: 'FAILED' | 'PASSED';
tests: Array<{
/** Device this result ran on (present when the API reports it). */
device?: TestDevice;
durationSeconds: null | number;
failReason?: string;
/** File path of the test (same as name, for clarity) */
Expand Down Expand Up @@ -311,6 +347,12 @@ export class ResultsPollingService {
? 'PASSED'
: 'FAILED',
tests: resultsWithoutEarlierTries.map((r) => ({
// r carries config/simulator_name at runtime; the committed generated
// types lag the API (regenerated wholesale from dev's swagger), so read
// them through the helper's structural type.
device: deviceFromResultRow(
r as { config?: unknown; simulator_name?: string | null },
),
durationSeconds: r.duration_seconds ?? null,
failReason:
r.status === 'FAILED' ? r.fail_reason || 'No reason provided' : undefined,
Expand Down
21 changes: 20 additions & 1 deletion src/services/test-submission.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { createHash } from 'node:crypto';
import * as path from 'node:path';

import { compressFilesFromRelativePath } from '../methods.js';
import { DeviceMatrixConfig } from '../types/domain/device.types.js';
import { toPortableRelativePath } from '../utils/paths.js';
import { IExecutionPlan } from './execution-plan.service.js';

Expand All @@ -15,6 +16,7 @@ export interface TestSubmissionConfig {
continueOnFailure?: boolean;
debug?: boolean;
deviceLocale?: string;
deviceMatrix?: DeviceMatrixConfig[];
disableAnimations?: boolean;
env?: string[];
executionPlan: IExecutionPlan;
Expand Down Expand Up @@ -85,6 +87,7 @@ export class TestSubmissionService {
maestroChromeOnboarding,
raw,
disableAnimations,
deviceMatrix,
debug = false,
logger,
} = config;
Expand Down Expand Up @@ -183,7 +186,23 @@ export class TestSubmissionService {
// Note: googlePlay is now included in configPayload below instead of as a separate field
// to work around a FormData parsing issue in the API

const targetPlatform = iOSDevice || iOSVersion ? 'ios' : 'android';
// Explicit device matrix (one upload, N cells). Only sent when present, so
// single-device submissions stay byte-identical.
if (deviceMatrix && deviceMatrix.length > 0) {
fields.deviceMatrix = JSON.stringify(deviceMatrix);
}

// Platform used only to pick which workspace-config disableAnimations flag
// applies. A device matrix is single-platform; its first cell decides. Fall
// back to the scalar iOS flags for single-device submissions.
const matrixPlatform =
deviceMatrix && deviceMatrix.length > 0
? 'iOSDevice' in deviceMatrix[0]
? 'ios'
: 'android'
: undefined;
const targetPlatform =
matrixPlatform ?? (iOSDevice || iOSVersion ? 'ios' : 'android');
const configYamlDisableAnimations =
targetPlatform === 'ios'
? Boolean(workspaceConfig?.platform?.ios?.disableAnimations)
Expand Down
14 changes: 14 additions & 0 deletions src/types/domain/device.types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,17 @@ export enum EAndroidApiLevels {
'thirtyTwo' = '32',
'twentyNine' = '29',
}

/**
* One explicit device-matrix cell. iOS entries carry {iOSDevice, iOSVersion};
* Android entries carry {androidDevice, androidApiLevel} plus an optional Play
* channel. Sent to the API as the `deviceMatrix` array; every non-targeted flow
* runs once per cell. There is no cross-product — each entry is one cell.
*/
export type DeviceMatrixConfig =
| { iOSDevice: string; iOSVersion: string }
| { androidApiLevel: string; androidDevice: string; googlePlay?: boolean };

export const isIosMatrixConfig = (
c: DeviceMatrixConfig,
): c is { iOSDevice: string; iOSVersion: string } => 'iOSDevice' in c;
Loading
Loading