Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/pgpm/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ pgpm handles extension creation during deploy. Declare extensions in your `.cont
| `pgpm tag <version>` | Tag current state for targeted deploys |
| `pgpm install <module>` | Install a pgpm module dependency |
| `pgpm extension [--add/--remove/--set a,b]` | Manage module dependencies (flags are non-interactive; no flags = interactive picker) |
| `pgpm test-packages` | Test all packages |
| `pgpm test-packages` | Test the minimal covering set (`--force-all` for every module) |
| `pgpm test-packages --full-cycle` | Test deploy → verify → revert → redeploy |
| `pgpm docker start` | Start PostgreSQL container |
| `pgpm docker stop` | Stop PostgreSQL container |
Expand Down
4 changes: 3 additions & 1 deletion .agents/skills/pgpm/references/ci-cd.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,9 @@ For running pgpm's built-in integration tests:
run: pgpm test-packages
```

This runs all package tests defined in the pgpm workspace.
This tests the minimal covering set — the modules nothing else in the workspace
requires — which deploys every module through those dependency closures. Add
`--force-all` to give every module its own database instead.

## SDK Generation Workflow

Expand Down
13 changes: 11 additions & 2 deletions .agents/skills/pgpm/references/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,12 +241,21 @@ pgpm package --check --no-fail-fast # list all drift instead of stopping

### Testing

**pgpm test-packages** — Run integration tests on all modules in workspace
**pgpm test-packages** — Run integration tests on the workspace's modules

Defaults to the minimal covering set: the modules nothing else in the workspace
requires. Testing one deploys its whole dependency closure, so every module is
still exercised — once per covering module rather than once per module.
`--force-all` restores a database per module, which is the only way to assert a
module's own `requires` is complete rather than satisfied by a sibling.

```bash
# Deploy only
# Deploy only, minimal covering set
pgpm test-packages

# Every module in its own database
pgpm test-packages --force-all

# Full deploy/verify/revert/deploy cycle
pgpm test-packages --full-cycle

Expand Down
4 changes: 4 additions & 0 deletions .agents/skills/pgpm/references/deploy-lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,10 @@ pgpm test-packages --full-cycle
pgpm test-packages --full-cycle --workspace --all
```

By default it tests the minimal covering set (the modules nothing else requires),
which still deploys every module through those closures; `--force-all` gives every
module its own database.

This is the gold standard for validating migrations — it proves:
1. Deploy scripts apply correctly
2. Verify scripts confirm the deployed state
Expand Down
17 changes: 15 additions & 2 deletions pgpm/cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -376,12 +376,24 @@ pgpm kill --no-drop

#### `pgpm test-packages`

Run integration tests on all modules in a workspace. Creates a temporary database for each module, deploys, and optionally runs verify/revert/deploy cycles.
Run integration tests on the modules in a workspace. Creates a temporary database per tested module, deploys, and optionally runs verify/revert/deploy cycles.

By default only the **minimal covering set** is tested: the modules nothing else in the
workspace requires. Testing one of those deploys its whole dependency closure in
dependency order, so every module is still exercised — once per covering module instead
of once per module. Excluding a covering module promotes whatever it alone covered.

The property the default trades away is standalone deployability — that a module's own
`requires` is complete rather than satisfied by a sibling in the same database. Use
`--force-all` to get it back.

```bash
# Test all modules in workspace (deploy only)
# Test the minimal covering set (deploy only)
pgpm test-packages

# Test every module in its own database
pgpm test-packages --force-all

# Run full deploy/verify/revert/deploy cycle
pgpm test-packages --full-cycle

Expand All @@ -397,6 +409,7 @@ pgpm test-packages --full-cycle --continue-on-fail --exclude legacy-module

**Options:**

- `--force-all` - Test every module in its own database instead of the minimal covering set
- `--full-cycle` - Run full deploy/verify/revert/deploy cycle (default: deploy only)
- `--continue-on-fail` - Continue testing all packages even after failures (default: stop on first failure)
- `--exclude <modules>` - Comma-separated module names to exclude
Expand Down
53 changes: 53 additions & 0 deletions pgpm/cli/__tests__/test-packages-minimal-set.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
import { minimalCoveringSet } from '../src/commands/test-packages';

const moduleMap: Record<string, { requires: string[] }> = {
plpgsql: { requires: [] },
ast: { requires: ['plpgsql'] },
'ast-plpgsql': { requires: ['ast'] },
metaschema: { requires: ['ast-plpgsql'] },
app: { requires: ['metaschema'] },
seeds: { requires: ['app'] },
standalone: { requires: [] }
};

const names = Object.keys(moduleMap);

describe('minimalCoveringSet', () => {
it('keeps only the modules nothing else requires', () => {
const { selected, coveredBy } = minimalCoveringSet(moduleMap, names);

expect(selected).toEqual(['seeds', 'standalone']);
expect(coveredBy.has('ast')).toBe(true);
expect(coveredBy.has('standalone')).toBe(false);
});

it('promotes a module whose only dependent was excluded', () => {
const { selected } = minimalCoveringSet(
moduleMap,
names.filter((name) => name !== 'seeds')
);

expect(selected).toEqual(['app', 'standalone']);
});

it('promotes a dependency shared by two excluded dependents', () => {
const { selected } = minimalCoveringSet(
{ ...moduleMap, other: { requires: ['metaschema'] } },
names.filter((name) => name !== 'seeds' && name !== 'app')
);

expect(selected).toEqual(['metaschema', 'standalone']);
});

it('covers every candidate through the selected closures', () => {
const { selected, coveredBy } = minimalCoveringSet(moduleMap, names);

expect(selected.length + coveredBy.size).toBe(names.length);
});

it('selects a module with no dependents even when it is also a dependency of nothing', () => {
const { selected } = minimalCoveringSet(moduleMap, ['plpgsql', 'standalone']);

expect(selected).toEqual(['plpgsql', 'standalone']);
});
});
109 changes: 102 additions & 7 deletions pgpm/cli/src/commands/test-packages.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { PgpmPackage } from '@pgpmjs/core';
import { PgpmPackage, resolveExtensionDependencies } from '@pgpmjs/core';
import { getEnvOptions } from '@pgpmjs/env';
import { Logger } from '@pgpmjs/logger';
import { CLIOptions, Inquirerer, ParsedArgs } from 'inquirerer';
Expand All @@ -19,18 +19,25 @@ Test Packages Command:

pgpm test-packages [OPTIONS]

Run integration tests on all PGPM packages in a workspace.
Run integration tests on the PGPM packages in a workspace.
Tests each package with a deploy/verify/revert/deploy cycle.

By default only the minimal covering set is tested: the modules nothing else
in the workspace requires. Deploying one of those deploys everything it
requires first, so every module in the workspace is still exercised — just
once per covering module instead of once per module.

Options:
--help, -h Show this help message
--exclude <pkgs> Comma-separated module names to exclude
--continue-on-fail Continue testing all packages even after failures
--full-cycle Run full deploy/verify/revert/deploy cycle (default: deploy only)
--force-all Test every module in its own database (pre-minimal-set behavior)
--cwd <directory> Working directory (default: current directory)

Examples:
pgpm test-packages Test all packages (stops on first failure)
pgpm test-packages Test the minimal covering set
pgpm test-packages --force-all Test every module in its own database
pgpm test-packages --full-cycle Run full test cycle with verify/revert
pgpm test-packages --continue-on-fail Test all packages, collect all failures
pgpm test-packages --exclude my-module Exclude specific modules
Expand All @@ -43,6 +50,83 @@ interface TestResult {
error?: string;
}

/**
* Reduce the module list to a minimal covering set: the modules that no other
* candidate requires, transitively.
*
* Testing a module deploys its whole dependency closure in dependency order, so
* a module that some other candidate requires is already deployed, verified,
* reverted and re-deployed inside that candidate's database. Selection runs over
* the post-exclude candidates, so excluding a covering module promotes whatever
* it alone covered rather than dropping it.
*
* The property this trades away is standalone deployability — that a module's own
* `requires` is complete rather than satisfied by a sibling in the same database.
* Use --force-all to get it back.
*/
export function minimalCoveringSet(
moduleMap: Record<string, { requires: string[] }>,
candidateNames: string[]
): { selected: string[]; coveredBy: Map<string, string> } {
const candidates = new Set(candidateNames);

// name -> its transitive requires, restricted to the candidates
const closures = new Map<string, string[]>();
for (const name of candidates) {
const { resolved } = resolveExtensionDependencies(name, moduleMap);
closures.set(
name,
resolved.filter((dep) => dep !== name && candidates.has(dep))
);
}

const coveredBy = new Map<string, string>();
for (const [name, deps] of closures) {
for (const dep of deps) {
if (!coveredBy.has(dep)) {
coveredBy.set(dep, name);
}
}
}

const selected = candidateNames.filter((name) => !coveredBy.has(name));

// Nothing may fall out of the selection. Covering-set membership is derived
// from pairwise closures, so assert the union actually reaches every candidate
// rather than trusting that argument.
const covered = new Set<string>();
for (const name of selected) {
covered.add(name);
for (const dep of closures.get(name) ?? []) {
covered.add(dep);
}
}
const uncovered = candidateNames.filter((name) => !covered.has(name));
if (uncovered.length > 0) {
throw new Error(
`Minimal module selection left ${uncovered.length} module(s) untested: ${uncovered.join(', ')}. ` +
'Re-run with --force-all and report this as a pgpm bug.'
);
}

return { selected, coveredBy };
}

function selectMinimalModules(
workspacePkg: PgpmPackage,
candidates: PgpmPackage[]
): { selected: PgpmPackage[]; coveredBy: Map<string, string> } {
const { selected, coveredBy } = minimalCoveringSet(
workspacePkg.getModuleMap(),
candidates.map((mod) => mod.getModuleName())
);
const selectedNames = new Set(selected);
return {
selected: candidates.filter((mod) => selectedNames.has(mod.getModuleName())),
coveredBy
};
}

function dbSafeName(moduleName: string): string {
return `test_${moduleName}`.replace(/[^a-zA-Z0-9]/g, '_').toLowerCase();
}
Expand Down Expand Up @@ -237,6 +321,7 @@ export default async (
const continueOnFail = argv['continue-on-fail'] === true || argv.continueOnFail === true;
const stopOnFail = !continueOnFail;
const fullCycle = argv['full-cycle'] === true || argv.fullCycle === true;
const forceAll = argv['force-all'] === true || argv.forceAll === true;
const cwd = argv.cwd || process.cwd();

// Parse excludes
Expand All @@ -246,7 +331,7 @@ export default async (
}

console.log('=== PGPM Package Integration Test ===');
console.log(`Testing all packages with ${fullCycle ? 'deploy/verify/revert/deploy cycle' : 'deploy only'}`);
console.log(`Testing ${forceAll ? 'all packages' : 'the minimal covering set'} with ${fullCycle ? 'deploy/verify/revert/deploy cycle' : 'deploy only'}`);
if (!stopOnFail) {
console.log('Mode: Test all packages (collect all failures)');
}
Expand Down Expand Up @@ -292,16 +377,26 @@ export default async (
console.log(`Excluding: ${excludes.join(', ')}`);
}

console.log(`Found ${filteredModules.length} modules to test:`);
for (const mod of filteredModules) {
let selectedModules = filteredModules;
if (!forceAll) {
const { selected, coveredBy } = selectMinimalModules(workspacePkg, filteredModules);
selectedModules = selected;
console.log(
`Minimal covering set: ${selected.length} of ${filteredModules.length} modules ` +
`(${coveredBy.size} covered transitively; --force-all to test each on its own)`
);
}

console.log(`Found ${selectedModules.length} modules to test:`);
for (const mod of selectedModules) {
console.log(` - ${mod.getModuleName()}`);
}
console.log('');

const failedPackages: TestResult[] = [];
const successfulPackages: TestResult[] = [];

for (const modulePkg of filteredModules) {
for (const modulePkg of selectedModules) {
const result = await testModule(workspacePkg, modulePkg, fullCycle);

if (result.success) {
Expand Down
Loading