[Bug]: specify preset set-priority does not update agent command files that are already written #5087

Bug Description

When two installed presets override the same command, specify preset set-priority changes which one wins resolution, but the agent's command file keeps the previous winner's content. preset_set_priority updates the registry and then calls only reconcile_constitution (src/specify_cli/presets/command_set_priority.py:55-59). Install and remove also call _reconcile_composed_commands and _reconcile_skills "so that install order doesn't determine the winning command file" (src/specify_cli/presets/_manager.py:479-489, :899-912). After set-priority, specify preset resolve therefore shows one winner while the agent runs the other.

The documentation only names install, remove and integration use as the times command files are rewritten (docs/reference/presets.md, "File Resolution"), and does not mention set-priority. Maintainers may therefore treat this as a documentation gap rather than a code bug. Either way, the current behaviour is not described anywhere, and the set-priority section reads as if it applies to every file a preset provides.

Steps to Reproduce

From a fresh clone after uv sync, save the following as repro.py and run uv run python -I repro.py:

import json, os, pathlib, tempfile
from typer.testing import CliRunner
from specify_cli import app

base = pathlib.Path(tempfile.mkdtemp())
proj = base / "proj"
(proj / ".specify" / "templates" / "commands").mkdir(parents=True)
(proj / ".specify" / "templates" / "commands" / "demo.md").write_text(
    "---\ndescription: core\n---\nCORE\n")
(proj / ".specify" / "init-options.json").write_text(json.dumps({"ai": "gemini"}))
(proj / ".gemini" / "commands").mkdir(parents=True)

def make_preset(pid, body):
    src = base / pid
    (src / "commands").mkdir(parents=True)
    (src / "commands" / "speckit.demo.md").write_text(f"---\ndescription: {pid}\n---\n{body}\n")
    (src / "preset.yml").write_text(
        'schema_version: "1.0"\n'
        f'preset:\n  id: {pid}\n  name: {pid}\n  version: "1.0.0"\n  description: d\n'
        'requires:\n  speckit_version: ">=0.0.1"\n'
        'provides:\n  templates:\n    - type: command\n      name: speckit.demo\n'
        '      file: commands/speckit.demo.md\n')
    return src

os.chdir(proj)
run = lambda *a: CliRunner().invoke(app, list(a))
agent_file = proj / ".gemini" / "commands" / "speckit.demo.toml"
body = lambda: "A" if "BODY-FROM-A" in agent_file.read_text() else "B"
print("add pa:", run("preset", "add", "--dev", str(make_preset("pa", "BODY-FROM-A")), "--priority", "10").exit_code)
print("add pb:", run("preset", "add", "--dev", str(make_preset("pb", "BODY-FROM-B")), "--priority", "20").exit_code)
print("agent file body after installs:", body())
print("set-priority pb 1:", run("preset", "set-priority", "pb", "1").exit_code)
r = run("preset", "resolve", "speckit.demo")
print("resolve:", [l.strip() for l in r.output.splitlines() if "from:" in l])
print("agent file body after set-priority:", body())

Expected Behavior

docs/reference/presets.md:229 says set-priority "Changes the resolution priority of an installed preset. Lower numbers take precedence. When multiple presets provide the same file, the one with the lowest priority number wins." The FAQ (docs/reference/presets.md:416) says to "Use specify preset set-priority to control the order". After set-priority pb 1, .gemini/commands/speckit.demo.toml should contain pb's body. If that is not intended, the docs should say that command files keep their content until the next install or removal, as the disable FAQ already does for preset disable (docs/reference/presets.md:424).

Actual Behavior

preset resolve reports pb as the top layer, but the Gemini command file still contains pa's body:

add pa: 0
add pb: 0
agent file body after installs: A
set-priority pb 1: 0
resolve: ['(top layer from: pb v1.0.0)']
agent file body after set-priority: A

Specify CLI Version

1.2.4.dev0 (main at 27d14e2)

AI Agent

Gemini CLI

Operating System

Linux 6.18 (Ubuntu container)

Python Version

Python 3.12.3

Additional Context

Running the same command and skill reconciliation that install and remove use, for the commands the re-prioritised preset provides, would bring set-priority into line with them.

Related: a test docstring in #4709 notes that preset commands "stay stale until removal", but that pull request does not change set-priority.

Also reproduced on macOS (Darwin 25.6, arm64) with Python 3.14.4 and uv run python -I repro.py against 27d14e2; the output matches the above.

AI Disclosure

Found and reproduced with Claude Code (model: Claude Opus 4.8), run autonomously under human supervision. The human reporter re-ran every reproduction above before filing.