Skip to content

docs(ai-policy): use AI-Assisted-By with model and reasoning effort - #817

Open
MiMoHo wants to merge 1 commit into
nextcloud:masterfrom
MiMoHo:docs/ai-policy-trailer-model-effort
Open

MiMoHo wants to merge 1 commit into
nextcloud:masterfrom
MiMoHo:docs/ai-policy-trailer-model-effort

Conversation

@MiMoHo

@MiMoHo MiMoHo commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

The policy asks for Assisted-by: AGENT_NAME:MODEL_VERSION in every AI-assisted commit. This PR asks for AI-Assisted-By: MODEL (REASONING_EFFORT) instead, for example AI-Assisted-By: Claude Opus 4.5 (medium): the model and, where the tool offers one, the reasoning effort. The agent no longer needs to be named.

The reasoning effort tells a reviewer more than the agent does, because the same model gives measurably different results depending on how much it is allowed to think. Anthropic reports that Opus 4.5 at medium effort matches Sonnet 4.5's best score on SWE-bench Verified and only pulls ahead, by 4.3 points, at its highest effort (announcement). Peer-reviewed work points the same way. Snell et al. (ICLR 2025) found that on problems a smaller model can partly solve, extra compute at inference time can beat a 14 times larger model. In Wu et al. (ICLR 2025), a 7B model with a better search strategy consistently outperformed a 34B model on the MATH benchmark. A strong model on low effort can do worse than a mid-size model on high effort, so the model name alone is only half the information.

The agent, on the other hand, mostly describes the contributor's setup, and that is what attackers care about. Coding agents run with access to the shell, the source code and often credentials. In the Nx supply chain attack, malicious packages used locally installed AI command line tools to search machines for secrets. Invariant Labs showed how a crafted GitHub issue can take over a coding agent and make it leak private repositories. A permanent, public record of which agent each GitHub user runs makes it easier to pick targets and to tailor such attacks.

The Linux kernel went further and dropped both agent and model in August 2026, because they add "little or no useful information" (commit, docs). I think the model and the effort are worth keeping for the reasons above. The agent is not.

The key changes too. Assisted-by: alone doesn't say what kind of help it was, and the kernel uses the same tag to list analysis tools without AI, such as coccinelle and sparse (docs). This PR uses AI-Assisted-By: for AI and leaves Assisted-by: for tools without AI, such as code generators. The workflow already recognises AI-Assisted-By: and no longer counts Assisted-by: as AI. Older commits don't need to be rewritten.

Some objections I expect:

  • "The agent affects quality too." It can. SWE-agent (NeurIPS 2024) showed that the interface an agent gets changes how many issues it solves. But a name like "Copilot" doesn't say which version, settings, tools or prompts were used, so a reviewer can't judge much from it. Under this policy a human reviews, cleans up and tests everything before submitting anyway, and anyone who wants to name the agent can still do so in the PR description.
  • "More effort isn't always better." True. Gema et al. (TMLR 2025) built tasks where longer reasoning makes models worse. That is a reason to record the effort, not to leave it out.
  • "The tool's terms matter for licensing." The model already names the provider whose terms apply. The licensing section makes checking them the contributor's job ("Ignorance of AI-generated provenance is not a defense"), and so does the Linux Foundation guidance.
  • "@llvm asks for the name of the code assistant." LLVM says its labelling is "intended to facilitate reviews, and not to track which parts of LLVM are generated". Model and effort serve that purpose better than the agent's name.
  • "Not every tool has a reasoning effort setting." Then the model alone is enough, for example AI-Assisted-By: GPT-4o.
  • "The kernel and other projects use Assisted-by: for AI." The kernel's tag mixes the model with other tools, so the key alone can't tell them apart. Here the key says it.
  • "Older PRs with Assisted-by: lose the label." They keep the AI checkbox from the PR template and the disclosure in the description, which the policy requires anyway. The label helps with sorting; the disclosure is the checkbox and the trailer.

CONTRIBUTING.md, the agent snippet and the AI Policy workflow now use the new format, both in this repository and in the template. The check looks for AI-assistant: and AI-Assisted-By:. Repositories that still run an older copy of the workflow, nextcloud/notes-android for example, need the updated template to recognise the new key.

I also fixed two examples: devstral-small-2507 is called Devstral Small 1.1 in its model card, and I couldn't find a Qwen3-Coder model with 32B parameters, so that example now uses Qwen3-Coder-30B-A3B-Instruct.

Like #816, this PR reduces what contributors have to make public about themselves: there their real name, here the tools they run. Each of the two can be merged on its own.

If the AI Policy check fails on this PR, that is #767; #815 fixes it.

🤖 AI (if applicable)

  • The content of this PR was assisted using AI

AI-Assisted-By: Claude Opus 5.5 (xhigh)

The model helped me draft this change. I reviewed and edited everything. The commit carries the same line as a trailer.

The policy asks for Assisted-by: AGENT_NAME:MODEL_VERSION. The agent
name mostly describes the contributor's environment. It tells reviewers
little and adds to a public record of which tools each contributor
runs, which helps attackers more than reviewers.

Ask for AI-Assisted-By: MODEL (REASONING_EFFORT) instead, for example
"AI-Assisted-By: Claude Opus 4.5 (medium)". The same model performs
measurably differently depending on the effort, so the pair says more
about a contribution than the agent does. Contributors can still name
the agent in the PR description.

The key now says that the help came from AI. Assisted-by: alone does
not, and the Linux kernel lists non-AI tools in the same tag. Keep
Assisted-by: for tools without AI and stop the AI Policy workflow from
counting it as AI. Older commits keep their trailers, and the AI
checkbox in the PR template still declares AI use for the whole PR.

Update CONTRIBUTING.md, the agent snippet and the AI Policy workflow
in this repository and in the template.

AI-Assisted-By: Claude Opus 5.5 (xhigh)
Signed-off-by: MiMoHo <37556964+MiMoHo@users.noreply.github.com>
Comment thread AGENT_MD_SNIPPETS.md
### What this agent must always do

- Add an `Assisted-by: AGENT_NAME:MODEL_VERSION` git trailer to every commit containing AI-assisted content.
- Add an `AI-Assisted-By: MODEL (REASONING_EFFORT)` git trailer to every commit containing AI-assisted content, for example `AI-Assisted-By: Claude Opus 4.5 (medium)`. Ask the contributor for the reasoning effort if you do not know it. Do not add your own agent name unless the contributor asks for it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the policy is also interested in the harness used, but I agree that it's confusing that we're saying AGENT_NAME when we mean HARNESS_NAME

Suggested change
- Add an `AI-Assisted-By: MODEL (REASONING_EFFORT)` git trailer to every commit containing AI-assisted content, for example `AI-Assisted-By: Claude Opus 4.5 (medium)`. Ask the contributor for the reasoning effort if you do not know it. Do not add your own agent name unless the contributor asks for it.
- Add an `AI-Assisted-By: HARNESS:MODEL (REASONING_EFFORT)` git trailer to every commit containing AI-assisted content, for example `AI-Assisted-By: Claude code:Claude Opus 4.5 (medium)`. Ask the contributor for the reasoning effort if you do not know it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is my own interpretation, I might be wrong the maintainers can correct me on it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants