direct engine hard-fails with UpdateSchema Nothing to update when a schema comment is set outside the bundl

[SimonDudanski]

Describe the issue

On the direct deployment engine, a resources.schemas.* entry that does not declare comment becomes permanently undeployable as soon as someone sets a description on that schema in Unity Catalog — manually, or by accepting an AI-generated description in Catalog Explorer.

The engine reads the remote comment, sees no local value, classifies it as drift and plans an update. The UC rejects it with 400 INVALID_PARAMETER_VALUE / Nothing to update.

it aborts bundle deploy for the whole bundle, so every other resource stops deploying to that target until someone edits the bundle YAML. There is no way out from the CLI side — re-running, --force-lock and repeated deploys all fail identically.

Configuration

bundle:
  name: schema-comment-repro
  engine: direct

targets:
  dev:
    default: true
    workspace:
      host: https://<workspace>.azuredatabricks.net

resources:
  schemas:
    repro_schema:
      name: repro_schema
      catalog_name: main   # any catalog you can create schemas in
      # no `comment:` here — that is what triggers the bug

Steps to reproduce the behavior

1. databricks bundle deploy -t dev — succeeds, main.repro_schema is created.
2. Set a description on the schema outside the bundle, either in Catalog Explorer (in my case it was done with "accept AI-generated description"), or:

   COMMENT ON SCHEMA main.repro_schema IS 'Some description added in the UI';

3. databricks bundle deploy -t dev again, with the configuration unchanged.
4. See error.

Expected Behavior

The deploy succeeds. Either of these would be acceptable:

• The engine treats the remotely set comment as real drift and clears it, sending comment: "" explicitly (via ForceSendFields) so the PATCH has an effective payload. This matches "the bundle is the source of truth".
• Or the engine does not plan an update for a field the configuration omits, the way apps: stop planning an update on every deploy #6328 handled apps.forward_user_access_token via backend_defaults.

Actual Behavior

bundle deploy fails on every run:

Error: cannot update resources.schemas.repro_schema: updating id=main.repro_schema: UpdateSchema Nothing to update. (400 INVALID_PARAMETER_VALUE)

Endpoint: PATCH https://<workspace>.azuredatabricks.net/api/2.1/unity-catalog/schemas/main.repro_schema
HTTP Status: 400 Bad Request
API error_code: INVALID_PARAMETER_VALUE
API message: UpdateSchema Nothing to update.

Updating deployment state...

OS and CLI version

First hit on v1.9.0 (Windows, Azure DevOps self-hosted agent); reproduced on v1.13.0 (macOS). Azure Databricks.

Is this a regression?

Unknown. This bundle has used engine: direct since it was set up, so I have not tried the same configuration on the terraform engine or on an earlier CLI version. It is present in every version I have run, up to and including the current v1.13.0.

Detailed plan

Redacted bundle plan -o json, taken on v1.13.0 while the schema was in the broken state. This is from the original bundle rather than the minimal configuration above; the schema is declared there without comment in exactly the same way. The <user-1>_ prefix comes from presets.name_prefix on the target and is unrelated to the issue.

{
  "plan_version": 2,
  "cli_version": "1.13.0",
  "plan": {
    "resources.schemas.sandbox_AIS_Default": {
      "action": "update",
      "new_state": {
        "value": {
          "catalog_name": "sandbox_dev",
          "name": "<user-1>_AIS_Default"
        }
      },
      "remote_state": {
        "browse_only": false,
        "catalog_name": "sandbox_dev",
        "catalog_type": "MANAGED_CATALOG",
        "comment": "The schema organizes data related to autonomous intelligent systems. It includes tables for system configurations, sensor data, and machine learning model outputs, along with functions for data processing and analysis. Use this schema for developing and testing intelligent systems, data integration, and performance evaluation.",
        "created_at": 1787306140198,
        "created_by": "<email-1>",
        "effective_predictive_optimization_flag": {
          "inherited_from_name": "<metastore-1>",
          "inherited_from_type": "METASTORE",
          "value": "ENABLE"
        },
        "enable_predictive_optimization": "INHERIT",
        "full_name": "sandbox_dev.<user-1>_ais_default",
        "metastore_id": "<uuid-1>",
        "name": "<user-1>_ais_default",
        "owner": "<email-1>",
        "schema_id": "<uuid-2>",
        "updated_at": 1787306382667,
        "updated_by": "<email-1>"
      },
      "changes": {
        "comment": {
          "action": "update",
          "remote": "The schema organizes data related to autonomous intelligent systems. It includes tables for system configurations, sensor data, and machine learning model outputs, along with functions for data processing and analysis. Use this schema for developing and testing intelligent systems, data integration, and performance evaluation."
        },
        "name": {
          "action": "skip",
          "reason": "id_field",
          "old": "<user-1>_AIS_Default",
          "new": "<user-1>_AIS_Default",
          "remote": "<user-1>_ais_default"
        }
      }
    },
    "resources.schemas.sandbox_AIS_Default.grants": {
      "depends_on": [
        {
          "node": "resources.schemas.sandbox_AIS_Default",
          "label": "${resources.schemas.sandbox_AIS_Default.id}"
        }
      ],
      "action": "skip",
      "remote_state": {
        "securable_type": "schema",
        "full_name": "sandbox_dev.<user-1>_ais_default",
        "__embed__": [
          {
            "principal": "<group-1>",
            "privileges": [
              "ALL_PRIVILEGES"
            ]
          }
        ]
      }
    }
  }
}

Debug Logs

Available on request — happy to attach a redacted --log-level=debug run showing the PATCH body if that helps confirm the serialization theory below.

Related

• apps: stop planning an update on every deploy #6328 (apps: stop planning an update on every deploy) — same root shape, "a remote value and no local one" reading as drift and firing a no-op update. The Apps backend accepts that no-op call, so it only surfaced as a perpetual 1 to change; UC returns 400 instead, which turns the same bug into a hard deploy failure.
• bundle plan perpetually reports update on catalog/schema/volume .grants sub-resources even when live state is confirmed converged #6030 / direct: Fix perpetual update on grants with ALL_PRIVILEGES #6064, Fix grant empty privileges never converges #6062, Fix pipeline allow_duplicate_names never converges #6076, Fix never-converging job diff on reordered webhook notifications #6060, direct: Fix RemoteAlreadySet skipping input-only fields #6112 — the broader family of direct-engine drift and convergence fixes.

Workaround

Declare the comment explicitly in the bundle, so the PATCH has a non-empty payload:

resources:
  schemas:
    repro_schema:
      name: repro_schema
      catalog_name: main
      comment: Some description added in the UI

Clearing the description in UC also works, but it comes back as soon as anyone accepts an AI-generated description again.