diff --git a/AGENTS.md b/AGENTS.md index 8b68c63e..c155f055 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -12,7 +12,8 @@ Use this file when changing this repository. It is not an operating guide for us ## Architecture and boundaries - `dataiku_mcp/server.py` constructs the FastMCP server, request middleware, and stdio/HTTP runners. `dataiku_mcp/__init__.py` re-exports that public server API and imports each tool module so decorators register their tools. -- `dataiku_mcp/tools/` contains thin adapters over the synchronous `dataikuapi` client. Shared client authentication and executors live at the package root; validation, serialization, and error helpers live under `dataiku_mcp/tools/utils/`. +- `dataiku_mcp/tools/` contains thin adapters over the synchronous `dataikuapi` client. Shared client authentication and executors live at the package root; shared validation, serialization, and redaction live under `dataiku_mcp/utils/`; tool-specific helpers remain under `dataiku_mcp/tools/utils/`. +- `dataiku_mcp/operations/` owns operation models and the explicit catalog; `operations/handlers/` contains undecorated SDK adapters. The four MCP entry points remain in `dataiku_mcp/tools/operations.py`. - `dataiku_mcp/tools/cobuild.py` owns retained Cobuild conversations. Cobuild is the default path for constructing or modifying project-level flow and analytic assets. - Cobuild owns construction and modification of project-level flow and analytic assets. Direct writes are allowed only for the fixed exception categories defined by the **Cobuild Write-Routing Convention** in `CODING_STANDARDS_AND_STRUCTURE.md`; do not infer permission for a new direct write from existing implementation. - `skills/dataiku-headless/SKILL.md` is the operator-facing router. Its `references/` directory owns object-specific inspection, mutation, and verification workflows. @@ -37,7 +38,7 @@ Do not copy volatile inventories, versions, or command details into this file wh - Keep tool handlers `async def`. Put every blocking SDK or filesystem operation inside `run_blocking(...)`; use `run_cobuild_blocking(...)` only for retained Cobuild turns. - Capture any client or instance identity needed for consistency before the first `await` when a concurrent instance switch could otherwise change the target. - Keep handlers thin: validate boundary invariants, call the SDK, shape a compact response, and let Dataiku validate deeper domain rules. -- Reuse helpers from `tools/utils/` instead of adding local variants. Use `compact_json`, `columnar`, and `omit_empty` for consistent responses where appropriate. +- Reuse shared helpers from `utils/` and tool-specific helpers from `tools/utils/` instead of adding local variants. Use `compact_json`, `columnar`, and `omit_empty` for consistent responses where appropriate. - Raise clear exceptions for whole-operation failures. Return structured per-item or uncertain-outcome payloads only when callers need partial results or durable job/run identities for recovery. - Never return API keys, credentials, unredacted connection secrets, or sensitive exception text. - Represent destructive intent with semantic parameters such as `overwrite`, `drop_data`, or a replace/run mode. Do not add `confirm*` parameters; confirmation is a harness concern. diff --git a/CODING_STANDARDS_AND_STRUCTURE.md b/CODING_STANDARDS_AND_STRUCTURE.md index e7c79a69..04c2e4e3 100644 --- a/CODING_STANDARDS_AND_STRUCTURE.md +++ b/CODING_STANDARDS_AND_STRUCTURE.md @@ -29,7 +29,7 @@ export DKU_API_KEY="your-api-key" | What to change | Where | | --- | --- | | MCP tool logic | `dataiku_mcp/tools/**/*.py` | -| Shared validation helpers | `dataiku_mcp/tools/utils/validation.py` | +| Shared validation helpers | `dataiku_mcp/utils/validation.py` | | Workflow prompts | `dataiku_mcp/prompts/workflows.py` | | Project/dataset/folder/recipe/ML skills | `skills/**/SKILL.md` | | Cobuild conversation tools | `dataiku_mcp/tools/cobuild.py` | @@ -74,7 +74,7 @@ export DKU_API_KEY="your-api-key" ## Tool Input Validation - Assume FastMCP/Pydantic type validation for tool signatures; avoid duplicating type checks in tool code unless there is a clear compatibility/safety reason. - Keep validation for structured inputs (JSON payloads/lists), enum-like parameters, and semantic risk parameters (`overwrite`, `drop_data`, replace modes, recursive build modes). -- Put reusable validation logic in `dataiku_mcp/tools/utils/validation.py` and reuse those helpers from tool modules instead of creating ad-hoc local validators. +- Put reusable validation logic in `dataiku_mcp/utils/validation.py` and reuse those helpers from tool modules instead of creating ad-hoc local validators. - When reducing validation, preserve existing destructive-action safeguards and return clear errors from downstream API calls. ## Async And Blocking Style @@ -110,9 +110,13 @@ export DKU_API_KEY="your-api-key" ## Fixed Tool Surface - `runtime/run_mcp.py` is the single launcher. It requires `--transport stdio` for the local plugin or `--transport http` for the authenticated Streamable HTTP deployment. Keep the transports explicitly selected and preserve the same registered tool catalog. -- Register one directly visible tool catalog. Do not add an MCP search mode. -- Classify explicit tools with `PROJECT_READ_POLICY`, `PROJECT_EXECUTION_POLICY`, `ADMIN_POLICY` or `PROJECT_EDIT_POLICY`; use `INSTANCE_MANAGEMENT_POLICY` for connection controls. Execution includes status/polling, administration includes instance inspection and project folders, and project edits include project Cobuild. Only connection management is exempt from requiring a selected profile. -- `SUPPORTED_DSS_INSTANCE_TYPES` defines the currently supported backend types independently of all valid `InstanceType` values. Agent Management, Govern and Deployer remain configurable profiles with instance controls only until their product support lands. +- Register one stable, directly visible MCP tool catalog. Do not change registration when instances switch. The bounded operation-registry pilot exposes fixed search/describe/read/write tools; operation discovery does not dynamically register MCP tools. +- Classify v1 tools with `PROJECT_READ_POLICY`, `PROJECT_EXECUTION_POLICY`, `ADMIN_POLICY` or `PROJECT_EDIT_POLICY`; use `API_OPERATIONS_POLICY` for shared operation tools and `INSTANCE_MANAGEMENT_POLICY` for connection controls. The categories describe purpose: execution includes status/polling, administration includes instance inspection and project folders, and project edits include project Cobuild. +- Only the `INSTANCE_MANAGEMENT_POLICY` group is exempt from requiring a selected profile. All v1 and v2 Dataiku tools, including operation search/describe, require one. Read/write execution handlers validate their selected operation directly. +- Every tool declares allowed_instance_types metadata. Middleware enforces it against the pinned request instance before backend token exchange. Operation handlers additionally enforce the selected operation’s instance policy, read/write effect, arguments and transport before invoking its SDK adapter; missing declarations fail closed. +- `SUPPORTED_DSS_INSTANCE_TYPES` identifies types served by the DSS client independently of project-tool policies. +- Product operations use public Python SDK methods only; do not call private transport helpers or add direct endpoint fallbacks. Deployer supports stdio and HTTP; other product support is deferred. +- Operation arguments use one Pydantic model for discovery and execution. Never accept caller-supplied HTTP methods or paths. Keep backend requirements and response projections explicit; catalog presence does not verify live server support. Preserve Cobuild routing for in-project asset construction. - The registered set is the contract; `tests/test_tool_surface.py` pins the exact catalog. Any tool add/remove/rename updates that pinned set in the same change. ## Guardrails diff --git a/README.md b/README.md index f1c84c3d..fe8be467 100644 --- a/README.md +++ b/README.md @@ -122,6 +122,20 @@ Dataiku Headless is an MCP server and agent skill library for operating Dataiku Cobuild runs as a retained conversation through MCP tools. This repository intentionally keeps its own tool surface small: inspection tools, three deterministic executions of existing assets (`build_datasets`, `run_recipe`, and `run_scenario`), narrow configuration actions such as selecting which container an existing object runs in, and a few bootstrap actions that Cobuild cannot perform, such as creating a project or uploading a local file. +Project and API Deployer operations discover infrastructures, published versions and deployments, +create deployment configurations, change target versions, apply updates asynchronously, +and inspect health, governance codes and update history. They operate on already published +bundles and API services. See the [Deployer guide](skills/dataiku-headless/references/deployer.md) +for node selection, production effects and coverage limits. + +Direct project reads, execution and administration support Design, Automation. +Project edits and Cobuild require Design. The four shared operation tools expose +13 public SDK actions for Deployer, restricted to their matching profiles. +Search returns all matching operation summaries; describe loads individual schemas +on demand, and separate read/write tools execute them. +See the [operation guide](skills/dataiku-headless/references/operations.md). + + ## Capability reference For a quick reference to what Headless can inspect, what Cobuild builds, and the @@ -133,7 +147,7 @@ To build or update a Dataiku plugin, follow the ## MCP Server -`dataiku_mcp` is a FastMCP server that exposes Dataiku operations as typed, async MCP tools. Tools are organized by domain: projects, project folders, flow, connections, datasets, data quality, managed folders, recipes, machine learning, insights, dashboards, scenarios, WebApps, wikis, agents, LLMs and knowledge banks, instance plugins, job management, administrative tasks, and Cobuild conversations. +`dataiku_mcp` is a FastMCP server that exposes Dataiku operations as typed, async MCP tools. Tools are organized by domain: projects, project folders, flow, connections, datasets, data quality, managed folders, recipes, machine learning, insights, dashboards, scenarios, WebApps, wikis, agents, LLMs and knowledge banks, instance plugins, Project and API Deployer, job management, administrative tasks, and Cobuild conversations. - Async execution for all Dataiku API calls - Progress notifications for long-running operations @@ -180,11 +194,8 @@ Each profile requires `instance_type`: `design`, `automation`, `deployer`, `gove `agent-management`. Existing stdio profiles missing this field are automatically assigned `design` and rewritten on disk before loading. This temporary migration is scheduled for deprecation by 0.9.0; existing values are preserved and validated. -The configured type determines which tools can run; see the -[instance-type policy](docs/capabilities.md#instance-types). Agent Management, Govern and Deployer -currently support instance controls only. Their status is `unsupported`, without -a DSS connection check. The setup page allows these profiles to be saved without testing. -Design and Automation require a successful connection test. +The declared type selects tool policy; backend features and permissions are validated by Dataiku. A local Deployer on a Design or Automation node requires a separate `deployer` profile for that URL. +Design, Automation and Deployer use the DSS client. Agent Management and Govern remain configurable profiles with instance controls only and report unsupported without a backend call. **Environment override.** To select an explicit target, including inside a Code Studio, set these three variables in your environment or copy @@ -246,6 +257,11 @@ The Dataiku Headless marketplace plugin uses stdio transport. For customer-manag ├── dataiku_mcp/ │ ├── auth.py # Dataiku client creation and HTTP token exchange │ ├── executors.py # Shared blocking and Cobuild executors +│ ├── operations/ # Operation models, catalog, and undecorated handlers +│ │ ├── models.py # Typed operation definitions and argument schemas +│ │ ├── catalog.py # Explicit registration and lookup +│ │ └── handlers/deployer.py # Internal Deployer SDK adapters +│ ├── utils/ # Shared validation, serialization, and redaction │ ├── tools/ │ │ ├── agents.py # Agent/agent-version/agent-tool inspection tools │ │ ├── agent_reviews.py # Agent review/test/run inspection tools @@ -258,6 +274,7 @@ The Dataiku Headless marketplace plugin uses stdio transport. For customer-manag │ │ ├── data_quality.py # Dataset Data Quality rule inspection tools │ │ ├── dashboards.py # Dashboard inspection tools │ │ ├── datasets.py # Dataset inspection tools + local-file upload writes +│ │ ├── operations.py # Shared discovery/read/write MCP tools │ │ ├── evaluation_stores.py # Evaluation Store inspection tools │ │ ├── flow.py # Flow inspection tools │ │ ├── instances.py # Multi-instance switching tools @@ -277,7 +294,7 @@ The Dataiku Headless marketplace plugin uses stdio transport. For customer-manag │ │ ├── project_libraries.py # Project library inspection/search + local-file write │ │ ├── recipes.py # Recipe inspection tools │ │ ├── machine_learning/ # ML analysis/saved-model inspection tools -│ │ └── utils/ # Tool validation and response-shaping utilities +│ │ └── utils/ # Tool-specific helpers │ ├── config/ # Models, stdio/HTTP configuration, and request routing │ ├── server.py # FastMCP construction, middleware, and transport startup │ ├── setup_server.py # Temporary loopback page used by URL elicitation diff --git a/dataiku_mcp/__init__.py b/dataiku_mcp/__init__.py index bd19b45b..9ecf0b30 100644 --- a/dataiku_mcp/__init__.py +++ b/dataiku_mcp/__init__.py @@ -42,6 +42,7 @@ licensing, llms_and_knowledge_banks, managed_folders, + operations, plugins, project_folders, project_libraries, diff --git a/dataiku_mcp/instance_policy.py b/dataiku_mcp/instance_policy.py index 2e77f5e0..01ae7421 100644 --- a/dataiku_mcp/instance_policy.py +++ b/dataiku_mcp/instance_policy.py @@ -18,17 +18,16 @@ from .config.models import InstanceType -# Product-node support is added separately from this Design/Automation foundation. -SUPPORTED_DSS_INSTANCE_TYPES = frozenset({"design", "automation"}) +# Instance types currently supported through DSSClient. +SUPPORTED_DSS_INSTANCE_TYPES = frozenset({"design", "automation", "deployer"}) # These are @mcp.tool metadata policies. V1 tools expose individual actions; # project authoring still routes through Cobuild and its fixed direct exceptions. -PROJECT_READ_POLICY = {"allowed_instance_types": sorted(SUPPORTED_DSS_INSTANCE_TYPES)} -PROJECT_EXECUTION_POLICY = { - "allowed_instance_types": sorted(SUPPORTED_DSS_INSTANCE_TYPES) -} -ADMIN_POLICY = {"allowed_instance_types": sorted(SUPPORTED_DSS_INSTANCE_TYPES)} +PROJECT_READ_POLICY = {"allowed_instance_types": ["design", "automation"]} +PROJECT_EXECUTION_POLICY = {"allowed_instance_types": ["design", "automation"]} +ADMIN_POLICY = {"allowed_instance_types": ["design", "automation"]} PROJECT_EDIT_POLICY = {"allowed_instance_types": ["design"]} +API_OPERATIONS_POLICY = {"allowed_instance_types": ["deployer"]} INSTANCE_MANAGEMENT_POLICY = {"allowed_instance_types": list(get_args(InstanceType))} diff --git a/dataiku_mcp/operations/__init__.py b/dataiku_mcp/operations/__init__.py new file mode 100644 index 00000000..722f8d8a --- /dev/null +++ b/dataiku_mcp/operations/__init__.py @@ -0,0 +1,15 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Operation definitions and catalog.""" diff --git a/dataiku_mcp/operations/catalog.py b/dataiku_mcp/operations/catalog.py new file mode 100644 index 00000000..96a60215 --- /dev/null +++ b/dataiku_mcp/operations/catalog.py @@ -0,0 +1,142 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Audited operation declarations for the Project and API Deployer.""" + +from types import MappingProxyType +from typing import Callable + +from .models import Operation, arguments_for +from .handlers import deployer + + +def _deployer( + operation_id: str, + handler: Callable, + example: dict, + *, + effect="read", + destructive=False, + idempotent=False, +): + return Operation( + operation_id=operation_id, + domain="deployer", + description=handler.__doc__, + effect=effect, + allowed_instance_types=("deployer",), + arguments=arguments_for(handler), + handler=handler, + requirements="Selected node must expose Project/API Deployer. The configured type does not verify feature availability or permissions.", + response_contract="Selected operational fields; credentials, raw settings, free-text diagnostics and update logs are omitted. Configuration saves do not apply deployments; start_update returns a future_id for deployer.get_future_status on the same instance.", + example=example, + destructive=destructive, + idempotent=idempotent, + ) + + +_DEPLOYER_OPERATIONS = [ + _deployer( + "deployer.list_stages", + deployer.list_deployer_stages, + {"deployer_type": "project"}, + ), + _deployer( + "deployer.list_infrastructures", + deployer.list_deployer_infrastructures, + {"deployer_type": "project"}, + ), + _deployer( + "deployer.get_infrastructure", + deployer.get_deployer_infrastructure, + {"deployer_type": "project", "infra_id": "discovered-infra"}, + ), + _deployer( + "deployer.list_published_items", + deployer.list_deployer_published_items, + {"deployer_type": "api"}, + ), + _deployer( + "deployer.get_published_item", + deployer.get_deployer_published_item, + {"deployer_type": "api", "source_id": "discovered-service"}, + ), + _deployer( + "deployer.list_deployments", + deployer.list_deployer_deployments, + {"deployer_type": "project"}, + ), + _deployer( + "deployer.get_deployment", + deployer.get_deployer_deployment, + {"deployer_type": "project", "deployment_id": "discovered-deployment"}, + ), + _deployer( + "deployer.list_updates", + deployer.list_deployer_updates, + {"deployer_type": "project", "deployment_id": "discovered-deployment"}, + ), + _deployer( + "deployer.get_update", + deployer.get_deployer_update, + {"deployer_type": "project", "deployment_id": "discovered-deployment"}, + ), + _deployer( + "deployer.create_deployment", + deployer.create_deployer_deployment, + { + "deployer_type": "project", + "deployment_id": "requested-deployment", + "infra_id": "discovered-infra", + "source_id": "discovered-project", + "version": "discovered-bundle", + }, + effect="write", + ), + _deployer( + "deployer.update_deployment", + deployer.update_deployer_deployment, + { + "deployer_type": "api", + "deployment_id": "discovered-deployment", + "version": "discovered-version", + }, + effect="write", + destructive=True, + idempotent=True, + ), + _deployer( + "deployer.start_update", + deployer.start_deployer_update, + {"deployer_type": "project", "deployment_id": "discovered-deployment"}, + effect="write", + destructive=True, + ), + _deployer( + "deployer.get_future_status", + deployer.get_deployer_future_status, + {"future_id": "returned-future"}, + ), +] + + +OPERATIONS = {op.operation_id: op for op in _DEPLOYER_OPERATIONS} +OPERATIONS = MappingProxyType(OPERATIONS) + + +def get_operation(operation_id: str) -> Operation: + try: + return OPERATIONS[operation_id] + except KeyError: + raise ValueError("Unknown operation_id. Use search_operations.") from None diff --git a/dataiku_mcp/operations/handlers/__init__.py b/dataiku_mcp/operations/handlers/__init__.py new file mode 100644 index 00000000..d871cf97 --- /dev/null +++ b/dataiku_mcp/operations/handlers/__init__.py @@ -0,0 +1,15 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Internal SDK operation handlers.""" diff --git a/dataiku_mcp/operations/handlers/deployer.py b/dataiku_mcp/operations/handlers/deployer.py new file mode 100644 index 00000000..40f0939b --- /dev/null +++ b/dataiku_mcp/operations/handlers/deployer.py @@ -0,0 +1,378 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Project and API Deployer handlers with body validation of identifiers.""" + +from typing import Annotated, Literal + +from dataikuapi.dss.future import DSSFuture +from dataikuapi.utils import DataikuException +from pydantic import Field +from requests import RequestException + +from ...auth import get_dss_client +from ...executors import run_blocking +from dataiku_mcp.utils.serialization import columnar +from dataiku_mcp.utils.validation import require_non_empty_string +from dataiku_mcp.utils.validation import require_path_segment + + +DeployerType = Literal["project", "api"] +Version = Annotated[ + str, Field(description="Published bundle ID (project) or service version ID (api).") +] + +# Raw Deployer status includes API keys, OAuth settings, scripts and endpoint code. +# Keep these allowlists explicit, including for nested objects. +_INFRA_FIELDS = ("id", "stage", "infraType", "governCheckPolicy", "canListHeavyStatus") +_ITEM_FIELDS = ("id", "name") +_DEPLOYMENT_FIELDS = ( + "id", + "infraId", + "publishedProjectKey", + "deployedProjectKey", + "bundleId", + "publishedServiceId", + "deployedServiceId", + "enabled", + "endpointId", +) +_PERMISSION_FIELDS = ( + "canDeploy", + "canWrite", + "isAdmin", + "hasFullDeployPermissions", + "isInfraAdmin", + "neverEverDeployed", +) +_UPDATE_FIELDS = ("startTimestamp", "endTimestamp", "type", "status", "requester") + + +def _pick(raw: dict, fields: tuple) -> dict: + return {key: raw[key] for key in fields if key in raw} + + +def _deployment_info(raw: dict) -> dict: + result = _pick(raw, _DEPLOYMENT_FIELDS) + mapping = raw.get("generationsMapping") + if mapping is not None: + result["generationsMapping"] = _pick(mapping, ("mode", "generation")) + if "entries" in mapping: + result["generationsMapping"]["entries"] = [ + _pick(entry, ("generation", "proba")) for entry in mapping["entries"] + ] + return result + + +def _summary(raw: dict) -> dict: + result = _pick(raw, _PERMISSION_FIELDS) + for key, fields in ( + ("infraBasicInfo", _INFRA_FIELDS), + ("projectBasicInfo", _ITEM_FIELDS), + ("serviceBasicInfo", _ITEM_FIELDS), + ): + if key in raw: + result[key] = _pick(raw[key], fields) + if "deploymentBasicInfo" in raw: + result["deploymentBasicInfo"] = _deployment_info(raw["deploymentBasicInfo"]) + if "packages" in raw: + result["packages"] = columnar( + raw["packages"], ["id", "publishedOn", "publishedBy"] + ) + if "deployments" in raw: + result["deployments"] = [_deployment_info(item) for item in raw["deployments"]] + if "infras" in raw: + result["infras"] = [_pick(item, _INFRA_FIELDS) for item in raw["infras"]] + return result + + +async def _execute(deployer_type: DeployerType, operation) -> dict: + def _run(): + client = get_dss_client() + deployer = ( + client.get_projectdeployer() + if deployer_type == "project" + else client.get_apideployer() + ) + try: + return operation(deployer) + except (DataikuException, RequestException): + # SDK exception bodies may include node credentials or raw settings. + raise RuntimeError( + "Deployer request failed. Check node capabilities, permissions and object IDs. " + "For a write, inspect current state and update history before retrying; " + "the request may have reached Dataiku." + ) from None + + return await run_blocking(_run) + + +async def list_deployer_stages(deployer_type: DeployerType) -> dict: + """Discover configured infrastructure stages for a Project or API Deployer.""" + return await _execute( + deployer_type, + lambda deployer: {"stages": columnar(deployer.list_stages(), ["id", "desc"])}, + ) + + +async def list_deployer_infrastructures(deployer_type: DeployerType) -> dict: + """Discover deployment targets and the current user’s deployment permissions.""" + return await _execute( + deployer_type, + lambda deployer: { + "infrastructures": [ + _summary(raw) for raw in deployer.list_infras(as_objects=False) + ] + }, + ) + + +async def get_deployer_infrastructure( + deployer_type: DeployerType, infra_id: str +) -> dict: + """Inspect an infrastructure’s summary and associated deployments without credentials.""" + infra_id = require_path_segment(infra_id, "infra_id") + return await _execute( + deployer_type, + lambda deployer: _summary(deployer.get_infra(infra_id).get_status().get_raw()), + ) + + +async def list_deployer_published_items(deployer_type: DeployerType) -> dict: + """Discover published projects or API services available for deployment.""" + + def _run(deployer): + items = ( + deployer.list_projects(as_objects=False) + if deployer_type == "project" + else deployer.list_services(as_objects=False) + ) + return {"items": [_summary(raw) for raw in items]} + + return await _execute(deployer_type, _run) + + +async def get_deployer_published_item( + deployer_type: DeployerType, source_id: str +) -> dict: + """Inspect a published project or API service and its available bundles or versions.""" + source_id = require_path_segment(source_id, "source_id") + + def _run(deployer): + item = ( + deployer.get_project(source_id) + if deployer_type == "project" + else deployer.get_service(source_id) + ) + return _summary(item.get_status().get_raw()) + + return await _execute(deployer_type, _run) + + +async def list_deployer_deployments(deployer_type: DeployerType) -> dict: + """Discover deployment IDs and configured target versions; use detail for live health.""" + return await _execute( + deployer_type, + lambda deployer: { + "deployments": [ + _summary(raw) for raw in deployer.list_deployments(as_objects=False) + ] + }, + ) + + +async def get_deployer_deployment( + deployer_type: DeployerType, deployment_id: str +) -> dict: + """Inspect configured deployment version, current health and governance status.""" + deployment_id = require_path_segment(deployment_id, "deployment_id") + + def _run(deployer): + deployment = deployer.get_deployment(deployment_id) + status = deployment.get_status() + governance = deployment.get_governance_status() + # Only structured codes and severity: diagnostic free text can contain secrets. + return { + **_summary(status.get_light()), + "settings": _deployment_info(deployment.get_settings().get_raw()), + "health": status.get_health(), + "health_messages": [ + _pick(message, ("severity", "isFatal", "code")) + for message in status.get_health_messages().get("messages", []) + ], + "governance_messages": [ + _pick(message, ("severity", "isFatal", "code")) + for message in governance.get("messages", []) + ], + } + + return await _execute(deployer_type, _run) + + +async def list_deployer_updates( + deployer_type: DeployerType, deployment_id: str +) -> dict: + """Find deployment update timestamps and outcomes for recovery and auditing.""" + deployment_id = require_path_segment(deployment_id, "deployment_id") + return await _execute( + deployer_type, + lambda deployer: { + "updates": [ + _pick(update.get_raw(), _UPDATE_FIELDS) + for update in deployer.get_deployment(deployment_id).list_updates() + ] + }, + ) + + +async def get_deployer_update( + deployer_type: DeployerType, + deployment_id: str, + timestamp: str | None = None, +) -> dict: + """Inspect one deployment update outcome, using a timestamp or the latest update.""" + deployment_id = require_path_segment(deployment_id, "deployment_id") + if timestamp is not None: + timestamp = require_path_segment(timestamp, "timestamp") + return await _execute( + deployer_type, + lambda deployer: _pick( + deployer.get_deployment(deployment_id).get_update(timestamp).get_raw(), + _UPDATE_FIELDS, + ), + ) + + +async def create_deployer_deployment( + deployer_type: DeployerType, + deployment_id: str, + source_id: str, + infra_id: str, + version: Version, + deployed_project_key: str | None = None, + project_folder_id: str | None = None, + endpoint_id: str | None = None, +) -> dict: + """Create deployment configuration from a published version; start an update to apply it.""" + deployment_id = require_path_segment(deployment_id, "deployment_id") + source_id = require_path_segment(source_id, "source_id") + infra_id = require_path_segment(infra_id, "infra_id") + if deployed_project_key is not None: + deployed_project_key = require_path_segment( + deployed_project_key, "deployed_project_key" + ) + if project_folder_id is not None: + project_folder_id = require_path_segment(project_folder_id, "project_folder_id") + if endpoint_id is not None: + endpoint_id = require_path_segment(endpoint_id, "endpoint_id") + version = require_non_empty_string(version, "version") + if deployer_type == "project" and endpoint_id is not None: + raise ValueError("endpoint_id is only supported for API deployments") + if deployer_type == "api" and ( + deployed_project_key is not None or project_folder_id is not None + ): + raise ValueError( + "deployed_project_key and project_folder_id are only supported for project deployments" + ) + + def _run(deployer): + if deployer_type == "project": + deployment = deployer.create_deployment( + deployment_id, + source_id, + infra_id, + version, + deployed_project_key=deployed_project_key, + project_folder_id=project_folder_id, + ) + else: + deployment = deployer.create_deployment( + deployment_id, source_id, infra_id, version, endpoint_id=endpoint_id + ) + return { + "deployment_id": deployment.id, + "status": "configuration_created", + "update_started": False, + } + + return await _execute(deployer_type, _run) + + +async def update_deployer_deployment( + deployer_type: DeployerType, + deployment_id: str, + version: Version | None = None, + enabled: bool | None = None, +) -> dict: + """Save a target version or API enablement; an update is still needed to apply it.""" + deployment_id = require_path_segment(deployment_id, "deployment_id") + if version is None and enabled is None: + raise ValueError("Provide version or enabled") + if version is not None: + version = require_non_empty_string(version, "version") + if enabled is not None and deployer_type != "api": + raise ValueError("enabled is only supported for API deployments") + + def _run(deployer): + settings = deployer.get_deployment(deployment_id).get_settings() + if version is not None: + if deployer_type == "project": + settings.bundle_id = version + else: + settings.set_single_version(version) + if enabled is not None: + settings.set_enabled(enabled) + settings.save() + return { + "deployment_id": deployment_id, + "status": "configuration_saved", + "update_started": False, + } + + return await _execute(deployer_type, _run) + + +async def start_deployer_update( + deployer_type: DeployerType, deployment_id: str +) -> dict: + """Apply saved deployment settings asynchronously, potentially replacing running production assets.""" + deployment_id = require_path_segment(deployment_id, "deployment_id") + + def _run(deployer): + future = deployer.get_deployment(deployment_id).start_update() + return { + "deployment_id": deployment_id, + "future_id": future.job_id, + "status": "update_submitted", + "hint": "Poll get_future_status on this same instance, then re-read deployment health and update history. If future_id is null, inspect update history. Do not resubmit an uncertain update.", + } + + return await _execute(deployer_type, _run) + + +async def get_deployer_future_status(future_id: str) -> dict: + """Poll a deployment future without fetching potentially sensitive result payloads.""" + future_id = require_path_segment(future_id, "future_id") + + def _run(): + try: + state = DSSFuture(get_dss_client(), future_id).peek_state() + except (DataikuException, RequestException): + raise RuntimeError( + "Deployer future polling failed. Inspect deployment health and update history on this instance; do not resubmit the deployment." + ) from None + fields = ("jobId", "hasResult", "aborted", "alive", "startTime", "endTime") + return {"future_id": future_id, "state": _pick(state, fields)} + + return await run_blocking(_run) diff --git a/dataiku_mcp/operations/models.py b/dataiku_mcp/operations/models.py new file mode 100644 index 00000000..18f52041 --- /dev/null +++ b/dataiku_mcp/operations/models.py @@ -0,0 +1,107 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Typed operation definitions and shared argument schemas.""" + +import inspect +from dataclasses import dataclass +from typing import Awaitable, Callable, Literal, get_type_hints + +from pydantic import BaseModel, ConfigDict, ValidationError, create_model + +from ..config.models import InstanceType +from dataiku_mcp.utils.redaction import contains_redacted_values + +Effect = Literal["read", "write"] + + +class OperationArguments(BaseModel): + model_config = ConfigDict( + extra="forbid", strict=True, hide_input_in_errors=True, allow_inf_nan=False + ) + + +@dataclass(frozen=True) +class Operation: + operation_id: str + domain: str + description: str + effect: Effect + allowed_instance_types: tuple[InstanceType, ...] + arguments: type[BaseModel] + handler: Callable[..., Awaitable[dict]] + requirements: str + response_contract: str + example: dict + destructive: bool = False + idempotent: bool = False + transports: tuple[str, ...] = ("stdio", "http") + + def validate(self, arguments: dict) -> dict: + if self.effect == "write" and contains_redacted_values(arguments): + raise ValueError( + "Do not submit redacted result placeholders as write values. Use a scoped update that preserves hidden fields, or report the coverage gap." + ) + try: + value = self.arguments.model_validate(arguments) + except ValidationError as exc: + # Pydantic errors otherwise echo potentially sensitive input payloads. + problems = [ + ".".join(map(str, item["loc"])) + ": " + item["type"] + for item in exc.errors(include_input=False) + ] + raise ValueError( + "Invalid operation arguments: " + "; ".join(problems) + ) from None + # Keep nested models intact; omitted arguments use handler defaults. + return {name: getattr(value, name) for name in value.model_fields_set} + + def summary(self) -> dict: + return { + "operation_id": self.operation_id, + "domain": self.domain, + "description": self.description, + "effect": self.effect, + "allowed_instance_types": list(self.allowed_instance_types), + "destructive": self.destructive, + "idempotent": self.idempotent, + "transports": list(self.transports), + } + + def describe(self) -> dict: + return { + **self.summary(), + "input_schema": self.arguments.model_json_schema(), + "example_arguments": self.example, + "requirements": self.requirements, + "server_support": "unknown", + "response_contract": self.response_contract, + } + + +def arguments_for(handler: Callable) -> type[BaseModel]: + hints = get_type_hints(handler, include_extras=True) + return create_model( + handler.__name__ + "Arguments", + __base__=OperationArguments, + **{ + name: ( + hints[name], + parameter.default + if parameter.default is not inspect.Parameter.empty + else ..., + ) + for name, parameter in inspect.signature(handler).parameters.items() + }, + ) diff --git a/dataiku_mcp/setup_server.py b/dataiku_mcp/setup_server.py index eead4654..ea0086bd 100644 --- a/dataiku_mcp/setup_server.py +++ b/dataiku_mcp/setup_server.py @@ -196,7 +196,7 @@ def _page(
A short name used when switching instances.
-
Agent Management, Govern and Deployer profiles can be saved without testing. Their API tools are not available yet.
+
Agent Management and Govern profiles can be saved without testing. Their API tools are not available yet.
Enter the URL of your Dataiku instance.
Create one in Dataiku under Profile & Settings → API keys.
diff --git a/dataiku_mcp/tools/agent_reviews.py b/dataiku_mcp/tools/agent_reviews.py index 1fcda289..f09543fa 100644 --- a/dataiku_mcp/tools/agent_reviews.py +++ b/dataiku_mcp/tools/agent_reviews.py @@ -20,8 +20,8 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json, omit_empty -from .utils.validation import require_non_empty_string +from dataiku_mcp.utils.serialization import columnar, compact_json, omit_empty +from dataiku_mcp.utils.validation import require_non_empty_string @mcp.tool( diff --git a/dataiku_mcp/tools/agents.py b/dataiku_mcp/tools/agents.py index b89f49b6..edff5dae 100644 --- a/dataiku_mcp/tools/agents.py +++ b/dataiku_mcp/tools/agents.py @@ -23,8 +23,8 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json, omit_empty -from .utils.validation import require_non_empty_string +from dataiku_mcp.utils.serialization import columnar, compact_json, omit_empty +from dataiku_mcp.utils.validation import require_non_empty_string def _get_version(raw, version_id=""): diff --git a/dataiku_mcp/tools/cobuild.py b/dataiku_mcp/tools/cobuild.py index 9d22d758..a4ffb2b4 100644 --- a/dataiku_mcp/tools/cobuild.py +++ b/dataiku_mcp/tools/cobuild.py @@ -31,8 +31,10 @@ from ..config import request from ..executors import run_blocking, run_cobuild_blocking from ..server import DSS_INDEPENDENT_TOOL_TAG, mcp -from .utils.serialization import columnar, compact_json, omit_empty -from .utils.validation import require_non_empty_string as _require_non_empty_string +from dataiku_mcp.utils.serialization import columnar, compact_json, omit_empty +from dataiku_mcp.utils.validation import ( + require_non_empty_string as _require_non_empty_string, +) TURN_WAIT_TIMEOUT_SECONDS = 240 diff --git a/dataiku_mcp/tools/code_environments.py b/dataiku_mcp/tools/code_environments.py index 65ff8b44..7cff3900 100644 --- a/dataiku_mcp/tools/code_environments.py +++ b/dataiku_mcp/tools/code_environments.py @@ -24,8 +24,8 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json -from .utils.validation import ( +from dataiku_mcp.utils.serialization import columnar, compact_json +from dataiku_mcp.utils.validation import ( require_non_empty_string as _require_non_empty_string, require_non_empty_strings as _require_non_empty_strings, require_non_negative_int as _require_non_negative_int, diff --git a/dataiku_mcp/tools/connections.py b/dataiku_mcp/tools/connections.py index 90912a3c..9d54ff7d 100644 --- a/dataiku_mcp/tools/connections.py +++ b/dataiku_mcp/tools/connections.py @@ -14,8 +14,6 @@ """Dataiku connection discovery and inspection tools.""" -from typing import Any - from typing import Annotated, Literal from fastmcp import Context @@ -24,57 +22,13 @@ from ..instance_policy import ADMIN_POLICY from ..server import mcp from ..executors import run_blocking -from .utils.serialization import columnar, compact_json, omit_empty +from dataiku_mcp.utils.serialization import columnar, compact_json, omit_empty from ..auth import get_dss_client -from .utils.validation import ( +from dataiku_mcp.utils.validation import ( require_non_empty_string as _require_non_empty_string, ) -_CONNECTION_SECRET_REDACTION = "__DATAIKU_REDACTED__" -_SENSITIVE_SUFFIXES = ( - "apikey", - "credential", - "credentials", - "password", - "secret", - "token", -) -_SENSITIVE_EXACT_KEYS = ("key",) -_SENSITIVE_KEY_FRAGMENTS = ( - "accesskey", - "appsecret", - "keybase64data", - "keyjsondata", - "passphrase", - "privatekey", - "secretkey", - "tokenkey", -) - - -def _is_sensitive_key(key: str) -> bool: - normalized = key.replace("_", "").replace("-", "").lower() - return ( - normalized in _SENSITIVE_EXACT_KEYS - or normalized.endswith(_SENSITIVE_SUFFIXES) - or any(fragment in normalized for fragment in _SENSITIVE_KEY_FRAGMENTS) - ) - - -def _redact_sensitive_data(value: Any) -> Any: - if isinstance(value, dict): - redacted = {} - for key, item in value.items(): - if _is_sensitive_key(str(key)): - redacted[key] = _CONNECTION_SECRET_REDACTION - else: - redacted[key] = _redact_sensitive_data(item) - return redacted - - if isinstance(value, list): - return [_redact_sensitive_data(item) for item in value] - - return value +from dataiku_mcp.utils.redaction import redact_sensitive_data as _redact_sensitive_data _CONNECTION_TYPES_BY_CATEGORY = { diff --git a/dataiku_mcp/tools/container_exec.py b/dataiku_mcp/tools/container_exec.py index b77b01f3..8202f30e 100644 --- a/dataiku_mcp/tools/container_exec.py +++ b/dataiku_mcp/tools/container_exec.py @@ -27,8 +27,8 @@ from .machine_learning.shared.common import require_single_ml_task from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import compact_json -from .utils.validation import require_allowed_value, require_non_empty_string +from dataiku_mcp.utils.serialization import compact_json +from dataiku_mcp.utils.validation import require_allowed_value, require_non_empty_string ContainerMode = Literal["INHERIT", "NONE", "EXPLICIT_CONTAINER"] ObjectType = Literal["recipe", "ml_task", "webapp", "knowledge_bank", "agent_tool"] diff --git a/dataiku_mcp/tools/cross_project_sharing.py b/dataiku_mcp/tools/cross_project_sharing.py index 7d0738df..0772d3c7 100644 --- a/dataiku_mcp/tools/cross_project_sharing.py +++ b/dataiku_mcp/tools/cross_project_sharing.py @@ -21,8 +21,10 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json -from .utils.validation import require_non_empty_string as _require_non_empty_string +from dataiku_mcp.utils.serialization import columnar, compact_json +from dataiku_mcp.utils.validation import ( + require_non_empty_string as _require_non_empty_string, +) def _raise_if_forbidden(project_key: str, exc: DataikuException) -> None: diff --git a/dataiku_mcp/tools/dashboards.py b/dataiku_mcp/tools/dashboards.py index 9ff725c8..bf8c0acc 100644 --- a/dataiku_mcp/tools/dashboards.py +++ b/dataiku_mcp/tools/dashboards.py @@ -20,8 +20,10 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json -from .utils.validation import require_non_empty_string as _require_non_empty_string +from dataiku_mcp.utils.serialization import columnar, compact_json +from dataiku_mcp.utils.validation import ( + require_non_empty_string as _require_non_empty_string, +) def _serialize_dashboard_list_item(item) -> dict: diff --git a/dataiku_mcp/tools/data_collections.py b/dataiku_mcp/tools/data_collections.py index 18701031..de48f4d2 100644 --- a/dataiku_mcp/tools/data_collections.py +++ b/dataiku_mcp/tools/data_collections.py @@ -19,9 +19,11 @@ from ..instance_policy import ADMIN_POLICY from ..server import mcp from ..executors import run_blocking -from .utils.serialization import columnar, compact_json +from dataiku_mcp.utils.serialization import columnar, compact_json from ..auth import get_dss_client -from .utils.validation import require_non_empty_string as _require_non_empty_string +from dataiku_mcp.utils.validation import ( + require_non_empty_string as _require_non_empty_string, +) @mcp.tool( diff --git a/dataiku_mcp/tools/data_quality.py b/dataiku_mcp/tools/data_quality.py index 2acf1e49..20b78462 100644 --- a/dataiku_mcp/tools/data_quality.py +++ b/dataiku_mcp/tools/data_quality.py @@ -24,8 +24,8 @@ from ..auth import get_dss_client from ..executors import run_blocking from .utils.parsing import coerce_json_array as _coerce_json_array -from .utils.serialization import compact_json, is_empty, omit_empty -from .utils.validation import ( +from dataiku_mcp.utils.serialization import compact_json, is_empty, omit_empty +from dataiku_mcp.utils.validation import ( require_non_empty_string as _require_non_empty_string, require_non_empty_strings as _require_non_empty_strings, require_non_negative_int as _require_non_negative_int, diff --git a/dataiku_mcp/tools/datasets.py b/dataiku_mcp/tools/datasets.py index 3ea83fb7..dff1d746 100644 --- a/dataiku_mcp/tools/datasets.py +++ b/dataiku_mcp/tools/datasets.py @@ -33,9 +33,11 @@ from ..executors import run_blocking from .utils.metrics import parse_metric_ids as _parse_metric_ids from .utils.metrics import select_metrics as _select_metrics -from .utils.serialization import columnar, compact_json, is_empty -from .utils.validation import require_non_empty_string as _require_non_empty_string -from .utils.validation import require_positive_int as _require_positive_int +from dataiku_mcp.utils.serialization import columnar, compact_json, is_empty +from dataiku_mcp.utils.validation import ( + require_non_empty_string as _require_non_empty_string, +) +from dataiku_mcp.utils.validation import require_positive_int as _require_positive_int _MAX_EXPORT_ROWS = 1_000_000 _MAX_UPLOAD_ROWS = 10_000 diff --git a/dataiku_mcp/tools/evaluation_stores.py b/dataiku_mcp/tools/evaluation_stores.py index a847ad2b..d84517a9 100644 --- a/dataiku_mcp/tools/evaluation_stores.py +++ b/dataiku_mcp/tools/evaluation_stores.py @@ -23,8 +23,8 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json -from .utils.validation import require_non_empty_string +from dataiku_mcp.utils.serialization import columnar, compact_json +from dataiku_mcp.utils.validation import require_non_empty_string @mcp.tool( diff --git a/dataiku_mcp/tools/flow.py b/dataiku_mcp/tools/flow.py index a64e72df..e3543f2f 100644 --- a/dataiku_mcp/tools/flow.py +++ b/dataiku_mcp/tools/flow.py @@ -23,8 +23,8 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json, omit_empty -from .utils.validation import ( +from dataiku_mcp.utils.serialization import columnar, compact_json, omit_empty +from dataiku_mcp.utils.validation import ( require_non_empty_string as _require_non_empty_string, require_positive_int as _require_positive_int, ) diff --git a/dataiku_mcp/tools/general_settings.py b/dataiku_mcp/tools/general_settings.py index ac35597d..e07dd74a 100644 --- a/dataiku_mcp/tools/general_settings.py +++ b/dataiku_mcp/tools/general_settings.py @@ -23,7 +23,7 @@ from ..server import mcp from ..auth import get_dss_client, require_admin from ..executors import run_blocking -from .utils.serialization import columnar, compact_json +from dataiku_mcp.utils.serialization import columnar, compact_json _CONTAINER_SUMMARY_COLUMNS = [ "name", diff --git a/dataiku_mcp/tools/groups.py b/dataiku_mcp/tools/groups.py index c2e3afa1..81710f0d 100644 --- a/dataiku_mcp/tools/groups.py +++ b/dataiku_mcp/tools/groups.py @@ -24,8 +24,8 @@ from ..auth import get_dss_client, require_admin from ..executors import run_blocking from .utils.identity_sources import IdentitySourceType -from .utils.serialization import columnar, compact_json -from .utils.validation import ( +from dataiku_mcp.utils.serialization import columnar, compact_json +from dataiku_mcp.utils.validation import ( require_non_empty_string as _require_non_empty_string, require_non_empty_strings as _require_non_empty_strings, require_non_negative_int as _require_non_negative_int, diff --git a/dataiku_mcp/tools/insights.py b/dataiku_mcp/tools/insights.py index c719a452..9f18a317 100644 --- a/dataiku_mcp/tools/insights.py +++ b/dataiku_mcp/tools/insights.py @@ -20,8 +20,10 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json -from .utils.validation import require_non_empty_string as _require_non_empty_string +from dataiku_mcp.utils.serialization import columnar, compact_json +from dataiku_mcp.utils.validation import ( + require_non_empty_string as _require_non_empty_string, +) def _serialize_insight_list_item(item) -> dict: diff --git a/dataiku_mcp/tools/instances.py b/dataiku_mcp/tools/instances.py index 13603254..51a88cbb 100644 --- a/dataiku_mcp/tools/instances.py +++ b/dataiku_mcp/tools/instances.py @@ -28,7 +28,7 @@ from ..executors import run_blocking from ..server import DSS_INDEPENDENT_TOOL_TAG, mcp from ..setup_server import SESSION_LIFETIME_SECONDS, start_setup_server -from .utils.serialization import columnar, compact_json, omit_empty +from dataiku_mcp.utils.serialization import columnar, compact_json, omit_empty InstanceName = Annotated[ str, Field(description="A configured instance name, from list_instances.") diff --git a/dataiku_mcp/tools/jobs.py b/dataiku_mcp/tools/jobs.py index be078068..8e6d8815 100644 --- a/dataiku_mcp/tools/jobs.py +++ b/dataiku_mcp/tools/jobs.py @@ -34,8 +34,8 @@ get_job_status_full as _get_job_status_full, summarize_listed_job as _summarize_listed_job, ) -from .utils.serialization import columnar, compact_json -from .utils.validation import ( +from dataiku_mcp.utils.serialization import columnar, compact_json +from dataiku_mcp.utils.validation import ( require_non_empty_list as _require_non_empty_list, require_non_empty_string as _require_non_empty_string, require_int_in_range as _require_int_in_range, diff --git a/dataiku_mcp/tools/licensing.py b/dataiku_mcp/tools/licensing.py index 51b44dc8..b76b5d75 100644 --- a/dataiku_mcp/tools/licensing.py +++ b/dataiku_mcp/tools/licensing.py @@ -24,7 +24,7 @@ from ..server import mcp from ..auth import get_dss_client, require_admin from ..executors import run_blocking -from .utils.serialization import compact_json +from dataiku_mcp.utils.serialization import compact_json def _profile_rows(status: dict, include_capabilities: bool) -> list[dict]: diff --git a/dataiku_mcp/tools/llms_and_knowledge_banks.py b/dataiku_mcp/tools/llms_and_knowledge_banks.py index d9f55919..3330e7ab 100644 --- a/dataiku_mcp/tools/llms_and_knowledge_banks.py +++ b/dataiku_mcp/tools/llms_and_knowledge_banks.py @@ -23,8 +23,8 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json -from .utils.validation import ( +from dataiku_mcp.utils.serialization import columnar, compact_json +from dataiku_mcp.utils.validation import ( require_allowed_value as _require_allowed_value, require_non_empty_string as _require_non_empty_string, require_positive_int as _require_positive_int, diff --git a/dataiku_mcp/tools/machine_learning/analyses.py b/dataiku_mcp/tools/machine_learning/analyses.py index 8d4ce91f..519ed58c 100644 --- a/dataiku_mcp/tools/machine_learning/analyses.py +++ b/dataiku_mcp/tools/machine_learning/analyses.py @@ -25,8 +25,10 @@ from ...server import mcp from ...auth import get_dss_client from ...executors import run_blocking -from ..utils.serialization import columnar, compact_json, omit_empty -from ..utils.validation import require_non_empty_string as _require_non_empty_string +from dataiku_mcp.utils.serialization import columnar, compact_json, omit_empty +from dataiku_mcp.utils.validation import ( + require_non_empty_string as _require_non_empty_string, +) from .shared.common import find_analysis_input_dataset, require_single_ml_task from .shared.summaries import build_ml_task_summary diff --git a/dataiku_mcp/tools/machine_learning/saved_models.py b/dataiku_mcp/tools/machine_learning/saved_models.py index f252f611..a48e1f01 100644 --- a/dataiku_mcp/tools/machine_learning/saved_models.py +++ b/dataiku_mcp/tools/machine_learning/saved_models.py @@ -20,8 +20,10 @@ from ...server import mcp from ...auth import get_dss_client from ...executors import run_blocking -from ..utils.serialization import columnar, compact_json -from ..utils.validation import require_non_empty_string as _require_non_empty_string +from dataiku_mcp.utils.serialization import columnar, compact_json +from dataiku_mcp.utils.validation import ( + require_non_empty_string as _require_non_empty_string, +) @mcp.tool( diff --git a/dataiku_mcp/tools/managed_folders.py b/dataiku_mcp/tools/managed_folders.py index 50c49f41..c9af275f 100644 --- a/dataiku_mcp/tools/managed_folders.py +++ b/dataiku_mcp/tools/managed_folders.py @@ -27,8 +27,8 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json, omit_empty -from .utils.validation import ( +from dataiku_mcp.utils.serialization import columnar, compact_json, omit_empty +from dataiku_mcp.utils.validation import ( require_non_empty_string as _require_non_empty_string, require_positive_int as _require_positive_int, ) diff --git a/dataiku_mcp/tools/operations.py b/dataiku_mcp/tools/operations.py new file mode 100644 index 00000000..2abaae2c --- /dev/null +++ b/dataiku_mcp/tools/operations.py @@ -0,0 +1,151 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Stable discovery and effect-separated dispatch for audited operations.""" + +from typing import Literal + +from ..config import request +from ..instance_policy import API_OPERATIONS_POLICY +from ..operations.catalog import OPERATIONS, get_operation +from ..server import DSS_INDEPENDENT_TOOL_TAG, mcp +from dataiku_mcp.utils.serialization import compact_json + + +@mcp.tool( + meta=API_OPERATIONS_POLICY, + title="Search Operations", + annotations={ + "readOnlyHint": True, + "destructiveHint": False, + "openWorldHint": False, + }, + tags={DSS_INDEPENDENT_TOOL_TAG}, +) +async def search_operations( + query: str = "", + domain: Literal["deployer"] | None = None, + effect: Literal["read", "write"] | None = None, +) -> str: + """Return all matching operation summaries for the selected instance.""" + instance = request.get_pinned_instance() + words = query.casefold().split() + matches = [ + op + for op in OPERATIONS.values() + if instance.instance_type in op.allowed_instance_types + and (domain is None or domain == op.domain) + and (effect is None or effect == op.effect) + and all( + word in (op.operation_id + " " + op.description).casefold() + for word in words + ) + ] + return compact_json( + { + "instance_name": instance.name, + "server_support": "unknown", + "total": len(matches), + "operations": [op.summary() for op in matches], + } + ) + + +@mcp.tool( + meta=API_OPERATIONS_POLICY, + title="Describe Operation", + annotations={ + "readOnlyHint": True, + "destructiveHint": False, + "openWorldHint": False, + }, + tags={DSS_INDEPENDENT_TOOL_TAG}, +) +async def describe_operation(operation_id: str) -> str: + """Load one operation's input schema, example, policy and backend requirements.""" + operation = get_operation(operation_id) + instance = request.get_pinned_instance() + return compact_json( + { + **operation.describe(), + "instance_name": instance.name, + "allowed_on_selected_instance": instance.instance_type + in operation.allowed_instance_types, + } + ) + + +async def _dispatch( + operation_id: str, + arguments: dict, + effect: Literal["read", "write"], + expected_instance: str | None, +) -> str: + instance = request.get_pinned_instance() + if expected_instance is not None and instance.name != expected_instance: + raise ValueError( + "Selected instance changed. Inspect get_current_instance before retrying." + ) + operation = get_operation(operation_id) + if operation.effect != effect: + raise ValueError( + f"Operation '{operation_id}' requires the {operation.effect} dispatcher." + ) + if instance.instance_type not in operation.allowed_instance_types: + raise ValueError( + f"Operation '{operation_id}' is unavailable for instance type '{instance.instance_type}'. Use switch_instance." + ) + values = operation.validate(arguments) + transport = "http" if request.is_http_request() else "stdio" + if transport not in operation.transports: + raise ValueError( + f"Operation '{operation_id}' is unavailable over transport '{transport}'." + ) + result = await operation.handler(**values) + return compact_json( + {"operation_id": operation_id, "instance_name": instance.name, "result": result} + ) + + +@mcp.tool( + meta=API_OPERATIONS_POLICY, + title="Run Read Operation", + annotations={ + "readOnlyHint": True, + "destructiveHint": False, + "openWorldHint": False, + }, +) +async def run_read_operation( + operation_id: str, arguments: dict, expected_instance: str | None = None +) -> str: + """Run a catalogued read; expected_instance rejects a stale instance selection.""" + return await _dispatch(operation_id, arguments, "read", expected_instance) + + +@mcp.tool( + meta=API_OPERATIONS_POLICY, + title="Run Write Operation", + annotations={ + "readOnlyHint": False, + "destructiveHint": True, + "idempotentHint": False, + "openWorldHint": False, + }, +) +async def run_write_operation( + operation_id: str, arguments: dict, expected_instance: str | None = None +) -> str: + """Run a catalogued mutation after reviewing its schema, scope and execution effects.""" + return await _dispatch(operation_id, arguments, "write", expected_instance) diff --git a/dataiku_mcp/tools/plugins.py b/dataiku_mcp/tools/plugins.py index f99ce7e2..44b1bb0a 100644 --- a/dataiku_mcp/tools/plugins.py +++ b/dataiku_mcp/tools/plugins.py @@ -38,8 +38,8 @@ from .code_environments import PythonInterpreter from .utils.errors import dataiku_message from .utils.parsing import parse_json_object -from .utils.serialization import columnar, compact_json, omit_empty -from .utils.validation import ( +from dataiku_mcp.utils.serialization import columnar, compact_json, omit_empty +from dataiku_mcp.utils.validation import ( require_allowed_value as _require_allowed_value, require_git_repository_url as _require_git_repository_url, require_int_in_range as _require_int_in_range, diff --git a/dataiku_mcp/tools/project_folders.py b/dataiku_mcp/tools/project_folders.py index 27f6b970..b42231e3 100644 --- a/dataiku_mcp/tools/project_folders.py +++ b/dataiku_mcp/tools/project_folders.py @@ -23,8 +23,8 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json -from .utils.validation import ( +from dataiku_mcp.utils.serialization import columnar, compact_json +from dataiku_mcp.utils.validation import ( require_non_empty_string as _require_non_empty_string, ) diff --git a/dataiku_mcp/tools/project_libraries.py b/dataiku_mcp/tools/project_libraries.py index 44a72922..8377a19c 100644 --- a/dataiku_mcp/tools/project_libraries.py +++ b/dataiku_mcp/tools/project_libraries.py @@ -38,9 +38,11 @@ from ..config import request from ..server import mcp from ..executors import run_blocking -from .utils.serialization import columnar, compact_json, omit_empty +from dataiku_mcp.utils.serialization import columnar, compact_json, omit_empty from ..auth import get_dss_client -from .utils.validation import require_non_empty_string as _require_non_empty_string +from dataiku_mcp.utils.validation import ( + require_non_empty_string as _require_non_empty_string, +) LibraryPath = Annotated[ str, Field(description="Folder inside the project library; / is its root.") diff --git a/dataiku_mcp/tools/projects.py b/dataiku_mcp/tools/projects.py index dd68dece..75418184 100644 --- a/dataiku_mcp/tools/projects.py +++ b/dataiku_mcp/tools/projects.py @@ -28,8 +28,8 @@ from ..auth import get_dss_client from ..executors import run_blocking from .utils.parsing import coerce_json_object as _coerce_json_object -from .utils.serialization import columnar, compact_json, omit_empty -from .utils.validation import ( +from dataiku_mcp.utils.serialization import columnar, compact_json, omit_empty +from dataiku_mcp.utils.validation import ( require_non_empty_string as _require_non_empty_string, ) diff --git a/dataiku_mcp/tools/recipes.py b/dataiku_mcp/tools/recipes.py index 828d0a19..699fdf98 100644 --- a/dataiku_mcp/tools/recipes.py +++ b/dataiku_mcp/tools/recipes.py @@ -24,7 +24,7 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json +from dataiku_mcp.utils.serialization import columnar, compact_json CODE_RECIPE_TYPES = { "python", diff --git a/dataiku_mcp/tools/scenarios.py b/dataiku_mcp/tools/scenarios.py index 171bfd61..50e3f12f 100644 --- a/dataiku_mcp/tools/scenarios.py +++ b/dataiku_mcp/tools/scenarios.py @@ -30,8 +30,8 @@ from ..auth import get_dss_client from ..executors import run_blocking from .utils.errors import safe_error_text as _safe_error_text -from .utils.serialization import columnar, compact_json -from .utils.validation import ( +from dataiku_mcp.utils.serialization import columnar, compact_json +from dataiku_mcp.utils.validation import ( require_non_empty_string as _require_non_empty_string, require_int_in_range as _require_int_in_range, require_positive_int as _require_positive_int, diff --git a/dataiku_mcp/tools/semantic_models.py b/dataiku_mcp/tools/semantic_models.py index b8194445..19807e4b 100644 --- a/dataiku_mcp/tools/semantic_models.py +++ b/dataiku_mcp/tools/semantic_models.py @@ -20,8 +20,10 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json -from .utils.validation import require_non_empty_string as _require_non_empty_string +from dataiku_mcp.utils.serialization import columnar, compact_json +from dataiku_mcp.utils.validation import ( + require_non_empty_string as _require_non_empty_string, +) @mcp.tool( diff --git a/dataiku_mcp/tools/users.py b/dataiku_mcp/tools/users.py index 14ae44f0..306aadfc 100644 --- a/dataiku_mcp/tools/users.py +++ b/dataiku_mcp/tools/users.py @@ -24,8 +24,8 @@ from ..auth import get_dss_client, require_admin from ..executors import run_blocking from .utils.identity_sources import IdentitySourceType -from .utils.serialization import columnar, compact_json -from .utils.validation import ( +from dataiku_mcp.utils.serialization import columnar, compact_json +from dataiku_mcp.utils.validation import ( require_non_empty_string as _require_non_empty_string, require_non_empty_strings as _require_non_empty_strings, require_non_negative_int as _require_non_negative_int, diff --git a/dataiku_mcp/tools/utils/metrics.py b/dataiku_mcp/tools/utils/metrics.py index c18375f1..5f22ae14 100644 --- a/dataiku_mcp/tools/utils/metrics.py +++ b/dataiku_mcp/tools/utils/metrics.py @@ -19,7 +19,7 @@ import json from typing import Any -from .validation import require_non_empty_strings +from dataiku_mcp.utils.validation import require_non_empty_strings def parse_metric_ids(metric_ids: str | None) -> list[str] | None: diff --git a/dataiku_mcp/tools/utils/parsing.py b/dataiku_mcp/tools/utils/parsing.py index 179592c0..f05e1da2 100644 --- a/dataiku_mcp/tools/utils/parsing.py +++ b/dataiku_mcp/tools/utils/parsing.py @@ -19,7 +19,7 @@ import json from typing import Any -from .validation import require_non_empty_strings +from dataiku_mcp.utils.validation import require_non_empty_strings def parse_json_object(raw_json: str, field_name: str) -> dict: diff --git a/dataiku_mcp/tools/webapps.py b/dataiku_mcp/tools/webapps.py index e6f22616..3a68f419 100644 --- a/dataiku_mcp/tools/webapps.py +++ b/dataiku_mcp/tools/webapps.py @@ -22,8 +22,10 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json, omit_empty -from .utils.validation import require_non_empty_string as _require_non_empty_string +from dataiku_mcp.utils.serialization import columnar, compact_json, omit_empty +from dataiku_mcp.utils.validation import ( + require_non_empty_string as _require_non_empty_string, +) WEBAPP_SECRET_REDACTION = "__DATAIKU_REDACTED__" _PRESERVED_SENSITIVE_KEYS = {"apiKey"} diff --git a/dataiku_mcp/tools/wikis.py b/dataiku_mcp/tools/wikis.py index 1c2572fc..386984d1 100644 --- a/dataiku_mcp/tools/wikis.py +++ b/dataiku_mcp/tools/wikis.py @@ -20,8 +20,10 @@ from ..server import mcp from ..auth import get_dss_client from ..executors import run_blocking -from .utils.serialization import columnar, compact_json, omit_empty -from .utils.validation import require_non_empty_string as _require_non_empty_string +from dataiku_mcp.utils.serialization import columnar, compact_json, omit_empty +from dataiku_mcp.utils.validation import ( + require_non_empty_string as _require_non_empty_string, +) def _build_parent_map(taxonomy: list, parent_id: str | None = None) -> dict: diff --git a/dataiku_mcp/utils/__init__.py b/dataiku_mcp/utils/__init__.py new file mode 100644 index 00000000..7bc69a93 --- /dev/null +++ b/dataiku_mcp/utils/__init__.py @@ -0,0 +1,15 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Utilities shared by MCP tools and operation handlers.""" diff --git a/dataiku_mcp/utils/redaction.py b/dataiku_mcp/utils/redaction.py new file mode 100644 index 00000000..b63ba91b --- /dev/null +++ b/dataiku_mcp/utils/redaction.py @@ -0,0 +1,111 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Recursive credential redaction for backend operation results.""" + +from typing import Any +from urllib.parse import urlsplit, urlunsplit, parse_qsl, urlencode + +_CONNECTION_SECRET_REDACTION = "__DATAIKU_REDACTED__" +_SENSITIVE_SUFFIXES = ( + "apikey", + "apikeys", + "credential", + "credentials", + "password", + "passwords", + "secret", + "secrets", + "token", + "tokens", +) +_SENSITIVE_EXACT_KEYS = ( + "key", + "authorization", + "cookie", + "setcookie", + "logs", + "stacktrace", + "traceback", + "errordetails", + "errormessage", + "error", + "errors", +) +_SENSITIVE_KEY_FRAGMENTS = ( + "accesskey", + "appsecret", + "keybase64data", + "keyjsondata", + "passphrase", + "privatekey", + "secretkey", + "tokenkey", +) + + +def _is_sensitive_key(key: str) -> bool: + normalized = key.replace("_", "").replace("-", "").lower() + return ( + normalized in _SENSITIVE_EXACT_KEYS + or normalized.endswith(_SENSITIVE_SUFFIXES) + or any(fragment in normalized for fragment in _SENSITIVE_KEY_FRAGMENTS) + ) + + +def redact_sensitive_data(value: Any) -> Any: + if isinstance(value, dict): + redacted = {} + for key, item in value.items(): + if _is_sensitive_key(str(key)): + redacted[key] = _CONNECTION_SECRET_REDACTION + else: + redacted[key] = redact_sensitive_data(item) + return redacted + + if isinstance(value, list): + return [redact_sensitive_data(item) for item in value] + + if isinstance(value, str) and value.startswith(("https://", "http://")): + try: + parts = urlsplit(value) + query = parse_qsl(parts.query, keep_blank_values=True) + if "@" in parts.netloc or any(_is_sensitive_key(key) for key, _ in query): + host = parts.netloc.rsplit("@", 1)[-1] + query = [ + ( + key, + _CONNECTION_SECRET_REDACTION + if _is_sensitive_key(key) + else item, + ) + for key, item in query + ] + return urlunsplit( + (parts.scheme, host, parts.path, urlencode(query), parts.fragment) + ) + except ValueError: + return _CONNECTION_SECRET_REDACTION + return value + + +def contains_redacted_values(value: Any) -> bool: + """Detect result placeholders that must never become replacement credentials.""" + if isinstance(value, str): + return _CONNECTION_SECRET_REDACTION in value + if isinstance(value, dict): + return any(contains_redacted_values(item) for item in value.values()) + if isinstance(value, list): + return any(contains_redacted_values(item) for item in value) + return False diff --git a/dataiku_mcp/tools/utils/serialization.py b/dataiku_mcp/utils/serialization.py similarity index 100% rename from dataiku_mcp/tools/utils/serialization.py rename to dataiku_mcp/utils/serialization.py diff --git a/dataiku_mcp/tools/utils/validation.py b/dataiku_mcp/utils/validation.py similarity index 90% rename from dataiku_mcp/tools/utils/validation.py rename to dataiku_mcp/utils/validation.py index 99a3cf62..c25ba147 100644 --- a/dataiku_mcp/tools/utils/validation.py +++ b/dataiku_mcp/utils/validation.py @@ -147,3 +147,19 @@ def require_relative_posix_path(value: str, field_name: str) -> str: if path.is_absolute() or ".." in path.parts or "\\" in value: raise ValueError(f"'{field_name}' must be a relative path without '..'") return value + + +def require_path_segment(value: str, field_name: str) -> str: + """Validate an identifier before the SDK interpolates it into a URL path.""" + cleaned = value.strip() + if ( + not cleaned + or cleaned in {".", ".."} + or any(char in value for char in "/\\%?#") + or any(ord(char) < 32 or ord(char) == 127 for char in value) + ): + raise ValueError( + f"'{field_name}' must be a non-empty path segment without URL delimiters, " + "escapes, dot segments or control characters" + ) + return cleaned diff --git a/docs/capabilities.md b/docs/capabilities.md index 7461058c..7e94fbc3 100644 --- a/docs/capabilities.md +++ b/docs/capabilities.md @@ -22,44 +22,15 @@ Local profile actions are listed separately because they only change which Datai instance the local client targets. -## Instance types - -The configured instance type controls which tools can run in both stdio and HTTP. -The catalog stays the same; each tool advertises its allowed types in metadata, -and dispatch rejects unsupported calls before authentication delegation or execution. -Dataiku permissions and API availability still apply to permitted calls. - -| Instance type | Available tools | -|---|---| -| design | All existing explicit tools | -| automation | Reads, existing asset execution, instance administration, and project-folder operations; no Cobuild or project definition/content writes | -| agent-management, govern, deployer | Instance controls only; API tools for these node types are not implemented yet | - -Automation excludes all six Cobuild tools, even conversation listing and polling, -and `create_project`, `update_project_settings`, `set_project_variables`, -`set_container_exec_config`, `create_upload_dataset`, `create_managed_folder`, -`upload_file_to_managed_folder`, and `write_project_library_file`. -It permits `build_datasets`, `run_recipe`, `run_scenario`, `abort_job`, and every -project-folder operation, including creation, deletion, and moving projects. -Existing execution can modify outputs; this policy does not make Automation read-only. - -Instance controls are `list_instances`, `switch_instance`, `configure_instance`, -`delete_instance`, and `get_current_instance`. The first four also work without an -active selection, subject to the existing HTTP restrictions on local configuration. -For Agent Management, Govern and Deployer, `get_current_instance` returns sanitized profile metadata -with connection_status set to unsupported, without contacting DSS or reporting a version. -Agent Management, Govern and Deployer profiles can be saved without a connection -test. Design and Automation require a successful DSS connection test in local setup. - ## Surface -**131 tools** · 93 read · 24 direct Dataiku write · 6 Cobuild · 4 execute · 3 local +**135 tools** · 96 read · 25 direct Dataiku write · 6 Cobuild · 4 execute · 3 local profile · 1 connection test | Bucket | # | Scope | |---|---|---| -| Read / inspect | 93 | Never mutates | -| Direct Dataiku write | 24 | Bootstrap, project configuration, cross-project, admin | +| Read / inspect | 96 | Never mutates | +| Direct Dataiku write | 25 | Bootstrap, project configuration, cross-project, admin | | Cobuild conversation | 6 | All flow and analytic building | | Execute | 4 | `build_datasets`, `run_recipe`, `run_scenario`, `abort_job` | | Local profile action | 3 | `configure_instance`, `switch_instance`, `delete_instance` | @@ -116,6 +87,7 @@ No Cobuild involved. Scope says what kind of access, and where a write lands. | Managed folders | `list_managed_folders`, `get_managed_folder_info`, `get_managed_folder_contents` | `create_managed_folder`, `upload_file_to_managed_folder` | Bootstrap, **in-project** — local-file upload is stdio only | | Project libraries | `list_project_library`, `read_project_library_file`, `search_project_library`, `validate_project_library_file` | `write_project_library_file` | Bootstrap, **in-project** — local-file write is stdio only | | Project folders | `list_project_folders`, `get_project_folder` | `create_project_folder`, `move_project_to_folder`, `delete_project_folder` | Cross-project | +| Operation registry pilot | `search_operations`, `describe_operation`, `run_read_operation` | `run_write_operation` | Fixed MCP interface over 13 operations: 13 Deployer. Backend-defined definition payloads are labelled explicitly. Tool instance policy is enforced before token exchange; read/write handlers validate operations before SDK execution. | | Code environments | `list_code_envs` | `create_code_env`, `update_code_env`, `delete_code_env` | Instance-level | | Plugins | `list_plugins`, `list_plugin_usages` | `install_plugin`, `ensure_plugin_code_env`, `update_plugin`, `delete_plugin` | Instance-level — local-path install/update are stdio only | | Users | `list_users` | `create_user`, `update_user`, `delete_user` | Instance-level | @@ -144,3 +116,24 @@ Local client configuration, not Dataiku objects. | `switch_instance` | Change the active instance; in HTTP mode saves only the authenticated user’s selected catalog instance | | `list_instances`, `get_current_instance` | Show configured instances, their declared types, and the active one; `get_current_instance` includes the Dataiku version when available | | `delete_instance` | Removes a **saved connection profile from the local config file**. Does not touch the Dataiku instance. | + +## Instance policies and operation discovery + +| Policy group | Allowed configured instance types | +| --- | --- | +| V1 explicit project reads, execution and administration | Design, Automation | +| V1 explicit project edits and project Cobuild | Design | +| V2 general API operation discovery and execution | Deployer | +| Instance management | All five types | + +All four v2 tools require a selected profile. Each catalogued operation requires its +matching instance type. Design and Automation cannot use v2. A local Deployer on +a Design or Automation node requires a separate Deployer profile for that URL. +The catalog remains stable across instance switches; new product actions are +operation IDs rather than individually registered MCP tools. + +Design, Automation and Deployer use the DSS client. Agent Management and Govern remain configurable profiles with instance controls only and report unsupported without a backend call. + +Discovery exposes schemas, effects, backend requirements and transport restrictions. +All adapters use public SDK methods; catalog presence does not verify live server +support. See [the operation guide](../skills/dataiku-headless/references/operations.md). diff --git a/docs/http-deployment.md b/docs/http-deployment.md index 8a796916..0dcfaa3c 100644 --- a/docs/http-deployment.md +++ b/docs/http-deployment.md @@ -110,11 +110,7 @@ The settings file has four sections: - `user_selections` records each authenticated user's current instance. Start with an empty object; the server manages it. -Instance types are operator-supplied and enforce the same -[tool policy](capabilities.md#instance-types) as stdio. Unsupported tool calls are -rejected before DSS token exchange. Agent Management, Govern and Deployer support instance controls -only; `get_current_instance` reports `unsupported` without contacting DSS. The -configured type does not trigger API discovery or select a different SDK client. +Instance types are operator-supplied and enforce the same [tool policy](capabilities.md#instance-policies-and-operation-discovery) as stdio. Unsupported tool calls fail before backend token exchange. Design, Automation and Deployer use the DSS client. Agent Management and Govern remain configurable profiles with instance controls only and report unsupported without a backend call. Authentication, transport, and instance settings are loaded at startup, so restart the server after changing them. User selections are updated in memory and persisted @@ -385,3 +381,7 @@ string-or-array format found in the exchanged token. > delegated token's `sub` must equal the selected Dataiku login or email. Validate this > constraint before production rollout. See Microsoft's > [access-token claims reference](https://learn.microsoft.com/en-us/entra/identity-platform/access-token-claims-reference). + +## Product authentication boundaries + +The shared operation catalog includes Deployer. Deployer supports delegated HTTP authentication. Agent Management and Govern product support is deferred. diff --git a/skills/dataiku-headless-setup/SKILL.md b/skills/dataiku-headless-setup/SKILL.md index d774cc89..152c0905 100644 --- a/skills/dataiku-headless-setup/SKILL.md +++ b/skills/dataiku-headless-setup/SKILL.md @@ -22,7 +22,7 @@ Bring a new or broken plugin installation to a verified Dataiku connection. A re A startup line on stderr followed by exit status 0 is expected. Do not substitute `uv sync --locked --script`: it caches downloads but leaves environment creation for the first server launch. 7. Retry `list_instances`. If the tools are still unavailable, reload the plugin or restart the agent once, then retry before diagnosing a Dataiku connection problem. -8. When `list_instances` succeeds, if no instance is configured, run `configure_instance` and have the user complete the local setup page. Design and Automation require a successful connection test before saving. Agent Management, Govern and Deployer testing is unavailable; save their locally validated profiles directly. If multiple instances exist without an active one, ask which to use and run `switch_instance`. -9. Verify the active profile with `get_current_instance`. If its `connection_status` is `failed`, the discovered profile cannot connect to Dataiku; run `configure_instance` to replace or add a working profile instead of treating it as set up. If it is `unsupported`, the Agent Management, Govern or Deployer profile is configured but its API tools are not implemented; explain this and offer to switch to a supported instance instead of repeating setup or probing DSS. When it is `connected`, make a lightweight read-only Dataiku call such as `list_projects` to validate the available Dataiku access. Never request or repeat the API key in chat. +8. When `list_instances` succeeds, if no instance is configured, run `configure_instance` and have the user complete the local setup page. Supported types require a successful connection test before saving. Agent Management profiles and Govern profiles can be saved without testing while their product support is deferred. If multiple instances exist without an active one, ask which to use and run `switch_instance`. +9. Verify the active profile with `get_current_instance`. If its `connection_status` is `failed`, the discovered profile cannot connect to Dataiku; run `configure_instance` to replace or add a working profile instead of treating it as set up. When it is `connected`, validate access with a read allowed on that type: use the shared operation interface with deployer.list_stages for Deployer and `list_projects` for Design or Automation. Govern and Agent Management report unsupported; explain that product support is deferred and switch to a supported profile instead of probing backend endpoints. Never request or repeat the API key in chat. -Report completion with the active instance name, URL, and configured type, Dataiku version when available, and whether the Dataiku read succeeded or the selected node type has no API tools yet. Include the uv version only when runtime recovery was needed. If a restart is required, say that setup is incomplete and give the single next action. +Report completion with the active instance name, URL, and configured type, Dataiku version when available, and whether the Dataiku read succeeded. Include the uv version only when runtime recovery was needed. If a restart is required, say that setup is incomplete and give the single next action. diff --git a/skills/dataiku-headless/SKILL.md b/skills/dataiku-headless/SKILL.md index 1353336c..9b9f05cc 100644 --- a/skills/dataiku-headless/SKILL.md +++ b/skills/dataiku-headless/SKILL.md @@ -1,47 +1,41 @@ --- name: dataiku-headless -description: Use for Dataiku tasks including projects, flows, datasets, recipes, jobs, machine learning, agents, and Cobuild conversations. Choose the right reference guide, inspect state first, route project changes through Cobuild by default, and validate results after execution. +description: Use for Dataiku tasks including projects, flows, datasets, recipes, jobs, machine learning, agents, deployments, and Cobuild conversations. Choose the right reference guide, inspect state first, route project changes through Cobuild by default, and validate results after execution. --- # Dataiku Headless -Use this for any Dataiku task. Choose the right reference guide first, inspect the current state before acting, route project changes through Cobuild by default, and validate by re-reading the resulting state. +Start by identifying the configured instance type. The default project workflow is for Design. Inspect before acting, use Cobuild for project construction, and verify results. + +## Instance Types + +| Configured type | Purpose | Headless workflow | +| --- | --- | --- | +| `design` | Development environment for building projects, pipelines, models, applications and agents. | Use the Design workflow below; project construction goes through Cobuild. | +| `automation` | Production environment for running deployed project bundles, schedules and monitoring. | Inspect and execute existing assets; return to Design for project edits. | +| `deployer` | Control plane for deploying published project bundles and API service versions. | Use [Deployer operations](./references/deployer.md). | +| `govern` | Governance and sign-off workflows for analytics, models and agents. | Instance controls only; product support is deferred. | +| `agent-management` | Agent inventory, monitoring and risk management. | Instance controls only; product support is deferred. | + +The configured instance type controls which tools are allowed; it does not establish available features or permissions. Use `list_instances` and `get_current_instance` first. ## Shared Operating Rules 1. If the user asks to install, set up, connect, or repair Dataiku Headless, or its MCP tools are unavailable just after installation, read `../dataiku-headless-setup/SKILL.md`. Choose either local stdio or customer-managed HTTP setup and never enable both. -2. Ensure an instance is configured before instance operations. In local stdio mode, if `get_current_instance` errors or `list_instances` is empty, run `configure_instance` first. In HTTP mode, use `list_instances` then `switch_instance`; the instance catalog is platform-managed. Keep the reported `dataiku_version` in context for version-sensitive requests. Check the configured `instance_type` against the policy below before choosing a tool. It controls dispatch; Dataiku permissions and API availability still apply. +2. Ensure an instance is configured before instance operations. In local stdio mode, if `get_current_instance` errors or `list_instances` is empty, run `configure_instance` first. In HTTP mode, use `list_instances` then `switch_instance`; the instance catalog is platform-managed. Keep the reported `dataiku_version` in context for version-sensitive requests. The required `instance_type` selects dispatch policy; follow the instance-type table rather than applying the Design workflow to every node. 3. Discover project keys and object identifiers through tools; do not invent them. 4. Read before write. Inspect the current object, flow context, jobs, or run history before changing anything. 5. Treat the matching reference guide as the source of truth for object-specific concepts, inspection steps, and required references. 6. Route in-project asset creation and modification through `./references/cobuild.md` unless the matching guide documents a narrow direct exception. -7. Direct-write exceptions are allowed for operations that Cobuild cannot perform, especially bootstrap, cross-project, instance-level, or administrative actions. Treat them as narrow documented exceptions, not as the default mutation path. +7. Use direct writes only for exceptions documented in the matching reference guide. Missing Cobuild coverage does not authorize a new direct-write path. 8. Use visual recipes by default. A code recipe is appropriate only when the user explicitly requests a code-based transformation. 9. Preserve surrounding flow, storage, and operational context unless the user requests a change. 10. If MCP or Cobuild coverage is insufficient, stop and report the gap rather than falling back to raw Python, `dataikuapi`, or ad hoc REST calls. Attribute the gap to the Dataiku version only when that requirement is known; otherwise, do not guess. -## Instance Types - -| Configured type | Available work | -| --- | --- | -| `design` | All tools | -| `automation` | Reads, execution of existing assets, instance administration, and all project-folder operations | -| `agent-management`, `govern`, `deployer` | Instance controls only: list, switch, configure, delete, and current profile | - -Automation excludes all Cobuild tools, including conversation listing and polling, -and project creation, project settings/variables updates, container placement, -dataset/folder creation, file upload, and library writes. Builds, recipe/scenario -runs, and job aborts remain available and can change outputs. For project -construction or edits, ask which Design instance to use and -switch to it; do not bypass the policy through raw APIs. - -Agent Management, Govern and Deployer report `connection_status=unsupported` without a DSS probe. -Explain that their API tools are not implemented, then switch to a supported -configured instance for Dataiku work. Local configuration actions remain subject -to the existing HTTP restrictions. The catalog stays visible across all types; -unsupported calls fail before execution. +## Default Workflow — Design Node -## Default Workflow +Use this workflow on a Design node. For other types, follow the instance-type table. +On Automation, use the relevant inspection/execution guide without creating or editing project definitions. 1. Identify the target project and the user's main object of work. 2. Read the matching reference guide from the routing table below. @@ -53,7 +47,7 @@ unsupported calls fail before execution. 8. For tasks that span multiple object types, keep one primary guide and read supporting guides for dependent objects or job/run follow-up. 9. Completion status is not enough. Validate by re-reading the changed object and comparing it with what the user asked for. Where the request is specific about names, titles, or values, check them in the saved settings or a data sample, and mention any requirement you could not confirm. Use `./references/jobs.md` when execution may still be in progress. -## Guide Routing +## Design Guide Routing | User intent | Guide to read next | | --- | --- | @@ -88,5 +82,5 @@ unsupported calls fail before execution. ## Routing Notes - `./references/cobuild.md` is the default write path for in-project assets, but it should be grounded by the relevant guide first. -- Direct writes may grow over time for documented non-project, cross-project, instance-level, or administrative operations that Cobuild does not handle. -- `./references/jobs.md` is the follow-up guide after builds, runs, training, deployment, or other uncertain execution. +- Direct writes are limited to documented exceptions. +- `./references/jobs.md` is the follow-up guide after builds, runs, training, or other uncertain execution. For Deployer updates, follow `./references/deployer.md` for future polling and deployment history. diff --git a/skills/dataiku-headless/references/api-deployer.md b/skills/dataiku-headless/references/api-deployer.md new file mode 100644 index 00000000..daf2135d --- /dev/null +++ b/skills/dataiku-headless/references/api-deployer.md @@ -0,0 +1,42 @@ +--- +name: api-deployer +description: Deploy already published API service versions, select a single target version and verify asynchronous API deployment updates. +--- + +# API Deployer + +Read [the shared Deployer guide](./deployer.md) first for instance selection, +coverage and recovery. Pass arguments through the shared read/write dispatcher. All operations below use `deployer_type="api"`. + +API services are authored in a Design/Automation project's API Designer and +published as versioned packages. API Deployer manages those packages and sends them +to an infrastructure. An API node serves requests; it is not the Deployer profile. +The tool does not configure or invoke prediction endpoints. + +1. Discover infrastructures and stages, then inspect the chosen target and its + reported deployment permissions and governance policy. Infrastructure type + determines available deployment behavior. +2. Discover published services and inspect the chosen source_id to identify its + package versions. Do not confuse a saved-model version with an API package version. +3. Inspect existing deployment configuration, traffic mapping, health and history. +4. Create configuration with deployment_id, source_id (published service ID), + infra_id and version. endpoint_id is optional for Deploy Anywhere infrastructures + that deploy a particular endpoint; use a known endpoint from the package/UI. + deployed_project_key and project_folder_id are project-only and rejected here. +5. Use `deployer.update_deployment(version=...)` to select one service version. + This replaces any existing multi-version traffic mapping with SINGLE_GENERATION. + Check the existing mapping and ensure a single-version transition matches intent. + An older published version can be selected for rollback. +6. Use `deployer.update_deployment(enabled=false)` to save a disabled target, or + enabled=true to save re-enablement. A save alone does not change serving state. + Authentication and other settings are preserved. +7. Apply with `deployer.start_update`, track the future and exact update record, and + re-read deployment health. A successful update does not prove prediction quality; + endpoint invocation and test queries are outside this surface. + +API service authoring/publication, advanced traffic splits, endpoint authentication, +API key handling, infrastructure provisioning and deployment deletion are not +exposed. Inspect the Dataiku UI for these operations and detailed error logs. +Never return raw light status: it can contain service and inherited API keys. + +[Official Python API reference](https://developer.dataiku.com/latest/api-reference/python/api-deployer.html) diff --git a/skills/dataiku-headless/references/deployer.md b/skills/dataiku-headless/references/deployer.md new file mode 100644 index 00000000..e3cbcef7 --- /dev/null +++ b/skills/dataiku-headless/references/deployer.md @@ -0,0 +1,82 @@ +--- +name: deployer +description: Inspect and operate Project and API Deployer deployments, selecting the correct node and tracking asynchronous updates. +--- + +# Deployer + +Deployer is an instance-level control plane with two independent components: +Project Deployer sends published project bundles to Automation nodes; API Deployer +sends published API service versions to API nodes or supported managed infrastructures. +Deployer may be standalone or enabled locally on a Design or Automation node. +All 13 operations, including future polling, use the schema-backed registry described in [Operations](./operations.md). Deployer operations require a profile declared as type `deployer`. +These deployment operations are direct instance-level exceptions to Cobuild routing. +Building the underlying Flow, models or API endpoint logic remains project work. + +## Select the target first + +1. Inspect `list_instances` and `get_current_instance`; select the intended Deployer + using `switch_instance`. Keep its name and reported Dataiku version in context. +2. The configured `instance_type` is metadata. A profile marked deployer does not + verify that Deployer is enabled or the credential has deployment privileges. + For a local Deployer on a Design or Automation node, configure a separate + `deployer` profile pointing at that URL. Design and Automation profiles cannot + call the v2 operation tools. Tools address the selected node; they do not follow its remote Deployer configuration automatically. +3. Read [Project Deployer](./project-deployer.md) for bundles or + [API Deployer](./api-deployer.md) for real-time services. +4. Supply `deployer_type="project"` or `deployer_type="api"` on every call. IDs + belong to that component on that instance, even when names happen to match. + +## Shared inspection and execution + +- `deployer.list_stages` discovers configured stage IDs and descriptions. +- `deployer.list_infrastructures` and `deployer.get_infrastructure` show safe target + summaries, governance policy and associated deployments. Permissions are reported + when the SDK response includes them; absence is not permission to deploy. +- `deployer.list_published_items` discovers projects/services. + `deployer.get_published_item(source_id=...)` returns available packages as a + columnar table with id, publishedOn and publishedBy. Use those IDs as versions. +- `deployer.list_deployments` reports configuration, not live health. + `deployer.get_deployment` adds health, saved version settings, and structured + health/governance codes. Review these before mutation. +- `deployer.create_deployment` creates configuration only. Existing IDs fail through + Dataiku; this tool has no overwrite mode. +- `deployer.update_deployment` saves only a target version and/or API enablement. + Saving is not execution. Other settings are preserved; API version selection + replaces the existing traffic mapping with a single-version mapping. +- `deployer.start_update` applies the saved settings asynchronously. It can replace + production assets, alter API traffic, or disable an API deployment. Inspect the + target and saved version first, and execute only within the user's request. + +## Track the same update + +Keep the selected instance, deployer_type, deployment_id and returned future_id. +Poll `deployer.get_future_status(future_id=...)` on that same instance. Do not use Flow job +IDs or assume a configuration save means an update completed. Then inspect +`deployer.list_updates`, pin the matching startTimestamp with +`deployer.get_update(timestamp=...)`, and re-read `deployer.get_deployment`. +Omitting timestamp reads the latest update, which may belong to another caller. +Completion of a future alone does not establish a healthy deployed service/project. + +If a request times out, is cancelled, returns no future ID, or fails after submission, +inspect deployment state and update history before retrying. Do not resubmit an +uncertain update. Reconnect to the same configured instance when resuming. + +## Coverage limits + +Tools intentionally expose selected operational fields. Raw infrastructure/settings, +API keys, OAuth settings, endpoint code, free-text diagnostics and update logs are +omitted because SDK payloads may contain credentials. Use the Dataiku UI for detailed +diagnostics. Unknown response fields are omitted rather than passed through. + +There are no tools here for infrastructure provisioning, remapping, authentication, +permissions, advanced traffic mappings, deletion, bundle export/upload, API package +publication, or deployment test execution. Work with already published packages. +Report missing coverage; do not bypass it with raw SDK or REST calls. Governance +warnings are not bypassed by create/save operations. Use Cobuild for supported +in-project asset construction; if it cannot publish an artifact, report that gap. + +## Official documentation + +- [Production deployments and bundles](https://doc.dataiku.com/dss/latest/deployment/index.html) +- [API Node and API Deployer](https://doc.dataiku.com/dss/latest/apinode/introduction.html) diff --git a/skills/dataiku-headless/references/operations.md b/skills/dataiku-headless/references/operations.md new file mode 100644 index 00000000..81b47e8f --- /dev/null +++ b/skills/dataiku-headless/references/operations.md @@ -0,0 +1,54 @@ +--- +name: operations +description: Discover and execute Deployer, Govern and Agent Management operations with typed arguments, node policy and transport checks. +--- + +# Shared operation interface + +Deployer profiles use four fixed MCP tools: search_operations, +describe_operation, run_read_operation and run_write_operation. Product actions +are operation IDs, not individually registered MCP tools. The catalog stays stable +across instance switches. Other profile types cannot use these tools. + +## Workflow + +1. Select the intended profile using list_instances, get_current_instance and + switch_instance. A configured node type is policy, not live feature detection. +2. Search by query, domain (deployer), and effect. + Search filters by selected type and returns all matching summaries in catalog + declaration order, without pagination. Narrow the query or filters when useful. + All four v2 tools require a selected profile; search and describe inspect the + local catalog without contacting Dataiku. +3. Describe a returned operation_id. Inspect input_schema, effect, allowed types, + requirements, transports and response_contract. Identifier strings are validated + in the handler before SDK calls; URL delimiters, percent escapes, dot segments + and control characters are rejected. + Definitions and settings vary by backend release, infrastructure or blueprint; + nested objects explicitly described as backend-defined are not fully specified + by the structural schema. Read their current definitions before replacing them. +Never submit redacted placeholders from a read as replacement values; the write +dispatcher rejects them. Prefer a scoped update that preserves hidden fields, or +report the coverage gap. + +4. Discover IDs through reads, then execute with the matching read/write tool and + an arguments object. Example IDs are structural placeholders, not live defaults. + Supply expected_instance with the selected instance name to reject stale context. +5. Inspect the returned result and re-read after mutation. Retain operation/future + IDs and revisions for recovery. Never resubmit an uncertain write automatically. + +Middleware checks the tool’s allowed instance types before exchanging credentials. +Execution handlers check operation IDs, node type, effects, argument shape, selected +instance and transport before calling the SDK. A read tool cannot +execute a mutation. Backend permissions remain authoritative. server_support is +unknown until actual read-only evidence establishes availability; do not infer a +missing feature from a single 404 or probe with writes. + +## Coverage and transport + +- [Deployer](./deployer.md): 13 public SDK operations for Project/API Deployer, + including deployment future polling. Select a Deployer profile even when it + shares a URL with a Design or Automation node. Supports stdio and HTTP. + +Adapters use public Python SDK methods only. Do not fall back to direct endpoint +calls or another identity when coverage is missing. Project construction remains +with Cobuild on supported Design profiles. No dynamic tool registration or code sandbox is used. diff --git a/skills/dataiku-headless/references/project-deployer.md b/skills/dataiku-headless/references/project-deployer.md new file mode 100644 index 00000000..18586fe8 --- /dev/null +++ b/skills/dataiku-headless/references/project-deployer.md @@ -0,0 +1,43 @@ +--- +name: project-deployer +description: Deploy already published project bundles to Automation infrastructures and verify the resulting project deployment. +--- + +# Project Deployer + +Read [the shared Deployer guide](./deployer.md) first for instance selection, +coverage and recovery. Pass arguments through the shared read/write dispatcher. All operations below use `deployer_type="project"`. + +A published project on Deployer is a collection of immutable bundle versions, not +a Design-node project that Cobuild can edit. A deployment associates a published +project and bundle with an Automation infrastructure and target project key. +Stages are infrastructure metadata, not configured instance types. + +1. Discover infrastructures and stages. Inspect the intended infrastructure, + associated deployments, governance policy and reported permissions. +2. Discover published projects and inspect the selected source_id to identify its + package IDs. Do not infer a bundle ID from a Design project name or version. +3. For an existing deployment, inspect its saved settings, health and update history. +4. Create configuration with deployment_id, source_id (published project key), + infra_id and version (bundle ID). Optional deployed_project_key and + project_folder_id select the Automation target key and folder. Without these, + Dataiku uses the published project key and root folder. Discover any folder ID + on the target Automation instance; switch back to Deployer before creating. + Never choose an existing target project key without understanding the overwrite + effects of applying the bundle. +5. To promote or roll back an existing deployment, save a discovered bundle ID using + `deployer.update_deployment(version=...)`. This preserves other deployment settings. + The enabled argument and endpoint_id are API-only and rejected here. +6. Run `deployer.start_update` to apply the saved bundle. This may replace the + Automation project's assets. Follow the future and exact update record, then + re-read deployment health as described in the shared guide. +7. When project/output validation is needed, switch to the configured Automation + instance, discover the actual deployed project and inspect it with the project, + dataset and scenario guides. Deployer success does not establish output parity. + +Bundle creation/publication, deployment remapping, code-environment behavior, +scenario activation and deployment tests are outside this tool surface. They +must already be configured in Dataiku. Do not use an arbitrary settings patch +or Cobuild on the Deployer to emulate those operations. + +[Official Python API reference](https://developer.dataiku.com/latest/api-reference/python/project-deployer.html) diff --git a/tests/test_deployer.py b/tests/test_deployer.py new file mode 100644 index 00000000..633f2bb8 --- /dev/null +++ b/tests/test_deployer.py @@ -0,0 +1,508 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Deployer SDK contract, async boundary and secret-exposure regressions.""" + +import asyncio +import copy +import json +import threading +from types import SimpleNamespace + +import pytest +from dataikuapi.dss.apideployer import DSSAPIDeployer +from dataikuapi.dss.projectdeployer import DSSProjectDeployer +from dataikuapi.utils import DataikuException +from fastmcp import Client +from requests import Timeout + +from dataiku_mcp import mcp +from dataiku_mcp.operations.handlers import deployer + +SECRET = "DO_NOT_RETURN_THIS_SECRET" + + +class Backend: + """Use real SDK handles against an in-memory HTTP boundary.""" + + def __init__(self, kind): + self.kind = kind + self.calls = [] + self.owner_thread = threading.get_ident() + self.settings = { + "id": "deployment", + "infraId": "target", + "bundleId": "old", + "publishedProjectKey": "source", + "publishedServiceId": "source", + "enabled": True, + "generationsMapping": { + "mode": "MULTI_GENERATION", + "entries": [ + {"generation": "old", "proba": 0.5}, + {"generation": "other", "proba": 0.5}, + ], + }, + "auth": {"apiKeys": [{"key": SECRET}]}, + "localVariables": {"credential": SECRET}, + "untouched": {"nested": "preserved"}, + } + self.info = { + "infraBasicInfo": { + "id": "target", + "stage": "stage", + "infraType": "STATIC", + "apiKey": SECRET, + "prePushScript": SECRET, + }, + "projectBasicInfo": {"id": "source", "name": "Source"}, + "serviceBasicInfo": {"id": "source", "name": "Source"}, + "deploymentBasicInfo": {**self.settings, "futureSecret": SECRET}, + "packages": [ + { + "id": "version", + "publishedOn": 42, + "publishedBy": "user", + "endpoints": [{"code": SECRET}], + } + ], + "deployments": [self.settings], + "infras": [{"id": "target", "password": SECRET}], + "canDeploy": True, + "canWrite": False, + "apiKeys": [{"key": SECRET}], + "authorizationInherited": {"apiKeys": [{"key": SECRET}]}, + "oauth2Config": {"clientSecret": SECRET}, + } + self.update = { + "startTimestamp": 123, + "endTimestamp": 456, + "status": "DONE", + "type": "UPDATE", + "logs": {"lines": [SECRET]}, + "error": SECRET, + } + + def _request(self, method, path, kwargs): + assert threading.get_ident() != self.owner_thread, "SDK called on event loop" + self.calls.append((method, path, copy.deepcopy(kwargs))) + if method == "PUT": + self.settings = copy.deepcopy(kwargs["body"]) + return None + if method == "POST" and path.endswith("/actions/update"): + return {"jobId": "future", "hasResult": False} + if method == "POST": + return {} + if path.endswith("/settings"): + return copy.deepcopy(self.settings) + if path.endswith("/governance-status"): + return { + "messages": [ + {"code": "GOVERN", "severity": "WARNING", "details": SECRET} + ] + } + if path.endswith("/status"): + return { + "health": "OUT_OF_SYNC", + "healthMessages": { + "messages": [ + {"code": "SYNC", "severity": "WARNING", "message": SECRET} + ] + }, + "apiNodes": [{"apiKey": SECRET}], + } + if path.endswith("/stages"): + return [{"id": "stage", "desc": "Stage"}] + if path.endswith("/update"): + return [self.update] + if path.endswith("/last-update") or "/update/" in path: + return self.update + if path.rsplit("/", 1)[-1] in {"deployments", "infras", "projects", "services"}: + return [copy.deepcopy(self.info)] + return copy.deepcopy(self.info) + + def _perform_json(self, method, path, **kwargs): + return self._request(method, path, kwargs) + + def _perform_empty(self, method, path, **kwargs): + return self._request(method, path, kwargs) + + +@pytest.fixture(params=["project", "api"]) +def backend(request, monkeypatch): + fake = Backend(request.param) + client = SimpleNamespace( + get_projectdeployer=lambda: DSSProjectDeployer(fake), + get_apideployer=lambda: DSSAPIDeployer(fake), + ) + + def get_client(): + assert threading.get_ident() != fake.owner_thread + return client + + monkeypatch.setattr(deployer, "get_dss_client", get_client) + return fake + + +def call(function, backend, **kwargs): + result = asyncio.run(function(deployer_type=backend.kind, **kwargs)) + assert SECRET not in json.dumps(result) + assert all( + path.startswith(f"/{backend.kind}-deployer/") for _, path, _ in backend.calls + ) + return result + + +@pytest.mark.parametrize( + "function, kwargs", + [ + (deployer.list_deployer_stages, {}), + (deployer.list_deployer_infrastructures, {}), + (deployer.get_deployer_infrastructure, {"infra_id": "target"}), + (deployer.list_deployer_published_items, {}), + (deployer.get_deployer_published_item, {"source_id": "source"}), + (deployer.list_deployer_deployments, {}), + (deployer.get_deployer_deployment, {"deployment_id": "deployment"}), + (deployer.list_deployer_updates, {"deployment_id": "deployment"}), + (deployer.get_deployer_update, {"deployment_id": "deployment"}), + ( + deployer.get_deployer_update, + {"deployment_id": "deployment", "timestamp": "123"}, + ), + ], +) +def test_read_tools_use_sdk_off_loop_and_omit_secrets(backend, function, kwargs): + result = call(function, backend, **kwargs) + assert result + assert all(method == "GET" for method, _, _ in backend.calls) + + +def test_deployment_detail_reports_health_mapping_and_codes(backend): + result = call(deployer.get_deployer_deployment, backend, deployment_id="deployment") + assert result["health"] == "OUT_OF_SYNC" + assert result["settings"]["enabled"] is True + assert result["settings"]["generationsMapping"]["entries"][1] == { + "generation": "other", + "proba": 0.5, + } + assert result["health_messages"] == [{"severity": "WARNING", "code": "SYNC"}] + assert result["governance_messages"] == [{"severity": "WARNING", "code": "GOVERN"}] + + +def test_published_versions_are_discoverable(backend): + result = call(deployer.get_deployer_published_item, backend, source_id="source") + assert result["packages"] == { + "columns": ["id", "publishedOn", "publishedBy"], + "rows": [["version", 42, "user"]], + } + assert result["canDeploy"] is True + assert result["canWrite"] is False + + +def test_create_only_saves_configuration(backend): + options = ( + {"deployed_project_key": "destination", "project_folder_id": "folder"} + if backend.kind == "project" + else {"endpoint_id": "endpoint"} + ) + result = call( + deployer.create_deployer_deployment, + backend, + deployment_id="new", + source_id="source", + infra_id="target", + version="version", + **options, + ) + assert result == { + "deployment_id": "new", + "status": "configuration_created", + "update_started": False, + } + [(method, path, params)] = backend.calls + assert method == "POST" + assert path.endswith("/deployments") + assert params["params"]["ignoreWarnings"] is False + assert ( + params["body"]["bundleId" if backend.kind == "project" else "version"] + == "version" + ) + assert ( + params["body"][ + "publishedProjectKey" if backend.kind == "project" else "publishedServiceId" + ] + == "source" + ) + if backend.kind == "project": + assert params["body"]["deployedProjectKey"] == "destination" + assert params["body"]["projectFolderId"] == "folder" + else: + assert params["body"]["endpointId"] == "endpoint" + + +def test_version_change_preserves_other_settings_and_governance(backend): + before = copy.deepcopy(backend.settings) + result = call( + deployer.update_deployer_deployment, + backend, + deployment_id="deployment", + version="version", + ) + assert result["update_started"] is False + after = backend.settings + assert after["untouched"] == before["untouched"] + assert after["auth"] == before["auth"] + assert after["localVariables"] == before["localVariables"] + if backend.kind == "project": + assert after["bundleId"] == "version" + assert after["generationsMapping"] == before["generationsMapping"] + else: + assert after["generationsMapping"] == { + "mode": "SINGLE_GENERATION", + "generation": "version", + } + assert [method for method, _, _ in backend.calls] == ["GET", "PUT"] + assert backend.calls[-1][2]["params"]["ignoreWarnings"] is False + + +def test_api_disable_preserves_version_mapping(backend): + if backend.kind == "project": + with pytest.raises(ValueError, match="API"): + call( + deployer.update_deployer_deployment, + backend, + deployment_id="deployment", + enabled=False, + ) + assert not backend.calls + else: + mapping = copy.deepcopy(backend.settings["generationsMapping"]) + call( + deployer.update_deployer_deployment, + backend, + deployment_id="deployment", + enabled=False, + ) + assert backend.settings["enabled"] is False + assert backend.settings["generationsMapping"] == mapping + + +def test_start_returns_durable_future_without_polling(backend): + result = call(deployer.start_deployer_update, backend, deployment_id="deployment") + assert result["future_id"] == "future" + assert result["status"] == "update_submitted" + assert [(method, path) for method, path, _ in backend.calls] == [ + ("POST", f"/{backend.kind}-deployer/deployments/deployment/actions/update") + ] + + +@pytest.mark.parametrize("options", [{}, {"version": ""}, {"version": " "}]) +def test_invalid_changes_do_not_call_sdk(backend, options): + with pytest.raises(ValueError): + call( + deployer.update_deployer_deployment, + backend, + deployment_id="deployment", + **options, + ) + assert not backend.calls + + +def test_rejects_component_specific_create_arguments(backend): + options = ( + {"endpoint_id": "endpoint"} + if backend.kind == "project" + else {"project_folder_id": "folder"} + ) + with pytest.raises(ValueError): + call( + deployer.create_deployer_deployment, + backend, + deployment_id="new", + source_id="source", + infra_id="target", + version="version", + **options, + ) + assert not backend.calls + + +@pytest.mark.parametrize("exception", [DataikuException, Timeout]) +def test_sdk_error_has_no_sensitive_text_or_cause(backend, monkeypatch, exception): + def fail(*args, **kwargs): + raise exception(SECRET) + + monkeypatch.setattr(backend, "_perform_json", fail) + with pytest.raises(RuntimeError) as error: + call(deployer.start_deployer_update, backend, deployment_id="deployment") + assert SECRET not in str(error.value) + assert "before retrying" in str(error.value) + assert error.value.__suppress_context__ + + +def test_mcp_schema_rejects_unknown_deployer_type(monkeypatch): + from dataiku_mcp.config import stdio + + monkeypatch.setattr( + stdio, + "_current_instance", + SimpleNamespace(name="test", instance_type="deployer"), + ) + + async def run(): + async with Client(mcp) as client: + with pytest.raises(Exception, match="Invalid operation arguments"): + await client.call_tool( + "run_read_operation", + { + "operation_id": "deployer.list_stages", + "arguments": {"deployer_type": "automation"}, + }, + ) + + asyncio.run(run()) + + +def test_mcp_keeps_deployer_target_when_active_instance_changes(backend, monkeypatch): + from dataiku_mcp.config import request, stdio + from dataiku_mcp.config.models import DSSInstance + + original = DSSInstance( + name="local-deployer", + url="https://local.example", + instance_type="deployer", + api_key="placeholder", + source="config", + no_check_certificate=False, + ) + other = DSSInstance( + name="other", + url="https://other.example", + instance_type="design", + api_key="placeholder", + source="config", + no_check_certificate=False, + ) + monkeypatch.setattr(stdio, "_current_instance", original) + observed = [] + + def get_client(): + stdio._current_instance = other + observed.append(request.get_pinned_instance().name) + assert request.get_pinned_instance().instance_type == "deployer" + return SimpleNamespace( + get_projectdeployer=lambda: DSSProjectDeployer(backend), + get_apideployer=lambda: DSSAPIDeployer(backend), + ) + + monkeypatch.setattr(deployer, "get_dss_client", get_client) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + "run_read_operation", + { + "operation_id": "deployer.list_stages", + "arguments": {"deployer_type": backend.kind}, + }, + ) + assert not result.is_error + + asyncio.run(run()) + assert observed == ["local-deployer"] + assert stdio.get_current_instance() == other + + +def test_deployer_is_exposed_only_as_operations(): + from dataiku_mcp.operations.catalog import OPERATIONS + + tools = asyncio.run(mcp.list_tools()) + assert not any("deployer" in tool.name for tool in tools) + operations = [op for op in OPERATIONS.values() if op.domain == "deployer"] + assert len(operations) == 13 + assert all(op.allowed_instance_types == ("deployer",) for op in operations) + assert OPERATIONS["deployer.start_update"].destructive + assert not OPERATIONS["deployer.list_stages"].destructive + + +def test_start_without_future_id_does_not_fetch_or_leak_result(backend, monkeypatch): + monkeypatch.setattr( + backend, + "_perform_json", + lambda *args, **kwargs: {"hasResult": True, "result": SECRET}, + ) + result = call(deployer.start_deployer_update, backend, deployment_id="deployment") + assert result["future_id"] is None + assert "inspect update history" in result["hint"] + + +def test_dispatch_normalizes_identifiers_before_sdk_calls(backend, monkeypatch): + from dataiku_mcp.config import stdio + from tests.test_instance_policy import _instance + from tests.test_operations import _call + + monkeypatch.setattr(stdio, "_current_instance", _instance("deployer")) + result = asyncio.run( + _call( + "run_read_operation", + operation_id="deployer.get_update", + arguments={ + "deployer_type": backend.kind, + "deployment_id": " deployment ", + "timestamp": " 123 ", + }, + ) + ) + assert result["result"]["status"] == "DONE" + assert [(method, path) for method, path, _ in backend.calls] == [ + ("GET", f"/{backend.kind}-deployer/deployments/deployment/update/123") + ] + + +@pytest.mark.parametrize( + "operation_id,field", + [ + ("deployer.get_infrastructure", "infra_id"), + ("deployer.get_published_item", "source_id"), + ("deployer.get_deployment", "deployment_id"), + ("deployer.list_updates", "deployment_id"), + ("deployer.get_update", "deployment_id"), + ("deployer.get_update", "timestamp"), + ("deployer.create_deployment", "deployment_id"), + ("deployer.create_deployment", "source_id"), + ("deployer.create_deployment", "infra_id"), + ("deployer.create_deployment", "deployed_project_key"), + ("deployer.create_deployment", "project_folder_id"), + ("deployer.create_deployment", "endpoint_id"), + ("deployer.update_deployment", "deployment_id"), + ("deployer.start_update", "deployment_id"), + ("deployer.get_future_status", "future_id"), + ], +) +def test_direct_handlers_validate_identifiers_before_sdk_calls( + monkeypatch, operation_id, field +): + from dataiku_mcp.operations.catalog import get_operation + + monkeypatch.setattr( + deployer, + "get_dss_client", + lambda: pytest.fail("Invalid identifier reached SDK"), + ) + operation = get_operation(operation_id) + arguments = {**operation.example, field: "invalid?identifier"} + if field == "endpoint_id": + arguments["deployer_type"] = "api" + with pytest.raises(ValueError, match=field): + asyncio.run(operation.handler(**arguments)) diff --git a/tests/test_http_middleware.py b/tests/test_http_middleware.py index 35efafcb..25128ce6 100644 --- a/tests/test_http_middleware.py +++ b/tests/test_http_middleware.py @@ -180,6 +180,8 @@ async def get_tagged_tool_names(): "switch_instance", "delete_instance", "configure_instance", + "search_operations", + "describe_operation", "get_cobuild_turn_status", "list_cobuild_conversations", } @@ -238,7 +240,7 @@ def test_http_unknown_tool_fails_without_token_exchange( ] -@pytest.mark.parametrize("instance_type", ["govern", "deployer", "agent-management"]) +@pytest.mark.parametrize("instance_type", ["govern", "agent-management"]) def test_http_unsupported_current_instance_skips_exchange(monkeypatch, instance_type): events = _call_middleware( monkeypatch, "get_current_instance", instance_type=instance_type @@ -306,3 +308,10 @@ async def call_next(_context): asyncio.run(middleware.on_call_tool(context, call_next)) assert events == [("reset_identity", "identity")] + + +def test_http_deployer_status_exchanges_token(monkeypatch): + events = _call_middleware( + monkeypatch, "get_current_instance", instance_type="deployer" + ) + assert ("exchange", "mcp-token") in events diff --git a/tests/test_instance_policy.py b/tests/test_instance_policy.py index 0e549048..88336952 100644 --- a/tests/test_instance_policy.py +++ b/tests/test_instance_policy.py @@ -59,6 +59,8 @@ def _instance(instance_type): def _allowed(tool_name): if tool_name in INSTANCE_TOOLS: return set(get_args(InstanceType)) + if tool_name in EXPECTED_TOOLS_BY_MODULE["operations"]: + return {"deployer"} if tool_name in PROJECT_EDIT_TOOLS: return {"design"} return {"design", "automation"} @@ -99,7 +101,7 @@ def test_invalid_policy_fails_closed_even_for_instance_controls(metadata): @pytest.mark.parametrize( - "instance_type", ["automation", "govern", "deployer", "agent-management"] + "instance_type", ["design", "automation", "govern", "deployer", "agent-management"] ) def test_mcp_dispatch_denies_every_disallowed_tool_before_validation_or_sdk( monkeypatch, instance_type @@ -152,7 +154,7 @@ async def dispatch(): assert calls == [instance_type] -@pytest.mark.parametrize("instance_type", ["govern", "deployer", "agent-management"]) +@pytest.mark.parametrize("instance_type", ["govern", "agent-management"]) def test_mcp_unsupported_status_needs_no_client(monkeypatch, instance_type): from dataiku_mcp.tools import instances diff --git a/tests/test_instances.py b/tests/test_instances.py index cc7e20e0..29ec69fe 100644 --- a/tests/test_instances.py +++ b/tests/test_instances.py @@ -95,7 +95,7 @@ def unexpected_client(): monkeypatch.setattr(instances, "get_dss_client", lambda: client) current = asyncio.run(instances.get_current_instance(FakeContext())) assert json.loads(current)["instance_type"] == instance_type - unsupported = instance_type in {"govern", "deployer", "agent-management"} + unsupported = instance_type in {"govern", "agent-management"} assert client.info_calls == (0 if unsupported else 1) assert json.loads(current)["connection_status"] == ( "unsupported" if unsupported else "connected" diff --git a/tests/test_onboarding_ui.py b/tests/test_onboarding_ui.py index 2541615f..c3a92ff9 100644 --- a/tests/test_onboarding_ui.py +++ b/tests/test_onboarding_ui.py @@ -34,10 +34,7 @@ def test_setup_page_allows_govern_save_without_a_connection_test(): assert "Advanced options" in page assert 'name="no_check_certificate"' in page assert 'name="action" value="save" disabled' not in page - assert ( - "Agent Management, Govern and Deployer profiles can be saved without testing." - in page - ) + assert "Agent Management and Govern profiles can be saved without testing." in page assert "button:disabled, button:disabled:hover" in page diff --git a/tests/test_operation_redaction.py b/tests/test_operation_redaction.py new file mode 100644 index 00000000..feea0950 --- /dev/null +++ b/tests/test_operation_redaction.py @@ -0,0 +1,63 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +import asyncio +import json +from types import SimpleNamespace + +import pytest + +from dataiku_mcp.config import stdio +from dataiku_mcp.operations.handlers import deployer +from tests.test_instance_policy import _instance +from tests.test_operations import _call + +SECRET = "do-not-return-this-secret" + + +def test_deployer_future_status_does_not_fetch_results(monkeypatch): + monkeypatch.setattr(stdio, "_current_instance", _instance("deployer")) + captured = [] + + def perform(method, path, **kwargs): + captured.append((method, path, kwargs)) + return {"hasResult": True, "result": {"password": SECRET}} + + monkeypatch.setattr( + deployer, "get_dss_client", lambda: SimpleNamespace(_perform_json=perform) + ) + result = asyncio.run( + _call( + "run_read_operation", + operation_id="deployer.get_future_status", + arguments={"future_id": "future"}, + ) + ) + assert result["result"]["state"] == {"hasResult": True} + assert captured == [("GET", "/futures/future", {"params": {"peek": True}})] + + +@pytest.mark.parametrize( + "value", + [ + {"apiKeys": [SECRET]}, + {"passwords": [SECRET]}, + {"url": "https://user:" + SECRET + "@example.com/path?api_key=" + SECRET}, + ], +) +def test_operation_redaction_covers_secret_containers_and_url_credentials(value): + from dataiku_mcp.utils.redaction import redact_sensitive_data + + assert SECRET not in json.dumps(redact_sensitive_data(value)) diff --git a/tests/test_operations.py b/tests/test_operations.py new file mode 100644 index 00000000..5ca21524 --- /dev/null +++ b/tests/test_operations.py @@ -0,0 +1,423 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +"""Operation discovery, schemas, execution policy and request pinning.""" + +import asyncio +import json +from types import SimpleNamespace +from dataclasses import replace + +import pytest +from fastmcp import Client +from dataikuapi.dss.apideployer import DSSAPIDeployer +from dataikuapi.dss.projectdeployer import DSSProjectDeployer + +from dataiku_mcp import mcp, server +from dataiku_mcp.config import http, request, stdio +from dataiku_mcp.operations.catalog import OPERATIONS, _DEPLOYER_OPERATIONS +from dataiku_mcp.operations.catalog import get_operation +from dataiku_mcp.operations.handlers import deployer +from dataiku_mcp.tools import operations +from tests.test_instance_policy import _instance +from tests.test_deployer import Backend + + +async def _call(name, **arguments): + async with Client(mcp) as client: + result = await client.call_tool(name, arguments) + return json.loads(result.content[0].text) + + +@pytest.mark.parametrize( + "tool, arguments", + [ + ("search_operations", {}), + ("describe_operation", {"operation_id": "deployer.list_stages"}), + ( + "run_read_operation", + {"operation_id": "deployer.list_stages", "arguments": {}}, + ), + ( + "run_write_operation", + { + "operation_id": "deployer.start_update", + "arguments": { + "deployer_type": "project", + "deployment_id": "deployment", + }, + }, + ), + ], +) +def test_all_v2_tools_require_selected_instance(monkeypatch, tool, arguments): + monkeypatch.setattr(stdio, "_current_instance", None) + monkeypatch.setattr(stdio, "get_instances", lambda: {}) + monkeypatch.setattr( + server, + "exchange_http_token", + lambda *_: pytest.fail("No exchange without a selected profile"), + ) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool(tool, arguments, raise_on_error=False) + assert result.is_error + assert "No Dataiku instances are configured" in result.content[0].text + assert request._pinned_instance.get() is None + + asyncio.run(run()) + + +def _http_setup(monkeypatch, kind="deployer"): + profiles = {kind: _instance(kind)} + monkeypatch.setattr( + http, + "get_instances_and_selections", + lambda: (profiles, {"issuer": {"alice": kind}}), + ) + monkeypatch.setattr( + server, + "get_access_token", + lambda: SimpleNamespace( + token="mcp-token", claims={"iss": "issuer", "sub": "alice"} + ), + ) + events = [] + + async def exchange(_): + events.append("exchange") + return "backend-token" + + monkeypatch.setattr(server, "exchange_http_token", exchange) + return events + + +def test_deployer_registry_uses_existing_sdk_handler(monkeypatch): + backend = Backend("project") + monkeypatch.setattr(stdio, "_current_instance", _instance("deployer")) + monkeypatch.setattr( + deployer, + "get_dss_client", + lambda: SimpleNamespace( + get_projectdeployer=lambda: DSSProjectDeployer(backend), + get_apideployer=lambda: DSSAPIDeployer(backend), + ), + ) + result = asyncio.run( + _call( + "run_write_operation", + operation_id="deployer.start_update", + arguments={"deployer_type": "project", "deployment_id": "deployment"}, + ) + ) + assert result["result"]["future_id"] == "future" + assert len(backend.calls) == 1 + + +@pytest.mark.parametrize("kind", ["design", "automation"]) +@pytest.mark.parametrize( + "tool, arguments", + [ + ("search_operations", {}), + ("describe_operation", {"operation_id": "deployer.list_stages"}), + ( + "run_read_operation", + { + "operation_id": "deployer.list_stages", + "arguments": {"deployer_type": "project"}, + }, + ), + ( + "run_write_operation", + { + "operation_id": "deployer.start_update", + "arguments": { + "deployer_type": "project", + "deployment_id": "deployment", + }, + }, + ), + ], +) +def test_v2_tools_reject_design_and_automation_before_exchange( + monkeypatch, kind, tool, arguments +): + events = _http_setup(monkeypatch, kind) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool(tool, arguments, raise_on_error=False) + assert result.is_error + assert f"instance type '{kind}'" in result.content[0].text + assert not request.is_http_request() + assert request._pinned_instance.get() is None + + asyncio.run(run()) + assert not events + + +def test_each_v2_operation_requires_its_matching_instance_type(monkeypatch): + for operation in OPERATIONS.values(): + assert operation.allowed_instance_types == (operation.domain,) + + async def handler(**arguments): + assert arguments == operation.validate(operation.example) + return {"ok": True} + + monkeypatch.setattr( + operations, "get_operation", lambda _: replace(operation, handler=handler) + ) + for kind in ("design", "automation", "agent-management", "deployer", "govern"): + monkeypatch.setattr(request, "get_pinned_instance", lambda: _instance(kind)) + call = operations._dispatch( + operation.operation_id, operation.example, operation.effect, None + ) + if kind == operation.domain: + assert json.loads(asyncio.run(call))["result"] == {"ok": True} + else: + with pytest.raises(ValueError, match="unavailable for instance type"): + asyncio.run(call) + + +def test_transport_rejected_before_sdk_execution(monkeypatch): + _http_setup(monkeypatch) + operation = replace(get_operation("deployer.list_stages"), transports=("stdio",)) + monkeypatch.setattr(operations, "get_operation", lambda _: operation) + monkeypatch.setattr( + deployer, + "get_dss_client", + lambda: pytest.fail("Unsupported transport reached SDK"), + ) + with pytest.raises(Exception, match="unavailable over transport 'http'"): + asyncio.run( + _call( + "run_read_operation", + operation_id=operation.operation_id, + arguments={"deployer_type": "project"}, + ) + ) + + +def test_operation_ids_are_unique(): + ids = [operation.operation_id for operation in _DEPLOYER_OPERATIONS] + assert len(ids) == len(set(ids)) + + +def test_all_schemas_accept_examples_and_are_closed(): + assert sum(op.domain == "deployer" for op in OPERATIONS.values()) == 13 + for operation in OPERATIONS.values(): + operation.validate(operation.example) + description = operation.describe() + assert description["input_schema"]["additionalProperties"] is False + assert description["server_support"] == "unknown" + assert description["requirements"] + assert description["response_contract"] + assert description["effect"] in {"read", "write"} + assert not {"category", "backend", "local_file_fields"} & description.keys() + + +def test_search_returns_all_matches_and_preserves_tool_catalog(monkeypatch): + monkeypatch.setattr(stdio, "_current_instance", _instance("deployer")) + + async def run(): + before = {tool.name for tool in await mcp.list_tools()} + result = await _call("search_operations", domain="deployer") + assert result["total"] == 13 + assert [op["operation_id"] for op in result["operations"]] == list(OPERATIONS) + assert set(result) == {"instance_name", "server_support", "total", "operations"} + assert all("input_schema" not in op for op in result["operations"]) + writes = await _call("search_operations", query="DEPLOYMENT", effect="write") + assert [op["operation_id"] for op in writes["operations"]] == [ + "deployer.create_deployment", + "deployer.update_deployment", + "deployer.start_update", + ] + assert writes["total"] == 3 + reads = await _call("search_operations", effect="read") + assert reads["total"] == 10 + assert all(op["effect"] == "read" for op in reads["operations"]) + empty = await _call("search_operations", query="no-such-operation") + assert empty["total"] == 0 + assert empty["operations"] == [] + search = await mcp.get_tool("search_operations") + assert set(search.parameters["properties"]) == {"query", "domain", "effect"} + detail = await _call("describe_operation", operation_id="deployer.start_update") + assert detail["allowed_on_selected_instance"] is True + assert {tool.name for tool in await mcp.list_tools()} == before + + asyncio.run(run()) + + +@pytest.mark.parametrize( + "tool,operation_id,arguments", + [ + ("run_read_operation", "deployer.start_update", {}), + ("run_write_operation", "deployer.list_stages", {"deployer_type": "project"}), + ("run_read_operation", "not-in-catalog", {}), + ( + "run_read_operation", + "deployer.list_stages", + {"deployer_type": "project", "path": "/admin"}, + ), + ("run_read_operation", "deployer.list_stages", 42), + ( + "run_write_operation", + "deployer.update_deployment", + {"deployer_type": "api", "deployment_id": "deployment", "enabled": "false"}, + ), + ], +) +def test_invalid_operation_rejected_before_sdk_execution( + monkeypatch, tool, operation_id, arguments +): + events = _http_setup(monkeypatch) + monkeypatch.setattr( + deployer, + "get_dss_client", + lambda: pytest.fail("Invalid operations must not reach SDK"), + ) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + tool, + {"operation_id": operation_id, "arguments": arguments}, + raise_on_error=False, + ) + assert result.is_error + assert not request.is_http_request() + assert request._pinned_instance.get() is None + + asyncio.run(run()) + assert events == ["exchange"] + + +@pytest.mark.parametrize( + "bad_id", + [ + "", + ".", + "..", + " .. ", + "x/y", + "x\\y", + "%2e%2e", + "%252f", + "x?y", + "x#y", + "a\n", + "\ta", + "a\x00b", + "a\x7f", + " ", + ], +) +def test_path_escape_rejected_before_sdk_execution(monkeypatch, bad_id): + _http_setup(monkeypatch) + monkeypatch.setattr( + deployer, + "get_dss_client", + lambda: pytest.fail("Invalid identifier reached SDK"), + ) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + "run_read_operation", + { + "operation_id": "deployer.get_infrastructure", + "arguments": {"deployer_type": "project", "infra_id": bad_id}, + }, + raise_on_error=False, + ) + assert result.is_error + + asyncio.run(run()) + + +def test_expected_instance_rejects_stale_selection_before_sdk_execution(monkeypatch): + _http_setup(monkeypatch) + monkeypatch.setattr( + deployer, "get_dss_client", lambda: pytest.fail("Stale selection reached SDK") + ) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + "run_read_operation", + { + "operation_id": "deployer.list_stages", + "arguments": {"deployer_type": "project"}, + "expected_instance": "other", + }, + raise_on_error=False, + ) + assert result.is_error + assert "Selected instance changed" in result.content[0].text + + asyncio.run(run()) + + +def test_http_pinning_survives_switch_during_exchange(monkeypatch): + profiles = {kind: _instance(kind) for kind in ("deployer", "design")} + selection = {"issuer": {"alice": "deployer"}} + _http_setup(monkeypatch) + monkeypatch.setattr( + http, "get_instances_and_selections", lambda: (profiles, selection) + ) + + async def exchange(_): + selection["issuer"]["alice"] = "design" + await asyncio.sleep(0) + return "backend-token" + + monkeypatch.setattr(server, "exchange_http_token", exchange) + + def get_client(): + assert request.get_pinned_instance().name == "deployer" + assert request.get_http_dss_token() == "backend-token" + return SimpleNamespace( + get_projectdeployer=lambda: SimpleNamespace(list_stages=lambda: []) + ) + + monkeypatch.setattr(deployer, "get_dss_client", get_client) + result = asyncio.run( + _call( + "run_read_operation", + operation_id="deployer.list_stages", + arguments={"deployer_type": "project"}, + ) + ) + assert result["instance_name"] == "deployer" + assert not request.is_http_request() + + +def test_redacted_credentials_cannot_be_written_back(): + operation = get_operation("deployer.update_deployment") + with pytest.raises(ValueError, match="redacted result placeholders"): + operation.validate( + {"deployer_type": "project", "deployment_id": "__DATAIKU_REDACTED__"} + ) + + +def test_identifier_schemas_use_plain_strings(): + schema = get_operation("deployer.get_infrastructure").describe()["input_schema"] + assert schema["properties"]["infra_id"] == {"title": "Infra Id", "type": "string"} + schema = get_operation("deployer.get_update").describe()["input_schema"] + assert schema["properties"]["timestamp"]["anyOf"] == [ + {"type": "string"}, + {"type": "null"}, + ] diff --git a/tests/test_setup_server.py b/tests/test_setup_server.py index e30114d2..0cbd323b 100644 --- a/tests/test_setup_server.py +++ b/tests/test_setup_server.py @@ -169,7 +169,7 @@ def _post_setup_form(url: str, form: dict[str, str]) -> tuple[int, str]: return error.code, error.read().decode() -@pytest.mark.parametrize("instance_type", ["design", "automation"]) +@pytest.mark.parametrize("instance_type", ["design", "automation", "deployer"]) def test_setup_server_requires_a_successful_test_before_saving( monkeypatch, instance_type ): @@ -235,7 +235,7 @@ def test_setup_server_invalidates_a_test_when_connection_settings_change( ) -@pytest.mark.parametrize("instance_type", ["govern", "deployer", "agent-management"]) +@pytest.mark.parametrize("instance_type", ["govern", "agent-management"]) def test_unsupported_profile_can_be_saved_after_unavailable_test( monkeypatch, instance_type ): @@ -276,7 +276,8 @@ def test_unsupported_profile_can_be_saved_after_unavailable_test( session.close() -def test_design_connection_test_uses_dss_client(monkeypatch): +@pytest.mark.parametrize("instance_type", ["design", "deployer"]) +def test_supported_connection_test_uses_dss_client(monkeypatch, instance_type): calls = [] class Client: @@ -292,6 +293,8 @@ def get_auth_info(self): monkeypatch.setattr(setup_server.dataikuapi, "DSSClient", Client) setup_server._test_connection( - _validate_form({**_form("https://example.com"), "instance_type": ["design"]}) + _validate_form( + {**_form("https://example.com"), "instance_type": [instance_type]} + ) ) assert calls == ["client", "auth"] diff --git a/tests/test_tool_surface.py b/tests/test_tool_surface.py index 378dd222..2fb78f31 100644 --- a/tests/test_tool_surface.py +++ b/tests/test_tool_surface.py @@ -164,6 +164,14 @@ "upload_file_to_managed_folder", } ), + "operations": frozenset( + { + "search_operations", + "describe_operation", + "run_read_operation", + "run_write_operation", + } + ), "plugins": frozenset( { "install_plugin",