Skip to content

feat(core): route interaction prompt ids and timestamps via providers - #2144

Open
DABH wants to merge 2 commits into
NVIDIA:developfrom
DABH:hitl-prompt-provider-routing
Open

feat(core): route interaction prompt ids and timestamps via providers#2144
DABH wants to merge 2 commits into
NVIDIA:developfrom
DABH:hitl-prompt-provider-routing

Conversation

@DABH

@DABH DABH commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Description

PR #2114 added the opt-in, process-wide provider hooks in nat.utils.providers and routed the runtime's generated identifiers and timestamps through them: Runner workflow run and trace ids, IntermediateStepPayload defaults, Span ids and timestamps, and Context.push_active_function invocation ids. One runtime surface was left out: human-in-the-loop interactions.

UserInteractionManager.prompt_user_input still stamps the outgoing InteractionPrompt with an inline uuid.uuid4() id, and stamps both the prompt and the returned InteractionResponse with wall-clock timestamps (time.gmtime() with no argument, which reads the system clock). Third-party plugin packages (docs/source/extend/third-party-plugins.md) that build on the toolkit's plugin system and install fixed providers for reproducible runs — record or replay style testing, golden-file trace comparison, or runtimes that re-execute workflow code and need identifiers to remain stable across re-executions — therefore still observe nondeterministic interaction ids and timestamps, so two otherwise identical runs cannot correlate their interaction prompts.

This PR completes the coverage:

  • The interaction prompt id is generated through generate_id().
  • Both the prompt and the response timestamps are derived from current_time() via time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(current_time())), preserving the exact existing timestamp format.
  • The deterministic identifiers section of the observability guide (docs/source/run-workflows/observe/observe.md) and the nat.utils.providers module docstring now list interaction prompts and responses among the values the runtime stamps through the hooks, so the two descriptions stay consistent.

Zero behavior change by default: when no provider is installed, the defaults remain uuid.uuid4 and time.time, and time.gmtime(time.time()) is equivalent to the previous time.gmtime() call.

No tracking issue exists for this yet; happy to file one if the team prefers.

Testing

  • New test test_prompt_user_input_uses_installed_providers in packages/nvidia_nat_core/tests/nat/builder/test_interactive.py, alongside the existing UserInteractionManager tests: it installs a fixed id provider and an advancing time provider (restored in finally), drives prompt_user_input through a stub callback, and asserts the InteractionPrompt handed to the callback and the returned InteractionResponse carry the deterministic id, the expected statuses, and each the formatted UTC timestamp of its own time-provider reading — proving the two timestamps come from two independent current_time() calls.
  • uv run pytest packages/nvidia_nat_core/tests/nat/builder/test_interactive.py packages/nvidia_nat_core/tests/nat/utils/test_providers.py — 21 passed.
  • Full core suite: uv run pytest packages/nvidia_nat_core — 2613 passed, 235 skipped.
  • uv run pre-commit run yapf --files <touched files> and uv run pre-commit run ruff-check --files <touched files> — both passed.
  • uv run python ci/scripts/copyright.py --verify-apache-v2 — passed.
  • uv run python ci/scripts/path_checks.py — passed.
  • uv run vale docs/source/run-workflows/observe/observe.md — 0 errors, 0 warnings.

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
    • Any contribution which contains commits that are not Signed-Off will not be accepted.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

Summary by CodeRabbit

  • Enhancements

    • Interaction prompts and responses now receive generated unique identifiers and wall-clock timestamps.
    • Interaction metadata can consistently reflect the prompt and response lifecycle while preserving existing statuses and callback behavior.
  • Documentation

    • Updated runtime metadata documentation to describe identifiers and timestamps assigned to interaction prompts and responses.

…e provider hooks

UserInteractionManager.prompt_user_input still stamps the interaction
prompt id with uuid.uuid4() and the prompt and response timestamps with
time.gmtime() wall-clock values called inline, so human-in-the-loop
interactions are the one remaining runtime surface that ignores the
opt-in id and time provider hooks introduced in nat.utils.providers.
Reproducible runs that install fixed providers therefore still see
nondeterministic interaction ids and timestamps.

Route the prompt id through generate_id() and derive both timestamps
from current_time() via time.gmtime(current_time()), preserving the
exact existing timestamp format. With the default providers this is
behavior-identical to the previous inline calls. Mention the
interaction prompt and response coverage in the deterministic
identifiers section of the observability guide and in the
nat.utils.providers module docstring, which enumerates the surfaces
the runtime stamps through the hooks.

Add a test alongside the existing UserInteractionManager tests that
installs a fixed id provider and an advancing time provider (restoring
them afterwards), drives prompt_user_input through a stub callback, and
asserts the InteractionPrompt and InteractionResponse carry the
deterministic id and that each is stamped with the formatted timestamp
of its own time provider reading.

Signed-off-by: David Hyde <DABH@users.noreply.github.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

Interaction prompts and responses now receive IDs and timestamps from runtime providers. The interactive test verifies deterministic provider behavior, statuses, content, callback output, and state restoration. Documentation describes these entities as runtime-stamped.

Changes

Interaction metadata

Layer / File(s) Summary
Provider integration
packages/nvidia_nat_core/src/nat/builder/user_interaction_manager.py, packages/nvidia_nat_core/src/nat/utils/providers.py, docs/source/run-workflows/observe/observe.md
prompt_user_input uses generate_id() and current_time() for prompt and response metadata. Documentation lists interaction prompts and responses among runtime-stamped entities.
Provider-driven interaction test
packages/nvidia_nat_core/tests/nat/builder/test_interactive.py
The test verifies generated IDs, distinct timestamps, interaction statuses, content, callback output, and provider state restoration.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise, descriptive, uses imperative mood, and accurately identifies the provider-based routing of interaction IDs and timestamps.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@DABH
DABH marked this pull request as ready for review August 5, 2026 20:17
@DABH
DABH requested a review from a team as a code owner August 5, 2026 20:17
@DABH DABH changed the title feat(core): generate interaction prompt ids and timestamps through the provider hooks feat(core): route interaction prompt ids and timestamps via providers Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant