Skip to content

feat(NoTicket): send NULL output format for server-side async queries - #555

Open
pscls wants to merge 1 commit into
mainfrom
feat/async-null-output-format
Open

pscls wants to merge 1 commit into
mainfrom
feat/async-null-output-format

Conversation

@pscls

@pscls pscls commented Oct 21, 2026 •

Copy link
Copy Markdown

Summary

Newer Firebolt versions accept server-side async SELECT queries if the request asks for no result serialization. This PR makes the SDK send output_format=NULL for every execute_async call. Before, it sent JSON_Compact.

  • _get_output_format returns the new NULL_OUTPUT_FORMAT constant for async execution. This covers the fb_numeric, qmark and qmark bulk-insert planners.
  • Requesting streaming and async execution together now fails an assert.
  • Docs: async SELECT queries are accepted, but their results can't be retrieved yet.
  • New integration test test_select_async, added to both the sync and asyncio test suites.

⚠️ Before merging

Integration tests don't run on PRs. If staging's default engine is older than the version that accepts output_format=NULL, the async integration tests will fail, including the existing INSERT ones. Run the Integration tests FB 2.0 workflow by hand with engine-version set to a version that supports this, and only merge once that version is on staging.

Testing

  • Unit tests: 794 passed, 1 skipped
  • black, isort and mypy pass
  • Integration tests not run yet (see above)

🤖 Generated with Claude Code


Note

Medium Risk
Changes the HTTP contract for all execute_async calls; requires a Firebolt engine version that accepts output_format=NULL, or existing async integration tests will fail.

Overview
Server-side execute_async requests now send output_format=NULL instead of JSON_Compact, so newer Firebolt engines can accept async SELECT when no result serialization is requested.

BaseStatementPlanner._get_output_format takes an async_execution flag and returns the new NULL_OUTPUT_FORMAT constant for async paths (fb_numeric, qmark, and bulk insert). Combining streaming and async now fails an assertion.

Docs clarify that async SELECT may be submitted but results cannot be fetched yet; fetch* after execute_async still raises MethodNotAllowedInAsyncError. Sync/async integration tests add test_select_async; unit tests and HTTP mocks expect NULL on async query URLs.

Reviewed by Cursor Bugbot for commit eedb55f. Bugbot is set up for automated code reviews on this repo. Configure here.

Newer Firebolt versions accept async SELECT queries when no result
serialization is requested, so async queries are now sent with
output_format=NULL. Streaming combined with async execution is rejected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pscls
pscls requested a review from a team as a code owner October 21, 2026 10:05
@sonarqubecloud

Copy link
Copy Markdown

Comment on lines +140 to +141
if async_execution:
return NULL_OUTPUT_FORMAT

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Major: This makes every execute_async depend on the engine accepting NULL, not just async SELECT. Engines from before the async SELECT change reject the Null output format without advanced_mode ("Output format 'NULL' is not supported, choose one of: ..."). So after the SDK upgrade, async INSERT / COPY / VACUUM break against any older engine. The "Before merging" note covers staging, but not customers. Firebolt Core users pin image tags. The SDK release is not tied to the engine rollout. I see three options. We keep JSON_Compact as the default and send NULL only for SELECT (when the statement is parsed). Or we fall back to JSON_Compact on that specific format error. Or we hold the release until every supported engine and Core image has the engine change, and document the minimum version. I'd prefer one of the first two, so the SDK stays compatible with older engines.


.. note::
Fetching data via SELECT is not supported and will raise an exception. execute_async is best suited for DML queries.
Async SELECT queries are accepted, but there is currently no way to retrieve their results - execute_async is best suited for DML queries.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Minor: This should say from which Firebolt version async SELECT is accepted. Older engines still reject it. With this PR as is, they would reject all async queries from this SDK version (see the comment on statement_planners.py). Also, the " - " reads like a broken dash. Two sentences would be cleaner.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants