ref(spans): Run process-segments as a task in devenv - #122555
Merged
untitaker merged 1 commit intoAug 25, 2026
Merged
Conversation
Segments are already processed by the `process_segment` task in production. Flip the default of `spans.buffer.process-segments-task-rollout-rate` to 1.0 and drop the dedicated `process-segments` Kafka consumer from devservices and the devserver, so devenv matches production. ref STREAM-1568
untitaker
marked this pull request as ready for review
August 25, 2026 08:43
| "spans.buffer.process-segments-task-rollout-rate", | ||
| type=Float, | ||
| default=0.0, | ||
| default=1.0, |
Contributor
There was a problem hiding this comment.
Bug: If apply_async_with_future returns None, span segments are silently dropped because there is no fallback to the Kafka-based processing path.
Severity: MEDIUM
Suggested Fix
When task_produce_future is None after calling apply_async_with_future, either log an error to make the failure visible or implement a fallback to the Kafka-based produce call. This ensures segments are not silently lost when the task cannot be queued.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: src/sentry/options/defaults.py#L3308
Potential issue: In `src/sentry/spans/consumers/process/flusher.py`, when processing
span segments, the code attempts to use a task-based approach if the
`spans.buffer.process-segments-task-rollout-rate` option is enabled. If the call to
`apply_async_with_future` returns `None`, the corresponding segment is silently dropped.
There is no fallback to the Kafka-based processing mechanism and no error is logged,
leading to silent data loss. This change makes the task-based approach the default in
development environments, increasing the risk of this issue occurring if the task broker
is misconfigured or unavailable.
Also affects:
devservices/config.yml:157~162src/sentry/runner/commands/devserver.py:336~336
Did we get this right? 👍 / 👎 to inform future reviews.
evanh
approved these changes
Aug 25, 2026
untitaker
deleted the
markusunterwaditzer/stream-1568-deploy-process-segments-tasks-in-devenv-and-self-hosted
branch
August 25, 2026 13:43
untitaker
added a commit
that referenced
this pull request
Aug 27, 2026
Segments have been processed by the `process_segment` task in production for a while, and `spans.buffer.process-segments-task-rollout-rate` has defaulted to 1.0 since #122555. This removes the now-dead Arroyo consumer and everything that only existed to feed it: - the `process-segments` consumer and its strategy factory - `spans.buffer.process-segments-task-rollout-rate`, and the flusher branch that produced to Kafka instead of spawning the task - `MultiProducer` and the `buffered-segments` topics - `skip_produce`, which was only set by the consumer's `--skip-produce` flag, and `start_new_transaction`, which the task always passed as `False` The segment processing helpers that `factory.py` shared with the task move to `tasks.py`, alongside their only remaining caller. `test_buffer.py` still validates the flushed payload against the `buffered-segments` schema, since that schema describes what the flusher hands to the task. It looks the codec up by name now that the topic has no enum entry. The ops-side shutdown is getsentry/ops#22790. ref STREAM-1590
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Segments are already processed by the
process_segmenttask in production.This flips the default of
spans.buffer.process-segments-task-rollout-rateto1.0 and drops the dedicated
process-segmentsKafka consumer from devservicesand the devserver, so devenv matches production.
The flusher spawns the task directly, so no raw-mode taskbroker topic is
involved.
Self-hosted counterpart: getsentry/self-hosted#4488
ref STREAM-1568