Skip to content

feat(request-loaders)!: Add get_pending_count to RequestLoader - #2368

Open
Mantisus wants to merge 3 commits into
apify:v2from
Mantisus:get-pending-count
Open

Mantisus wants to merge 3 commits into
apify:v2from
Mantisus:get-pending-count

Conversation

@Mantisus

Copy link
Copy Markdown
Collaborator

Description

RequestLoader has a new abstract get_pending_count method, which returns the number of requests that haven't been handled yet, like getPendingCount in Crawlee JS. RequestQueue, SitemapRequestLoader, RequestManagerTandem and ThrottlingRequestManager implement it. RequestList reads its source lazily, so it counts only the fetched requests that aren't handled yet.

RequestManagerTandem.get_total_count no longer counts a request twice once it moves from the loader to the manager, so the crawler's status message shows the right total. The v2 upgrade guide covers the change for custom request loaders.

Issues

✍️ Drafted by Claude Code

@Mantisus Mantisus self-assigned this Oct 30, 2026
@github-actions github-actions Bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Oct 30, 2026
@Mantisus
Mantisus requested review from Pijukatel and vdusek October 30, 2026 01:18

@vdusek vdusek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor comment, otherwise LGTM

Mantisus and others added 3 commits October 31, 2026 15:47
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants