Skip to content

feat!: pass the skipped Request to on_skipped_request - #2374

Open
vdusek wants to merge 1 commit into
v2from
feat/skipped-request-callback-request
Open

vdusek wants to merge 1 commit into
v2from
feat/skipped-request-callback-request

Conversation

@vdusek

@vdusek vdusek commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator

The on_skipped_request callback gets the skipped Request as its first argument, so handlers can read its label and user_data. In v1 it got only the URL string.

All skip paths pass a Request: add_requests (string URLs go through Request.from_url), requests fetched from the request queue, and the robots.txt skips in extract_links of both the HTTP and Playwright crawlers. For those links, the Request carries the label, user_data and enqueue strategy passed to enqueue_links. transform_request_function isn't applied to them.

The docs example and the upgrade guide are updated.

Closes: #2007

✍️ Drafted by Claude Code

@vdusek vdusek added the t-tooling Issues with this label are in the ownership of the tooling team. label Oct 31, 2026
@vdusek vdusek self-assigned this Oct 31, 2026
@codecov

codecov Bot commented Oct 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.72727% with 6 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (v2@a65cd99). Learn more about missing BASE report.

Files with missing lines Patch % Lines
.../crawlers/_abstract_http/_abstract_http_crawler.py 62.50% 3 Missing ⚠️
...rawlee/crawlers/_playwright/_playwright_crawler.py 62.50% 3 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff          @@
##             v2    #2374   +/-   ##
=====================================
  Coverage      ?   94.30%           
=====================================
  Files         ?      187           
  Lines         ?    13855           
  Branches      ?        0           
=====================================
  Hits          ?    13066           
  Misses        ?      789           
  Partials      ?        0           
Flag Coverage Δ
unit 94.30% <72.72%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek
vdusek marked this pull request as ready for review October 31, 2026 11:48
@vdusek
vdusek requested review from Mantisus and Pijukatel October 31, 2026 11:48

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant