Emit --disable-features once instead of letting a later one discard it - #2219
Open
alkaz-nodemaven wants to merge 1 commit into
Open
Emit --disable-features once instead of letting a later one discard it#2219alkaz-nodemaven wants to merge 1 commit into
alkaz-nodemaven wants to merge 1 commit into
Conversation
Chrome parses --disable-features last-wins: given the switch twice it keeps the last value and silently drops every earlier one, whole. crawl4ai builds its flag list by appending, so in two configurations the browser never received the defaults: light_mode=True in effect: TranslateUI extra_args=[--disable-features] in effect: the user's names only Both discard OptimizationHints, MediaRouter and DialMediaRouteProvider. Read off the running Chrome's command line via Win32_Process, not off the options object. merge_feature_switches() folds repeats of --disable-features and --enable-features into a single switch at the position of the first occurrence, so the surrounding flag order is untouched and nothing declared is lost. Wired into build_browser_flags(), the launch_persistent_context args and _build_browser_args(). Also fixes the feature name itself: "TranslateUI" is absent from the shipped Chromium binary, so the switch matched nothing. The feature is "Translate".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Chrome parses
--disable-featureslast-wins: given the switch twice it keeps thelast value and silently discards every earlier one, whole. There is no warning, and
the options object still shows both, so nothing about the Python side looks wrong.
build_browser_flags()and_build_browser_args()both assemble the flag list byappending, and two ordinary configurations put a second
--disable-featuresafter thedefault one. In those two configurations crawl4ai's own
OptimizationHints,MediaRouter,DialMediaRouteProvidernever reaches the browser.Measured on
develop@023c87d, Windows 10, Playwright 1.62 / Chromium 151.0.7922.34.The command line is read back off the running process through
Win32_Process, becausewhat Playwright receives and what
CreateProcessgets are not the same list:--disable-featuresswitches on the command lineOptimizationHints, MediaRouter, DialMediaRouteProviderlight_mode=TrueTranslateUIextra_args=["--disable-features=CalculateNativeWinOcclusion"]CalculateNativeWinOcclusionThe two bold rows are the bug.
light_mode=TrueappendsBROWSER_DISABLE_OPTIONS,whose own
--disable-featuresentry lands last and takes the browser with it, soturning light mode on is what switches the default three off. Same for any user who
passes their own
--disable-featuresthroughextra_args- they get their names andnone of crawl4ai's, which is very likely not what they meant by "extra".
After the patch, all four configurations emit exactly one switch and nothing crawl4ai
declared is dropped:
OptimizationHints, MediaRouter, DialMediaRouteProviderlight_mode=TrueOptimizationHints, MediaRouter, DialMediaRouteProvider, Translateextra_args=[...]OptimizationHints, MediaRouter, DialMediaRouteProvider, CalculateNativeWinOcclusionThe second fix:
TranslateUIis not a feature nameBROWSER_DISABLE_OPTIONScarried--disable-features=TranslateUI. That name does notexist in Chromium any more; the feature is
Translate. Chrome ignores an unknownfeature name silently, so the switch has been matching nothing.
Behaviour cannot tell "the flag worked" from "the flag was ignored", so this was checked
against the binary instead - every
base::Featurecarries its name as a string literal,and a literal that is absent cannot be matched at parse time. Counting whole-word
occurrences across
chrome.dllandchrome.exeof Chromium 151.0.7922.34:Absence is conclusive here; presence only says the name exists somewhere in the build.
The last row is a name nobody could have implemented, included so that "ABSENT" is
shown to be a result the test can produce rather than an artefact of how it searches.
List of files changed and why
crawl4ai/browser_manager.pymerge_feature_switches()- new module-level helper. Folds repeats of--disable-featuresand--enable-featuresinto one switch each, emitted at theposition of the first occurrence so the surrounding flag order is untouched.
Names keep first-seen order and are de-duplicated.
ManagedBrowser.build_browser_flags()- merge after the existing dedupe.BrowserManagerlaunch_kwargs["args"](thelaunch_persistent_contextpath) -same.
BrowserManager._build_browser_args()- same.BROWSER_DISABLE_OPTIONS-TranslateUI->Translate, with the reason in acomment.
It merges rather than drops the duplicate on purpose:
build_browser_flags()alsofeeds
ManagedBrowser._get_browser_args(), which spawns Chrome directly with noPlaywright in the picture, so there is no outer list to defer to.
tests/regression/test_reg_browser.py- a new "Launch flags" section with threetests: the helper's own contract (including that
--enable-featuresis foldedseparately and that a list with nothing to fold is returned unchanged),
build_browser_flags()under bothlight_modevalues, and_build_browser_args()with a user
--disable-featuresinextra_args.How Has This Been Tested?
pytest tests/regression/test_reg_browser.py- 33 passed ondevelop, 36 passedwith this branch, so the 33 pre-existing tests are unaffected and the 3 new ones
are additions rather than adjustments.
developfirst and all three failthere, each for its own reason -
2 == 1on the switch count for both call sites,ImportErrorfor the helper. A regression test that passes before the fix is nottesting the fix.
pytest --doctest-modules crawl4ai/browser_manager.py- passes; the helper'sdocstring carries a worked example.
chromium.launch(args=...)per configuration, with the resultingcommand line read back through
Win32_Process. Both tables above come from that,base and patched, run in the same session on the same host so the two sides differ
only in the patch.
blackon the added block only. The rest of both files is not currentlyblack-clean, and reformatting it would have buried this diff.
Two things this does not claim
No performance or bandwidth claim.
OptimizationHintscoming back on is acorrectness result, not a measured saving. I looked for the Optimization Guide
model download in idle traffic and did not observe it in these runs, so there is
nothing to quote.
The collision with Playwright's own list is not fixed, and cannot be fixed here.
Playwright emits its own
--disable-featuresfromchromiumSwitches()and then doeschromeArguments.push(...args), so crawl4ai's single switch still lands afterPlaywright's 16-name one and displaces it. Measured, patched branch, defaults:
13 of Playwright's names are dropped,
HttpsUpgradesandPaintHoldingamong them.This is true of any library that passes
--disable-featuresthrough Playwright,it is unchanged by this PR in either direction, and Playwright exposes no merge API to
fix it from the outside. Worth knowing about; a separate problem.
Checklist:
changes beyond the flags now taking effect, so there is nothing documented to update