Skip to content

Emit --disable-features once instead of letting a later one discard it - #2219

Open
alkaz-nodemaven wants to merge 1 commit into
unclecode:developfrom
alkaz-nodemaven:fix/merge-duplicate-feature-switches
Open

Emit --disable-features once instead of letting a later one discard it#2219
alkaz-nodemaven wants to merge 1 commit into
unclecode:developfrom
alkaz-nodemaven:fix/merge-duplicate-feature-switches

Conversation

@alkaz-nodemaven

Copy link
Copy Markdown

Summary

Chrome parses --disable-features last-wins: given the switch twice it keeps the
last value and silently discards every earlier one, whole. There is no warning, and
the options object still shows both, so nothing about the Python side looks wrong.

build_browser_flags() and _build_browser_args() both assemble the flag list by
appending, and two ordinary configurations put a second --disable-features after the
default one. In those two configurations crawl4ai's own
OptimizationHints,MediaRouter,DialMediaRouteProvider never reaches the browser.

Measured on develop @ 023c87d, Windows 10, Playwright 1.62 / Chromium 151.0.7922.34.
The command line is read back off the running process through Win32_Process, because
what Playwright receives and what CreateProcess gets are not the same list:

configuration --disable-features switches on the command line names Chrome actually applies
defaults 2 OptimizationHints, MediaRouter, DialMediaRouteProvider
light_mode=True 3 TranslateUI
extra_args=["--disable-features=CalculateNativeWinOcclusion"] 3 CalculateNativeWinOcclusion

The two bold rows are the bug. light_mode=True appends BROWSER_DISABLE_OPTIONS,
whose own --disable-features entry lands last and takes the browser with it, so
turning light mode on is what switches the default three off. Same for any user who
passes their own --disable-features through extra_args - they get their names and
none of crawl4ai's, which is very likely not what they meant by "extra".

After the patch, all four configurations emit exactly one switch and nothing crawl4ai
declared is dropped:

configuration switches names applied
defaults 1 OptimizationHints, MediaRouter, DialMediaRouteProvider
light_mode=True 1 OptimizationHints, MediaRouter, DialMediaRouteProvider, Translate
extra_args=[...] 1 OptimizationHints, MediaRouter, DialMediaRouteProvider, CalculateNativeWinOcclusion

The second fix: TranslateUI is not a feature name

BROWSER_DISABLE_OPTIONS carried --disable-features=TranslateUI. That name does not
exist in Chromium any more; the feature is Translate. Chrome ignores an unknown
feature name silently, so the switch has been matching nothing.

Behaviour cannot tell "the flag worked" from "the flag was ignored", so this was checked
against the binary instead - every base::Feature carries its name as a string literal,
and a literal that is absent cannot be matched at parse time. Counting whole-word
occurrences across chrome.dll and chrome.exe of Chromium 151.0.7922.34:

TranslateUI                       0  ABSENT - cannot be matched
Translate                        59  present
OptimizationHints                 5  present
MediaRouter                      48  present
DialMediaRouteProvider            2  present
NodeMavenDefinitelyNotAFeature    0  ABSENT   <- negative control

Absence is conclusive here; presence only says the name exists somewhere in the build.
The last row is a name nobody could have implemented, included so that "ABSENT" is
shown to be a result the test can produce rather than an artefact of how it searches.

List of files changed and why

  • crawl4ai/browser_manager.py

    • merge_feature_switches() - new module-level helper. Folds repeats of
      --disable-features and --enable-features into one switch each, emitted at the
      position of the first occurrence so the surrounding flag order is untouched.
      Names keep first-seen order and are de-duplicated.
    • ManagedBrowser.build_browser_flags() - merge after the existing dedupe.
    • BrowserManager launch_kwargs["args"] (the launch_persistent_context path) -
      same.
    • BrowserManager._build_browser_args() - same.
    • BROWSER_DISABLE_OPTIONS - TranslateUI -> Translate, with the reason in a
      comment.

    It merges rather than drops the duplicate on purpose: build_browser_flags() also
    feeds ManagedBrowser._get_browser_args(), which spawns Chrome directly with no
    Playwright in the picture, so there is no outer list to defer to.

  • tests/regression/test_reg_browser.py - a new "Launch flags" section with three
    tests: the helper's own contract (including that --enable-features is folded
    separately and that a list with nothing to fold is returned unchanged),
    build_browser_flags() under both light_mode values, and _build_browser_args()
    with a user --disable-features in extra_args.

How Has This Been Tested?

  • pytest tests/regression/test_reg_browser.py - 33 passed on develop, 36 passed
    with this branch
    , so the 33 pre-existing tests are unaffected and the 3 new ones
    are additions rather than adjustments.
  • The three new tests were run against unpatched develop first and all three fail
    there, each for its own reason - 2 == 1 on the switch count for both call sites,
    ImportError for the helper. A regression test that passes before the fix is not
    testing the fix.
  • pytest --doctest-modules crawl4ai/browser_manager.py - passes; the helper's
    docstring carries a worked example.
  • End to end: a real chromium.launch(args=...) per configuration, with the resulting
    command line read back through Win32_Process. Both tables above come from that,
    base and patched, run in the same session on the same host so the two sides differ
    only in the patch.
  • black on the added block only. The rest of both files is not currently
    black-clean, and reformatting it would have buried this diff.

Two things this does not claim

  • No performance or bandwidth claim. OptimizationHints coming back on is a
    correctness result, not a measured saving. I looked for the Optimization Guide
    model download in idle traffic and did not observe it in these runs, so there is
    nothing to quote.

  • The collision with Playwright's own list is not fixed, and cannot be fixed here.
    Playwright emits its own --disable-features from chromiumSwitches() and then does
    chromeArguments.push(...args), so crawl4ai's single switch still lands after
    Playwright's 16-name one and displaces it. Measured, patched branch, defaults:

    [14] --disable-features=AvoidUnnecessaryBeforeUnloadCheckSync,...,OptimizationHints,...   <- Playwright
    [57] --disable-features=OptimizationHints,MediaRouter,DialMediaRouteProvider              <- crawl4ai, wins
    

    13 of Playwright's names are dropped, HttpsUpgrades and PaintHolding among them.
    This is true of any library that passes --disable-features through Playwright,
    it is unchanged by this PR in either direction, and Playwright exposes no merge API to
    fix it from the outside. Worth knowing about; a separate problem.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation - no user-facing behaviour
    changes beyond the flags now taking effect, so there is nothing documented to update
  • I have added/updated unit tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Chrome parses --disable-features last-wins: given the switch twice it keeps
the last value and silently drops every earlier one, whole. crawl4ai builds
its flag list by appending, so in two configurations the browser never
received the defaults:

  light_mode=True                 in effect: TranslateUI
  extra_args=[--disable-features] in effect: the user's names only

Both discard OptimizationHints, MediaRouter and DialMediaRouteProvider.
Read off the running Chrome's command line via Win32_Process, not off the
options object.

merge_feature_switches() folds repeats of --disable-features and
--enable-features into a single switch at the position of the first
occurrence, so the surrounding flag order is untouched and nothing declared
is lost. Wired into build_browser_flags(), the launch_persistent_context
args and _build_browser_args().

Also fixes the feature name itself: "TranslateUI" is absent from the shipped
Chromium binary, so the switch matched nothing. The feature is "Translate".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant