Tests with @TestOn('browser') are not executed by bots

[polina-c]

I added this test and the bots passed:

@TestOn('browser')
import 'package:flutter_test/flutter_test.dart';
 
void main() {
 test('I must fail', () async {
   expect(1, 2);
 });
}

We need to migrate all tests with this annotation.
Related issue: #2437

[polina-c]

Related discussion:

Kenzie: Hey Yegor! Is there updated guidance for writing tests for Flutter web? We are hitting an issue in DevTools right now where we wanted to add "TestOn("browser")" to our test so that it runs on a web target, and discovered that that is no longer supported. In reading through a rabbit hole of issues (example flutter/flutter#50594), I also discovered that flutter test --platform=chrome is no longer supported / recommended. Everything I read was from 2020, so wanted to check if there was some up-to-date guidance on this. thanks.

Yegor: flutter test --platform=chrome or some variant of it is here to stay. Flutter itself uses it extensively. The reason we don't recommend it is because many people try to use this mode for integration tests where they need a real world browser environment. However, this mode tried to emulate the VM flutter_tester environment where many things are mocked out. Even the passage of time is faked using FakeAsync. So people get confused when they try to do something really asynchronous, like make an HTTP request, and it doesn't work.
However, we don't mind when this mode is used by advanced users. Googlers use it in google3 and in Fuchsia and we don't get many complaints. To test something in real time and without monkey-patched browser environment we recommend package:integration_test. Some examples: https://github.com/flutter/flutter/tree/master/dev/integration_tests/web_e2e_tests/test_driver

[polina-c]

nullsafety must be enabled to run tests with test --target chrome:

flutter/flutter#74898

[kenzieschmoll]

Now that we have completed the null safety migration, we may be able to revisit this. @polina-c

[srawlins]

This has been resolved. packages/devtools_extensions/test/web/devtools_extension_test.dart is now the only test in the repo annotated with @TestOn('browser'), and it is executed by the bots: tool/ci/package_tests.sh runs flutter test test/web --platform chrome for the devtools_extensions package. It is skipped on Windows only, which is tracked by flutter/flutter#162798.

The related issue #2437 has also been closed. Closing this as fixed.

If there is more work to do here, please let us know by filing a new issue with up to date information. Thanks!