Repository navigation
Conversation
Coverage (unit tests)No unit baseline recorded in
Line and branch coverage from unit test reports. History is recorded in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Five fixes to
:visual-testing, all in the Gemini path ofBaseVisualTestandGeminiVisualTestHelper.1. A Gemini pass no longer overwrites a golden that diverged. With
updateGoldenOnPass = true(the default), a screenshot that failed the pixel diff but got a Gemini "PASSED" replaced the golden ingoldensDirectory. That directory is checked before the bundled assets, so on a reused device or emulator the next runs fast-passed against the regressed screenshot without asking Gemini again. NowupdateGoldenOnPassonly records a golden when none existed, which keeps the first-run recording flow. A diverged golden is left alone and the log explains how to accept an intended change.2. The default pass condition rejects negated verdicts.
it.contains("PASSED", ignoreCase = true)also matched "NOT PASSED" and "BYPASSED". The new default (isPassedVerdict, internal) passes only when the last whole-wordPASSED/FAILEDin the response isPASSEDand is not preceded by "not". The samples and docs that passed the old lambda explicitly now rely on the default.3. The API key goes in the
x-goog-api-keyheader. It used to be a?key=query parameter on every request, so it could end up in logs, proxies and exception messages that include the URL.4. The model is pinned by default. With no configuration, the helper used to call
models.listand pick the newest stable Flash model, so verdicts could change from one day to the next without any change in the code under test. It now usesDEFAULT_FALLBACK_MODELwithout a network call. Discovery is still available as an opt-in:preferredModel = GeminiVisualTestHelper.LATEST_MODELor-Dgemini.model=latest(new, additive constant).5. Docs warn against
src/main/assets/goldens. The baseline lookup still checks it, but the KDoc now points tosrc/androidTest/assets, because files undersrc/main/assetsship in the release APK.6. The emulator job works again. It was failing on
mainand on every PR, and this PR now also includes the fix from #11 (closed in favour of this one):kotlinx-coroutines-androidis added to the sample app. Its runtime classpath resolved coroutines 1.9.0, and AGP pins the test APK to it ({strictly 1.9.0}), so Ktor 3.6 crashed withNoSuchMethodErrorinJob.invokeOnCompletionon its first request. It now resolves{strictly 1.11.0}.DEFAULT_REQUEST_TIMEOUT_MILLIS, covered by a test). With the crash fixed, the Gemini request inVisualMapSampleTesthit CIO's default 15 s timeout.The two belong in the same PR as the header change. Ktor's timeout exception includes the request URL, and with the key in
?key=that URL ended up in the uploaded test report.7. snippet-bot passes on the skill docs. The skills referred to code regions as
`// [START tag]`in prose (Reference: ...,See ...) right above a copy of the same region, so snippet-bot saw each tag started twice or never ended (40 mismatches across the 9 skill files onmain). It only checks changed files, so any PR touching the skills failed. Those prose mentions now read "regiontag"; the copied code blocks keep their real tags. A local scan of every.md,.ktand.javafile finds no mismatched tags.helperinBaseVisualTestis nowopenso tests can inject a client (additive, see the.apidiff).Tests:
ktor-client-mockis added as a test-only dependency of:visual-testing; nothing changes for consumers. With it:VisualVerdictTestcovers the verdict rules.GoldenFallbackLogicTestcovers a diverged golden staying untouched, a missing golden being recorded, and "NOT PASSED" failing the test.GeminiModelResolutionTestcovers the pinned default (no request made),latestdiscovery, its fallback and session cache, and the key being sent only in the header. Two of its existing tests used to call the real Gemini endpoint with a dummy key and now use the mock engine.The doc snippets copied from the
maps_visual_golden_fallbackregion were re-synced from the source, and the#Lline ranges that point at it were updated.Ran
./gradlew test lint spotlessCheck apiCheck docs:dokkaGenerate :app:assembleDebugAndroidTest --rerun-tasks --no-build-cache: all green. The emulator job on this PR is the check for the device side.BEGIN_COMMIT_OVERRIDE
fix(visual): never overwrite a diverged golden after a Gemini pass
fix(visual): reject negated verdicts in the default pass condition
fix(visual): send the Gemini API key in a header instead of the URL
feat(visual): pin the default Gemini model and make latest Flash discovery opt-in
docs(visual): warn against bundling goldens in src/main/assets
fix(visual): give Gemini requests a 60s timeout
fix: align sample app coroutines with Ktor so visual tests run on device
docs(skills): refer to code regions by name so snippet-bot can parse the skills
END_COMMIT_OVERRIDE