A test suite can be green while the product looks broken. That is not a failure of discipline. It is a property of how most user-interface tests are built.
What Standard UI Tests Cannot See
Automated UI tests check structure: that a button exists, carries the right label, and does the right thing when pressed. They cannot see a collapsed margin, a dark mode that renders black text on a black background, or a label that truncates when a customer enlarges the font. The structure is unchanged, so every test passes.
Screenshot tests close that gap. Google describes the approach as taking "a screenshot of a piece of UI" and comparing it "against a previously approved reference image"; if the images differ, "the test fails and produces an HTML report" (Google). Google's testing strategy uses exactly this as its example of a component test: a "Screenshot test for a custom button."
Why It Is Now Cheap Enough
The modern tools render on a build machine rather than on phones or emulators, which puts them in the category Google calls local tests: "usually small and fast" (Google). That changes screenshot testing from a periodic visual audit into a check on every code change. Three tools lead: Google's own preview-based tool, Cash App's Paparazzi, and Roborazzi, which can drive the app into a state before capturing it.
Adopt It, With Eyes Open
Google's tool is still marked experimental and "subject to change substantially during the alpha phase," and it has just changed its setup model: from Android Gradle Plugin 9.5.0-alpha03, Google recommends configuring it through the build system's test suites, and "the standalone plugin method described on this page is deprecated." That argues for a measured start, not for waiting.
- Start with the design system. Shared components change least and break the most screens at once when they regress.
- Record and verify in one environment. Google notes that pixel-perfect comparison requires screenshots taken "in the same conditions," which in practice means the CI server.
- Keep the review human. Image changes belong in the same code review as the change that caused them. A suite whose diffs nobody reads becomes noise.
-Rocky
#TestAutomation #QualityEngineering #JetpackCompose #EngineeringDreams #StrategiaX
Originally published on ANDROID-ARCHITECT, the Android engineering blog of Strategia-X.