SaySpec
GitHub Blog Contact us Get SaySpec app

Blog

Automating Flutter tests: why mobile is harder than web

On the web, end-to-end tests are a solved problem. On mobile they are still hard work. Here is why, and the most practical way to automate a Flutter app today.

· 4 min read

Illustration of a robotic hand tapping a phone emulator on a laptop, next to a tree of passed and failed test steps.

Web testing is the easy case

With Playwright, a web test is a few lines of code. It starts a real browser in about a second, runs headless on any CI machine, reads the page’s DOM directly and waits for elements on its own. To try a change, you reload the page.

Mobile has none of these shortcuts.

Why mobile tests are harder

  • You need a device. iOS simulators only run on a Mac with Xcode. Android emulators need hardware virtualization and several gigabytes of memory, and they take a while to boot. In CI that means macOS runners, emulator-capable Linux machines or a device farm, all slower and more expensive than a headless browser.
  • You need to build the app. Every run starts with a Gradle or Xcode build that can take minutes, then an install on the device. There is no “just reload the page”.
  • The operating system gets in the way. Permission dialogs, notifications, the share sheet and system sign-in screens are not part of your app, but your users see them, so your tests have to get through them too.
  • State sticks around. App data, granted permissions and keychain entries survive between runs unless you clear them, so one test can quietly break the next.

Why Maestro and Appium struggle with Flutter

Maestro, and Appium with its standard Android and iOS drivers, are black-box tools. They drive the compiled app from the outside and see only what the operating system’s accessibility layer reports.

Flutter doesn’t use native views. It draws every pixel itself, so the only thing these tools can see is the semantics tree Flutter exposes for screen readers. In practice that means:

  • Text widgets show up, but icon buttons, images and custom widgets often don’t, until someone wraps them in Semantics or adds a semanticLabel.
  • Widget keys are invisible. For stable selectors you add Semantics(identifier: …) across your codebase, or you match on visible text and fix your tests whenever the copy or a translation changes.
  • The test can’t reach inside the app. It can’t mock the network, seed data or switch a feature flag, so every precondition has to be tapped through the UI.

Maestro is still a good tool. Its YAML flows are quick to write and it waits for the UI on its own, which makes it a fine choice for a handful of smoke tests. The limits show up as a Flutter suite grows.

Patrol: tests that live inside your app

Flutter’s own integration_test package runs inside the app, so it finds widgets by key, type or text and can call your app’s code. But it can’t touch anything outside the app. It can’t even tap “Allow” on a permission dialog.

Patrol, the open-source framework from LeanCode, builds on integration_test and adds native automation: granting permissions, opening and tapping notifications, pressing the home button and more, on Android and iOS. Tests are plain Dart next to your app code, so they reuse your models and helpers. They run through the native test runners, so they also work in CI and on device farms such as Firebase Test Lab.

That’s why we recommend Patrol for Flutter apps. The catch is the start: Patrol needs native setup in your Android and iOS projects, and every test is code someone has to write. That’s fine for Flutter developers, and a wall for a QA team that tests by hand today.

The easiest way to start a Patrol suite

SaySpec is the easiest way to turn the manual tests you already do into automated tests written with Patrol:

  1. Show it. Run your app on an Android emulator or iOS simulator and walk through the flow the way you test it today.
  2. Say it. Explain what should happen while you do it: “after I apply the coupon, the total drops by $20”.
  3. Run it again. SaySpec records your taps and screenshots and transcribes your narration on your Mac. Your coding agent, Claude Code or Codex, writes the Patrol test and runs it.

SaySpec’s machine setup installs Flutter, Patrol and the Android and iOS tooling, and it can add its small recorder package to your app for you. Saved tests run again on demand, and when one fails, SaySpec explains why and proposes a fix that is applied only after you approve it. The desktop app is open source.