assertNoDefectsWithAI

AI-powered visual testing to detect UI defects and anomalies.

This is an experimental feature that uses LLM technology. All feedback is welcome.

The assertNoDefectsWithAI command takes a screenshot of the current view and sends it to an LLM to analyze for common visual defects. The command checks for issues such as text or UI elements that are cut off, overlapping, or not centered correctly within their containers.

Use this command as a general smoke test to verify that UI elements in your application render as expected.

Command specifications

The assertNoDefectsWithAI accepts only one parameter:

Parameter	Type	Description
optional	boolean	Optional. Determines if the Flow should continue if the assertion fails. Default is true.

Since assertNoDefectsWithAI is an experimental feature, optional is set to true by default to prevent unstable AI responses from breaking your CI/CD pipelines. If you want a failed AI assertion to stop the test, you must explicitly set optional: false.

Output

The command generates an analysis report in both HTML and JSON formats. The output files are saved in the directory for the specific test run.

~/.maestro
└── tests
    ├── 2024-08-20_213616
    │   ├── ai-(My first flow).json
    │   ├── ai-(My second flow).json
    │   ├── ai-report-(My first flow).html
    │   ├── ai-report-(My second flow).html

The HTML report provides a visual summary of the findings.

AI analysis report showing a screenshot with highlighted defects.

Usage examples

The following video demonstrates the command in action.

Related content

Access the AI test analysis to learn how to configure the workspace to use the AI based solutions.