web-infra-dev /midscene

See interfaces like a human. E2E testing in natural language.

★ 15,616⑂ 1,223👁 69⑃ 2,697● 689▤ 228
TypeScriptMITCreated 2024-07-23midscenejs.com
Download: v1.17.8v1.17.8 ↓1.17.82026-11-11
+6
Daily
15,616
stars
1,223
forks
69
watchers
2,697
prs
689
issues
228
releases

README

See interfaces like a human. E2E testing in natural language.
  Bring AI vision to E2E testing with a newly designed test framework for web, mobile, and desktop apps.



  Website · English / 简体中文



  
  
  
  
  


 Write a test in natural language

Midscene.js brings AI vision to E2E testing. Describe UI steps and expected results in natural language, then use the visual GUI Agent to carry them out and verify what users see. Organize tests with Midscene Test, or integrate the Agent APIs into your existing testing framework.

 YAML test cases with Midscene Test

After creating a Midscene Test project and configuring a model and the Web runtime, write a case like this. Replace the example URL with your app's address:

yaml
beforeEach:
  - gotoUrl: https://your-shop.example

cases:
  - name: Find headphones under $100
    steps:
      - aiAct: Search for headphones and filter the results to prices under $100
      - aiWaitFor: The filtered search results are visible
      - aiAssert: There are products in the search results, and each costs less than $100


 JavaScript in your existing tests

You can also use the Agent directly in your existing tests. Configure a model and open your app in a Playwright page, then describe the same steps in code:

javascript
import { PlaywrightAgent } from '@midscene/web/playwright';

const agent = new PlaywrightAgent(page);

await agent.aiAct('Search for headphones and filter the results to prices under $100');
await agent.aiWaitFor('The filtered search results are visible');
await agent.aiAssert('There are products in the search results, and each costs less than $100');


For mobile and desktop apps, use the platform Agent with Rstest or Vitest. Open the generated HTML report to inspect screenshots, element locations, the AI decision process, and action and assertion results.

 👁️ See, act, and verify like a human

E2E testing verifies complete workflows from the user's perspective, so Midscene tests the way a user does: observe the screen, act on what you see, and verify the visible result.

 Locate and act from screenshots

Midscene locates elements by their appearance and position, then clicks, types, or scrolls to carry out your instructions. You can target icon-only buttons, custom controls, , and elements inside cross-origin iframes without maintaining selectors or adding semantic annotations.

 Verify what users see

Describe the expected appearance in natural language to check colors, selection highlights, layout, and visual feedback — including content drawn on  or displayed in native apps.

javascript
await agent.aiAssert('The selected plan has a blue border and a checkmark');
await agent.aiAssert('The error message is visible below the email field');


 🌐 One API, across platforms

Visual localization does not depend on the UI rendering stack, so the same Agent APIs work on every platform:

| Platform | Integration |
| --- | --- |
| Web | Playwright, Puppeteer, or desktop Chrome |
| Android | Devices a

Top issues

Latest release

v1.17.8 v1.17.8 2026-11-11
English

 v1.17.8

 Changes

 Features 🎉
 feat(core): add Tencent HY model adapter by @EAGzzyCSL in https://github.com/web-infra-dev/midscene/pull/3362
 feat(test): remove automatic credential redaction from run artifacts by @xiong35 in https://github.com/web-infra-dev/midscene/pull/3371
 feat(badge): support skipped test counts by @yuyutaotao in https://github.com/web-infra-dev/midscene/pull/3380

 Fixes 🐞
 fix(webdriver): enforce request timeout through response body reads by @quanru in https://github.com/web-infra-dev/midscene/pull/3232
 fix(web-integration): honor explicit slow swipe durat

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON