Accessibility Scan with Playwright

Extend Playwright E2E Tests with Level CI Accessibility Analysis

Level CI integrates accessibility analysis into your existing Playwright end-to-end tests through the @level-ci/a11y-playwright npm package. This library works with Playwright’s Page instance, collects accessibility insights during test execution, and saves reports in the level-ci-reports directory.

1. Configure `.npmrc` and Install `@level-ci/a11y-playwright` Package

Follow these steps to configure your project and install the Level CI Playwright package:

A. Configure your `.npmrc` file

Run the following commands in the root of your project to configure the registry and authentication:


npm config --location project set @level-ci:registry https://npm.levelaccess.net/level-ci/
npm config --location project set //npm.levelaccess.net/level-ci/:\_authToken=\${CLOUDSMITH_TOKEN}

B. Declare your Cloudsmith Token

Export your Cloudsmith token as an environment variable:


export CLOUDSMITH_TOKEN=<token>

(Replace the example token above with your actual token.)

C. Install `@level-ci/a11y-playwright`

Install the package in the root of your Playwright E2E project:


npm install --save-dev @level-ci/a11y-playwright

For more details on installation, refer to the Level CI documentation.

2. Update Playwright E2E Tests with `setupLevelCI`

Import and invoke the setupLevelCI() function in a setup or test initialization file (e.g., setupPlaywright.ts) to initialize accessibility analysis. Then import this setup file in all your tests.


// playwright/e2e/setupPlaywright.ts
 
import { setupLevelCI } from '@level-ci/a11y-playwright'
 
test.beforeAll(async () => {
  setupLevelCI({
    printViolationsTable: true,
    reportPath: './level-ci-reports',
  })
})

For more information about configuration options, see the Level CI Playwright documentation.

3. Update Playwright E2E Tests with `levelAnalyze`

Add the levelAnalyze() function call in your test cases to perform accessibility analysis at any point. Each invocation generates a report saved in level-ci-reports.


// playwright/e2e/my-test.js
 
const { levelAnalyze } = require('@level-ci/a11y-playwright')
 
test('example test', async ({ page }) => {
  await page.goto('http://localhost:5000')
  await levelAnalyze(page)
})

Best Practice: Invoke levelAnalyze() at the end of each test case.

Advanced Usage:


await levelAnalyze(page, {
  strict: false,
  savePageOrigin: true,
  saveReport: true,
})

For full configuration options, see the API documentation.

4. Optional Configuration

TypeScript Users: Update your tsconfig.json to include the package types:


{
  "compilerOptions": {
    "types": ["playwright", "@level-ci/a11y-playwright"]
  }
}

Git Ignore Reports: Add the report directory to .gitignore to avoid committing generated reports:


level-ci-reports

5. Verify Configuration

Validate your setup before committing:

Apply all the steps above.
Run your E2E tests locally.
Ensure there are no errors and all tests pass.
Check level-ci-reports/ for JSON files. The number of reports should match the number of levelAnalyze() invocations. Example file: level-ci-report-lvoeobzh.json.

For troubleshooting, refer to the Level CI troubleshooting documentation.

6. Prerequisites

Compatible with Chrome and Chromium only
Playwright v15.3.0 or higher
Node.js v16.20.2 or higher