Auto Playwright

Run Playwright tests using AI.

Setup

1. Install auto-playwright dependency:

``bash
npm install auto-playwright -D
`

2. This package relies on talking with OpenAI (https://openai.com/). You must export the API token as an enviroment variable or add it to your .env file:

`bash
export OPENAI_API_KEY='sk-..."
`

3. Import and use the auto function:

`ts
import { test, expect } from "@playwright/test";
import { auto } from "auto-playwright";

test("auto Playwright example", async ({ page }) => {
await page.goto("/");

// auto can query data
// In this case, the result is plain-text contents of the header
const headerText = await auto("get the header text", { page, test });

// auto can perform actions
// In this case, auto will find and fill in the search text input
await auto(Type "${headerText}" in the search box, { page, test });

// auto can assert the state of the website
// In this case, the result is a boolean outcome
const searchInputHasHeaderText = await auto(
Is the contents of the search box equal to "${headerText}"?,
{ page, test },
);

expect(searchInputHasHeaderText).toBe(true);
});
`

$3

Include the StepOptions type with the values needed for connecting to Azure OpenAI.

`ts
import { test, expect } from "@playwright/test";
import { auto } from "auto-playwright";
import { StepOptions } from "../src/types";

const apiKey = "apikey";
const resource = "azure-resource-name";
const model = "model-deployment-name";

const options: StepOptions = {
model: model,
openaiApiKey: apiKey,
openaiBaseUrl: https://${resource}.openai.azure.com/openai/deployments/${model},
openaiDefaultQuery: { "api-version": "2023-07-01-preview" },
openaiDefaultHeaders: { "api-key": apiKey },
};

test("auto Playwright example", async ({ page }) => {
await page.goto("/");

// auto can query data
// In this case, the result is plain-text contents of the header
const headerText = await auto("get the header text", { page, test }, options);

// auto can perform actions
// In this case, auto will find and fill in the search text input
await auto(Type "${headerText}" in the search box, { page, test }, options);

// auto can assert the state of the website
// In this case, the result is a boolean outcome
const searchInputHasHeaderText = await auto(
Is the contents of the search box equal to "${headerText}"?,
{ page, test },
options,
);

expect(searchInputHasHeaderText).toBe(true);
});
`

Usage

At minimum, the auto function requires a _plain text prompt_ and an _argument_ that contains your page and test (optional) objects.

`ts
auto("", { page, test });
`

$3

Running without the test parameter:

`ts
import { chromium } from "playwright";
import { auto } from "auto-playwright";

(async () => {
const browser = await chromium.launch({ headless: true });
const context = await browser.newContext();
const page = await context.newPage();
// Navigate to a website
await page.goto("https://www.example.com");

// auto can query data
// In this case, the result is plain-text contents of the header
const res = await auto("get the header text", { page });

// use res.query to get a query result.
console.log(res);
await page.close();
})();
`

$3

You may pass a debug attribute as the third parameter to the auto function. This will print the prompt and the commands executed by OpenAI.

`ts
await auto("get the header text", { page, test }, { debug: true });
`

You may also set environment variable AUTO_PLAYWRIGHT_DEBUG=true, which will enable debugging for all auto calls.

`bash
export AUTO_PLAYWRIGHT_DEBUG=true
`

Supported Browsers

Every browser that Playwright supports.

Additional Options

There are additional options you can pass as a third argument:

`ts
const options = {
// If true, debugging information is printed in the console.
debug: boolean,
// The OpenAI model (https://platform.openai.com/docs/models/overview)
model: "gpt-4-1106-preview",
// The OpenAI API key
openaiApiKey: "sk-...",
};

auto("", { page, test }, options);
`

Supported Actions & Return Values

Depending on the type of action (inferred by the auto function), there are different behaviors and return types.

$3

An action (e.g. "click") is some simulated user interaction with the page, e.g. a click on a link. Actions will return undefined` if they were successful and will throw an error if they failed, e.g.

`ts
try {
await auto("click the link", { page, test });
} catch (e) {
console.error("failed to click the link");
}
`

$3

A query will return requested data from the page as a string, e.g.

`ts
const linkText = await auto("Get the text of the first link", { page, test });

console.log("The link text is", linkText);
`

$3

An assertion is a question that will return true or false, e.g.

`ts
const thereAreThreeLinks = await auto("Are there 3 links on the page?", {
page,
test,
});

console.log("There are 3 links" is a ${thereAreThreeLinks} statement);
`

Why use Auto Playwright?

| Aspect | Conventional Approach | Testing with Auto Playwright |
| ------------------------------ | ----------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| Coupling with Markup | Strongly linked to the application's markup. | Eliminates the use of selectors; actions are determined by the AI assistant at runtime. |
| Speed of Implementation | Slower implementation due to the need for precise code translation for each action. | Rapid test creation using simple, plain text instructions for actions and assertions. |
| Handling Complex Scenarios | Automating complex scenarios is challenging and prone to frequent failures. | Facilitates testing of complex scenarios by focusing on the intended test outcomes. |
| Test Writing Timing | Can only write tests after the complete development of the functionality. | Enables a Test-Driven Development (TDD) approach, allowing test writing concurrent with or before functionality development. |

Supported Playwright Actions

- locator.blur
- locator.boundingBox
- locator.check
- locator.clear
- locator.click
- locator.count
- locator.fill
- locator.getAttribute
- locator.innerHTML
- locator.innerText
- locator.inputValue
- locator.isChecked
- locator.isEditable
- locator.isEnabled
- locator.isVisible
- locator.pressKey
- locator.selectOption
- locator.textContent
- locator.uncheck
- page.goto
- page.keyboard.press

Adding new actions is easy: just update the functions in src/completeTask.ts.

Pricing

This library is free. However, there are costs associated with using OpenAI. You can find more information about pricing here: https://openai.com/pricing/.