Contributing Playwright Tests to Sefaria: A Volunteer Guide

Welcome! We're grateful that you have decided to contribute to Sefaria's test coverage. We are so appreciative of our enthusiastic community of developers.

Playwright tests bulletproof Sefaria from bugs. They are easy to contribute to and make our digital library more resilient. To take a look at the list of tests we have in the works, check out the Sefaria Playwright Tests repository on Github.

In this tutorial, you will learn how to set up your local environment to write Playwright-based end-to-end (E2E) tests for Sefaria, contribute new tests, and submit your changes through a pull request.

Prerequisites
Claim a Feature from the Github Issue Tracker
Set Up the Environment
Fork and Clone the Repository
Set Up Playwright
Write Your Tests
Example: Choosing and Creating a New Test
Run and Debug Locally
Submit Your Contribution
Tips for Writing Great Playwright Tests

Prerequisites

Before you begin, make sure you have the following installed:

Git

Node.js (v18+ recommended)

Python (3.10+)- only if you want to run the full Sefaria app locally

Docker -for running local services like Elasticsearch, Redis, MongoDB

GitHub account

Note: While Playwright tests are written in TypeScript, Sefaria’s backend is powered by Django (Python). If you want to run the app locally to simulate full app behavior in your tests, you will need Python. If you are testing frontend behavior against a live Cauldron or staging server, only Node.js is required.

Claim a Feature from the Github Issue Tracker

Sefaria uses Github Issues to track which features or areas still need Playwright test coverage. Before writing a test, browse Sefaria's Playwright GitHub Issues list to see what features still need testing.

How to Claim an Issue

Find an open issue you'd like to contribute to. Make sure it does not yet have an assignee or "In Progress" status. Features are labeled with different complexity levels - select one that matches your comfort level.
Comment on the issue saying something like:
"I would like to volunteer to create these tests"
A maintainer will assign the issue to you shortly.

⚠️
Note: GitHub only allows maintainers or collaborators with write access to assign issues.
If you're not yet a collaborator, you won’t be able to assign yourself. Instead, add a comment on the issue that you would like to write a test for.

Once you're assigned, you can:

Create a new branch in your local repo.
Write your test(s).
Open a pull request referencing the issue (e.g., Fixes #23 in your PR description).

Set Up the Environment

Follow the Sefaria setup instructions to get the backend and frontend running locally.

If you run into issues, check the README.md or fill out this form to contact Sefaria's engineering team.

Fork and Clone the Sefaria Repository

Fork the Repository

Go to https://github.com/Sefaria/Sefaria-Project
Click "Fork".

Forking creates your personal copy of the Sefaria repo on GitHub so you can make changes without affecting the main project directly.

Clone Your Fork Locally


git clone https://github.com/YOUR_USERNAME/Sefaria-Project.git

cd Sefaria-Project

This downloads your fork to your computer so you can work on it using your local development tools.

Take some time to familiarize yourself with the repository, specifically the e2e-tests folder and its contents. Review the page objects, existing test files, and helper files to understand the structure.

Add the Main Repo as a Remote


git remote add upstream https://github.com/Sefaria/Sefaria-Project.git

This links your local copy to the original (main) Sefaria repo, allowing you to fetch updates or feature branches from it.

Fetch the Relevant Branch

If the feature you're writing tests for is still in development, pull it from the developer's branch.


git fetch upstream

git checkout -b test-feature-name upstream/dev-feature-branch

Replace test-feature-name with something descriptive of the test you're adding, and dev-feature-branch with the actual name of the developer’s branch.

If the feature is already complete and merged into master, (which will likely be the case):


git fetch upstream

git checkout -b test-feature-name upstream/master

This creates your test branch based on the latest code in master.

Set Up Playwright

Install Playwright and its test runner:


npm install playwright@latest

npm install @playwright/test

npx playwright install

Verify your setup by running existing tests:


npx playwright test

Write Your Tests

Sefaria's Playwright tests live under the e2e-tests/tests directory.

Each test file should:

Be scoped to a single feature or user interaction
Use a consistent naming pattern (e.g., newTestFeature.spec.ts)
Leverage the Page Object Model already implemented in e2e-tests/pages

Basic Test Structure

The basic form of a Sefaria Playwright test is:


test('Description of what is being tested', async ({ context }) => {

  // Setup, actions, and assertions

});

Key Features:

Name: Give each test a clear name describing the user action and expected behavior
async ({ context }): All Playwright tests are asynchronous because they interact with a browser. context simulates a private browsing session
Actions: Simulate real user actions (typing, clicking, etc.), encapsulated in Page Object methods
Assertions: Use expect() to verify the app responded correctly

Finding the Right Locators

When writing tests, use Playwright's best practices for locators:

Prefer role-based locators that are accessible and user-friendly:


// Good - uses role and accessible name

await page.getByRole('button', { name: 'Submit' }).click();

await page.getByRole('textbox', { name: 'Search' }).fill('Genesis');

Use test IDs when role-based locators aren't sufficient:
```
await page.getByTestId('language-selector').click();
```

Avoid CSS selectors and XPath when possible:


// Avoid this approach

await page.locator('.complex-css-selector').click();

Finding Locators:

Inspect the HTML: Use browser developer tools to examine the page structure
Use Playwright Codegen: Run npx playwright codegen to generate locators automatically
Check accessibility: Ensure your locators work with screen readers

Common Imports

Your test files will typically need these imports:


import { test, expect } from '@playwright/test';

import { goToPageWithLang } from '../utils';

import { LANGUAGES } from '../globals';

import { PageManager } from '../pages/pageManager';

Add additional imports as needed based on your specific test requirements.

You may find that you want to add helper methods or certain variables to make your tests more concise and efficient. You are welcome and encouraged to do so! If you need assistance choosing the correct format or location for these additions, please reach out to Sefaria's engineering team.

You're all set! Now that we've covered the basics, let us take a deep dive into the process of choosing, developing, and pushing your Playwright test.

Example: Choosing and Creating a New Test

Let's walk through creating a test for "Table of Contents (ToC) Language Control" (Issue #16).

After browsing the list of available features to test on Sefaria's Github Issues, we select "Table of Contents (ToC) Language Control", Issue #16.

We click on the issue, and add a comment: I would like to volunteer to create these tests.

The issue provides:

Title: Verify language display in ToC based on content language.
Test:

Set contentLanguage to translation.
Open the Table of Contents for a Hebrew translation text.
Verify that ToC items are displayed in English.
Set contentLanguage to source.
Open the Table of Contents for an English source text.
Verify that ToC items are displayed in Hebrew.

After a Sefaria team member grants us assignee status, we are ready to begin!

Create the Test File

Now that we have selected and been assigned a feature to test, we can go ahead and get started on creating the test file. This can be done through the terminal, or manually.

The standard naming convention for a Playwright test file consists of the feature you are testing, followed by .spec.ts. Since we are testing language control for the Table of Contents, we will name our test file toc-language-control.spec.ts.

Using the terminal:


touch e2e-tests/tests/toc-language-control.spec.ts

Alternatively, to create the file manually in VS Code:

Open the Explorer sidebar (Ctrl+Shift+E or Cmd+Shift+E).
Right-click the tests folder.
Click "New File".
Type toc-language-control.spec.ts and press Enter.

Add Imports

Your list of imports will include common Playwright commands, the Sefaria Page Object Models relevant to the feature you are testing, and valuable functions or variables from helper files such as utils.ts or constants.ts.

If the Page Object Model does not yet exist for the page you are testing, you will need to create one. For some tests, like this one, we will use the PageManager available in the pages folder, which holds all the different pages as variables. Here are our imports:


import { test, expect } from '@playwright/test';

import { goToPageWithLang } from '../utils';

import { LANGUAGES } from '../globals';

import { PageManager } from '../pages/pageManager';

If you see that you need to add more import statements as you continue to write your tests, please do so.

Write the Tests

Let's take a look at the two tests we developed to verify language display for the Table of Contents of a given text.


test.describe('Content Language affects Table of Contents display correctly', () => {

  test('ToC displays English when contentLanguage is "Translation"', async ({ context }) => {
    const page = await goToPageWithLang(context, '/Genesis.1', LANGUAGES.EN);
    const pm = new PageManager(page, LANGUAGES.EN);
    const sourceTextPage = pm.onSourceTextPage();
    await sourceTextPage.setContentLanguage('Translation');
    // Click on translated text to open sidebar - using accessible locator
    await page.getByRole('region', { name: 'text content' })
							.getByText('In the beginning').first().click();
    await sourceTextPage.openTableOfContents();
    // Verify English display using accessible locators
    await expect(page.getByRole('heading', { name: 'Chapters' })).toBeVisible();
    await expect(page.getByRole('link', { name: '1' }).first()).toBeVisible();
  });

  test('ToC displays Hebrew when contentLanguage is set to "Source"', async ({ context }) => {
    const page = await goToPageWithLang(context, '/Genesis.1', LANGUAGES.EN);
    const pm = new PageManager(page, LANGUAGES.EN);
    const sourceTextPage = pm.onSourceTextPage();
    await sourceTextPage.setContentLanguage('Source');
    // Click on Hebrew text to open sidebar
    await page.getByRole('region', { name: 'text content'})
							.locator('[lang="he"]').first().click();
		await sourceTextPage.openTableOfContents();
    // Verify Hebrew display
    await expect(page.getByRole('heading', { name: 'פרקים' })).toBeVisible();
    await expect(page.getByRole('link', { name: 'א' }).first()).toBeVisible();
  });

});

The first test:

Goes to the first chapter of Genesis (/Genesis.1) with the interface language set to English.
Tells the page to display the translation (i.e., the English version of the Hebrew text).
Clicks on a piece of the translated text to open the sidebar.
Opens the Table of Contents (ToC) for the text.
Checks that the ToC contains the word “Chapters” (in English), and the section number is shown as 1 (not in Hebrew letters).

The second test checks the opposite case, i.e. the ToC contains the word "פרקים" and uses Hebrew letters to display the chapters rather than numbers.

Understanding the Test Structure

test.describe groups related tests under a common label. It helps with:

Organization and readability
Shared setup with test.beforeEach/test.afterEach
Better test reporting

test.beforeEach and test.afterEach can be used for shared setup:


test.beforeEach(async ({ context }) => {

  // Setup that runs before each test

});

Use these when:

Multiple tests share common setup
You want to ensure consistency across tests
You're repeating setup code

Run and Debug Locally

Running Tests

Run your specific test file:


npx playwright test toc-language-control.spec.ts

Run with the Playwright UI (recommended):


npx playwright test --ui toc-language-control.spec.ts

Debug interactively:


npx playwright test toc-language-control.spec.ts --debug

Using the Playwright UI

The UI provides excellent debugging capabilities:

Run individual tests or groups
See page state before, during, and after actions
Hover over the timeline to see page changes
View errors in the dedicated panel
Step through test execution

Playwright UI Example

Common Debugging Tips

When tests fail:

Check the error message in the UI
Verify locators are correct using browser dev tools
Ensure elements are visible before interacting
Add debugging statements: await page.pause()

Locator issues:

Use npx playwright codegen to generate locators
Prefer stable locators over brittle CSS selectors

Submit Your Contribution

Create Branch and Commit

Create a new branch:


git checkout -b test/toc-language-control

Stage your changes:


git add e2e-tests/tests/toc-language-control.spec.ts

Commit with a descriptive message:


git commit -m "test(ToC): add language control tests [Issue #16]"

Note: Sefaria strives to use the "Conventional Commits" method when it comes to writing commit messages. Check out this Conventional Commits Cheat Sheet to learn more about it!

Push your branch:


git push origin test/toc-language-control

Open a Pull Request

Go to your forked repository on GitHub
Click "Compare & pull request"
Ensure the base repository is Sefaria/Sefaria-Project and base branch is master
Write a clear title and description
Include Fixes #issue_number or Closes #issue_number in the description. This will automatically link your pull request to the issue.

Review Process

A Sefaria team member will review your PR. This may take a few days - we appreciate your patience! Be responsive to feedback and make requested changes. Once approved and merged, your test becomes part of the automated test suite.

Tips for Writing Great Playwright Tests

Best Practices

Be specific: Focus on one behavior or flow per test


// Good - tests one specific behavior

test('User can search for a text', async ({ page }) => { ... });

// Avoid - tests multiple behaviors

test('User can search and navigate and bookmark', async ({ page }) => { ... });

Be clear: Use descriptive names and meaningful assertions


// Good - clear expectation

await expect(page.getByRole('heading', { name: 'Search Results' })).toBeVisible();

// Avoid - unclear expectation

await expect(page.locator('.results')).toHaveCount(1);

Be reliable: Use Playwright's built-in waiting mechanisms


// Good - waits for element to be visible

await expect(page.getByRole('button', { name: 'Submit' })).toBeVisible();

// Avoid - arbitrary waits

await page.waitForTimeout(3000);

Be DRY: Reuse Page Object methods to keep tests clean


// Good - uses page object method

await sourceTextPage.openTableOfContents();

// Avoid - duplicating complex interactions

await page.click('.menu-button');

await page.click('.toc-option');

Locator Strategy

Start with role-based locators - they're accessible and stable
Use test IDs when roles aren't sufficient
Examine the HTML to understand the page structure
Use codegen to generate initial locators, then improve them
Test your locators in browser dev tools

Common Pitfalls to Avoid

Flaky waits: Don't use waitForTimeout() unless absolutely necessary
Overly specific locators: Avoid CSS selectors that break easily
Missing assertions: Always verify the expected outcome
Too much in one test: Keep tests focused and atomic

Troubleshooting Common Issues

Test Failures

Element not found: Check if the locator is correct and the element exists


// Debug by checking element existence

console.log(await page.getByRole('button', { name: 'Submit' }).count());

Timing issues: Ensure proper waiting for dynamic content


// Wait for element to be ready

await expect(page.getByRole('button', { name: 'Submit' })).toBeEnabled();

Language/locale issues: Make sure you're testing in the correct language context


// Use the language utilities provided

const page = await goToPageWithLang(context, '/Genesis.1', LANGUAGES.EN);

Setup Issues

Node.js version problems: Ensure you're using Node.js 18.x - 20.x

Missing dependencies: Run npm install to ensure all packages are installed

Playwright browsers: Run npx playwright install to install browser binaries

Additional Resources

Getting Help

Feel free to reach out to Sefaria's engineering team with any questions! We're here to help you contribute successfully.

Thank you for helping build a better Sefaria! Every test you contribute makes our site more reliable and accessible to users around the world.

Writing Playwright Tests for Sefaria

Contributing Playwright Tests to Sefaria: A Volunteer Guide

Table of Contents

Prerequisites

Claim a Feature from the Github Issue Tracker

How to Claim an Issue

⚠️
Note: GitHub only allows maintainers or collaborators with write access to assign issues.

Set Up the Environment

Fork and Clone the Sefaria Repository

Fork the Repository

Clone Your Fork Locally

Add the Main Repo as a Remote

Fetch the Relevant Branch

Set Up Playwright

Write Your Tests

Basic Test Structure

Finding the Right Locators

Common Imports

Example: Choosing and Creating a New Test

Create the Test File

Add Imports

Write the Tests

Understanding the Test Structure

Run and Debug Locally

Running Tests

Using the Playwright UI

Common Debugging Tips

Submit Your Contribution

Create Branch and Commit

Open a Pull Request

Review Process

Tips for Writing Great Playwright Tests

Best Practices

Locator Strategy

Common Pitfalls to Avoid

Troubleshooting Common Issues

Test Failures

Setup Issues

Additional Resources

Getting Help

Contributing Playwright Tests to Sefaria: A Volunteer Guide

Table of Contents

Prerequisites

Claim a Feature from the Github Issue Tracker

How to Claim an Issue

⚠️Note: GitHub only allows maintainers or collaborators with write access to assign issues.

Set Up the Environment

Fork and Clone the Sefaria Repository

Fork the Repository

Clone Your Fork Locally

Add the Main Repo as a Remote

Fetch the Relevant Branch

Set Up Playwright

Write Your Tests

Basic Test Structure

Finding the Right Locators

Common Imports

Example: Choosing and Creating a New Test

Create the Test File

Add Imports

Write the Tests

Understanding the Test Structure

Run and Debug Locally

Running Tests

Using the Playwright UI

Common Debugging Tips

Submit Your Contribution

Create Branch and Commit

Open a Pull Request

Review Process

Tips for Writing Great Playwright Tests

Best Practices

Locator Strategy

Common Pitfalls to Avoid

Troubleshooting Common Issues

Test Failures

Setup Issues

Additional Resources

Getting Help

⚠️
Note: GitHub only allows maintainers or collaborators with write access to assign issues.