forgejo/tests/e2e
Yaroslav Halchenko 5ae2dbcb14
Some checks are pending
Integration tests for the release process / release-simulation (push) Waiting to run
/ release (push) Waiting to run
testing / backend-checks (push) Waiting to run
testing / frontend-checks (push) Waiting to run
testing / test-unit (push) Blocked by required conditions
testing / test-remote-cacher (map[image:docker.io/valkey/valkey:7.2.5-alpine3.19 port:6379]) (push) Blocked by required conditions
testing / test-remote-cacher (map[image:ghcr.io/microsoft/garnet-alpine:1.0.14 port:6379]) (push) Blocked by required conditions
testing / test-remote-cacher (map[image:redis:7.2 port:6379]) (push) Blocked by required conditions
testing / test-remote-cacher (map[image:registry.redict.io/redict:7.3.0-scratch port:6379]) (push) Blocked by required conditions
testing / test-mysql (push) Blocked by required conditions
testing / test-pgsql (push) Blocked by required conditions
testing / test-sqlite (push) Blocked by required conditions
testing / security-check (push) Blocked by required conditions
Adjust codespell config + make it fix few typos which sneaked in since addition of codespell support (#4857)
Now that my colleague just posted a wonderful blog post https://blog.datalad.org/posts/forgejo-runner-podman-deployment/ on forgejo runner, some time I will try to add that damn codespell action to work on CI here ;)  meanwhile some typos managed to sneak in and this PR should address them (one change might be functional in a test -- not sure if would cause a fail or not)

### Release notes

- [ ] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4857
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-committed-by: Yaroslav Halchenko <debian@onerussian.com>
2024-08-08 16:07:35 +00:00
..
.eslintrc.yaml [CHORE] Add playwright eslint plugin 2024-07-22 20:03:32 +02:00
actions.test.e2e.js [CHORE] Add playwright eslint plugin 2024-07-22 20:03:32 +02:00
commit-graph-branch-selector.test.e2e.js [CHORE] Add playwright eslint plugin 2024-07-22 20:03:32 +02:00
dashboard-ci-status.test.e2e.js Fixed link to commit status and tooltip on dashboard 2024-07-14 13:03:03 +02:00
debugserver_test.go [CHORE] Remove println 2024-04-12 13:36:31 +02:00
e2e_test.go tests(e2e): only run one test, not all of them each time 2024-07-14 14:19:05 +02:00
edit-comment.test.e2e.js show edit textfield always initially when editing comments 2024-06-14 15:42:21 +02:00
example.test.e2e.js [CHORE] Add playwright eslint plugin 2024-07-22 20:03:32 +02:00
explore.test.e2e.js [CHORE] Add playwright eslint plugin 2024-07-22 20:03:32 +02:00
issue-sidebar.test.e2e.js [CHORE] Add playwright eslint plugin 2024-07-22 20:03:32 +02:00
markdown-editor.test.e2e.js [CHORE] Add playwright eslint plugin 2024-07-22 20:03:32 +02:00
markup.test.e2e.js [CHORE] Add playwright eslint plugin 2024-07-22 20:03:32 +02:00
profile_actions.test.e2e.js Adjust codespell config + make it fix few typos which sneaked in since addition of codespell support (#4857) 2024-08-08 16:07:35 +00:00
README.md docs(e2e): explain how to use VSCodium or VSCode 2024-07-16 17:22:39 +02:00
release.test.e2e.js Implement external assets 2024-07-29 20:35:55 +02:00
repo-code.test.e2e.js fix(ui): handle out-of-bounds end line in code selection (#4788) 2024-08-05 04:45:07 +00:00
right-settings-button.test.e2e.js Fixes that the settings button moves in the overflow menu 2024-04-28 23:31:31 +02:00
utils_e2e.js [Port] gitea#29982 Introduce .secondary-nav and handle .page-content spacing universally 2024-03-28 16:43:09 +01:00
utils_e2e_test.go Add testifylint to lint checks (#4535) 2024-07-30 19:41:10 +00:00

End to end tests

E2e tests largely follow the same syntax as integration tests. Whereas integration tests are intended to mock and stress the back-end, server-side code, e2e tests the interface between front-end and back-end, as well as visual regressions with both assertions and visual comparisons. They can be run with make commands for the appropriate backends, namely:

make test-sqlite
make test-pgsql
make test-mysql

Make sure to perform a clean front-end build before running tests:

make clean frontend

Install playwright system dependencies

npx playwright install-deps

Interactive testing

You can make use of Playwright's integrated UI mode to run individual tests, get feedback and visually trace what your browser is doing.

To do so, launch the debugserver using:

make test-e2e-debugserver

Then launch the Playwright UI:

npx playwright test --ui

You can also run individual tests while the debugserver using:

npx playwright test actions.test.e2e.js:9

First, specify the complete test filename, and after the colon you can put the linenumber where the test is defined.

Run all tests via local act_runner

act_runner exec -W ./.github/workflows/pull-e2e-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest

Run sqlite e2e tests

Start tests

make test-e2e-sqlite

Run MySQL e2e tests

Setup a MySQL database inside docker

docker run -e "MYSQL_DATABASE=test" -e "MYSQL_ALLOW_EMPTY_PASSWORD=yes" -p 3306:3306 --rm --name mysql mysql:latest #(just ctrl-c to stop db and clean the container)
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --rm --name elasticsearch elasticsearch:7.6.0 #(in a second terminal, just ctrl-c to stop db and clean the container)

Start tests based on the database container

TEST_MYSQL_HOST=localhost:3306 TEST_MYSQL_DBNAME=test TEST_MYSQL_USERNAME=root TEST_MYSQL_PASSWORD='' make test-e2e-mysql

Run pgsql e2e tests

Setup a pgsql database inside docker

docker run -e "POSTGRES_DB=test" -p 5432:5432 --rm --name pgsql postgres:latest #(just ctrl-c to stop db and clean the container)

Start tests based on the database container

TEST_PGSQL_HOST=localhost:5432 TEST_PGSQL_DBNAME=test TEST_PGSQL_USERNAME=postgres TEST_PGSQL_PASSWORD=postgres make test-e2e-pgsql

Running individual tests

Example command to run example.test.e2e.js test file:

Note: unlike integration tests, this filtering is at the file level, not function

For SQLite:

make test-e2e-sqlite#example

For PostgreSQL databases(replace mysql to pgsql):

TEST_MYSQL_HOST=localhost:1433 TEST_MYSQL_DBNAME=test TEST_MYSQL_USERNAME=sa TEST_MYSQL_PASSWORD=MwantsaSecurePassword1 make test-e2e-mysql#example

Visual testing

Although the main goal of e2e is assertion testing, we have added a framework for visual regress testing. If you are working on front-end features, please use the following:

  • Check out main, make clean frontend, and run e2e tests with VISUAL_TEST=1 to generate outputs. This will initially fail, as no screenshots exist. You can run the e2e tests again to assert it passes.
  • Check out your branch, make clean frontend, and run e2e tests with VISUAL_TEST=1. You should be able to assert you front-end changes don't break any other tests unintentionally.

VISUAL_TEST=1 will create screenshots in tests/e2e/test-snapshots. The test will fail the first time this is enabled (until we get visual test image persistence figured out), because it will be testing against an empty screenshot folder.

ACCEPT_VISUAL=1 will overwrite the snapshot images with new images.

With VSCodium or VSCode

To debug a test, you can use "Playwright Test" for VScodium or VSCode. Before doing that you will need to manually start a Forgejo instance and populate it with data from models/fixtures by running:

make TAGS='sqlite sqlite_unlock_notify' 'test-e2e-debugserver'