Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(testType): add support for test.fail.only method #33001

Open
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

pengooseDev
Copy link

Description

Following up on issue #30662

This PR adds support for test.fail.only in Playwright Test, allowing developers to focus on a specific test that is expected to fail.

  • Implemented the test.fail.only method.
  • Added regression tests for test.fail.only to ensure correct behavior.
  • Updated documentation and TypeScript definitions (type.d.ts) to include test.fail.only.

As this is my first contribution, I would appreciate feedback on the regression tests and documentation updates. If there are any changes needed, I'm happy to make them. :)

@pengooseDev
Copy link
Author

@microsoft-github-policy-service agree

This comment has been minimized.

This comment has been minimized.

Copy link
Member

@pavelfeldman pavelfeldman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your PR needs rebaseline

@@ -1096,6 +1096,27 @@ test('less readable', async ({ page }) => {
});
```

You can use `test.fail.only` to focus on a specific test that is expected to fail. This is particularly useful when debugging a failing test or working on a specific issue.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this section - I would omit this as implied.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback. Removed! :)

@@ -26,6 +26,25 @@ Now, if you run these shards in parallel on different jobs, your test suite comp

Note that Playwright can only shard tests that can be run in parallel. By default, this means Playwright will shard test files. Learn about other options in the [parallelism guide](./test-parallel.md).


## Balancing Shards
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like you need a rebaseline.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase applied, thanks for the feedback. :)

@pengooseDev pengooseDev force-pushed the feature-fail-only branch 2 times, most recently from a13163d to 88866a0 Compare October 9, 2024 09:45

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

github-actions bot commented Oct 9, 2024

Test results for "tests 1"

10 failed
❌ [playwright-test] › basic.spec.ts:591:5 › should not allow mixing test types @macos-latest-node18-1
❌ [playwright-test] › types-2.spec.ts:19:5 › basics should work @macos-latest-node18-1
❌ [playwright-test] › basic.spec.ts:591:5 › should not allow mixing test types @ubuntu-latest-node18-1
❌ [playwright-test] › types-2.spec.ts:19:5 › basics should work @ubuntu-latest-node18-1
❌ [playwright-test] › basic.spec.ts:591:5 › should not allow mixing test types @ubuntu-latest-node20-1
❌ [playwright-test] › types-2.spec.ts:19:5 › basics should work @ubuntu-latest-node20-1
❌ [playwright-test] › basic.spec.ts:591:5 › should not allow mixing test types @ubuntu-latest-node22-1
❌ [playwright-test] › types-2.spec.ts:19:5 › basics should work @ubuntu-latest-node22-1
❌ [playwright-test] › basic.spec.ts:591:5 › should not allow mixing test types @windows-latest-node18-1
❌ [playwright-test] › types-2.spec.ts:19:5 › basics should work @windows-latest-node18-1

2 flaky ⚠️ [chromium-library] › library/chromium/oopif.spec.ts:283:3 › should click @chromium-ubuntu-22.04-node20
⚠️ [playwright-test] › ui-mode-test-setup.spec.ts:98:5 › should show errors in config @macos-latest-node18-1

35900 passed, 620 skipped
✔️✔️✔️

Merge workflow run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants