-
Notifications
You must be signed in to change notification settings - Fork 72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Wait for network idle on page.goto #50
Comments
The following works when using await Promise.all([
page.waitForLoadState(`networkidle`),
page.waitForLoadState(`domcontentloaded`),
]) However, with
For now, we overcome this with |
We have the same problem here. Especially with image loading it can be very tricky to get stable snapshot tests. |
Same. I'm only using |
Hey there! We recently introduced a way to completely customize the way the test-runner prepares for running the tests. Can you check the documentation and examples here and see if it suits your needs? Sharing your findings would be great too! Thank you! |
Same here. The following helps a little, but not in a consistent manner: // Make sure assets (images, fonts) are loaded and ready
await page.waitForLoadState('domcontentloaded')
await page.waitForLoadState('networkidle');
await page.waitForFunction(() => !!document.fonts.ready); |
Would label this as a bug by the way rather than a feature request. |
I believe I've found a workaround that works consistently for us: await page.waitForLoadState('domcontentloaded')
await page.waitForLoadState('load')
await page.waitForLoadState('networkidle');
await page.waitForFunction(() =>
document.readyState === 'complete'
&& document.fonts.ready
// naturalWidth will be zero if image file is not yet loaded.
&& ![...document.images].some(({ naturalWidth }) => !naturalWidth)
); Let me know if it helps anyone. |
@joriswitteman it probably wouldn't work in a case when an
|
I have not. Our components probably don't cover enough vertical real estate for that to occur. But you could amend the code to take lazy loading into account like so: await page.waitForLoadState('domcontentloaded')
await page.waitForLoadState('load')
await page.waitForLoadState('networkidle');
await page.waitForFunction(() =>
document.readyState === 'complete'
&& document.fonts.ready
// naturalWidth will be zero if image file is not yet loaded.
&& ![...document.images].some(
({ naturalWidth, loading }) => !naturalWidth && loading !== 'lazy'
)
); |
would it be possible to export |
Hey everyone! I dug into this issue and it seems like the best solution is to deal with it yourself in the Let me try to explain: The test-runner works in the following order:
So if you have flakiness happening in your stories, you actually want to act on step 5. If you were to add all these waiting utilities to the // .storybook/test-runner.ts
import { waitForPageReady } from '@storybook/test-runner'
import { toMatchImageSnapshot } from 'jest-image-snapshot'
const customSnapshotsDir = `${process.cwd()}/__snapshots__`
export const setup = () => {
expect.extend({ toMatchImageSnapshot })
}
export const postRender = async (page, context) => {
await page.waitForLoadState('domcontentloaded');
await page.waitForLoadState('load');
await page.waitForLoadState('networkidle');
await page.evaluate(() => document.fonts.ready);
// + whatever extra loading you'd like
const image = await page.screenshot()
expect(image).toMatchImageSnapshot({
customSnapshotsDir,
customSnapshotIdentifier: context.id,
})
} I thought of adding that built-in to the test-runner, however this adds a few extra milliseconds of wait per test, which will affect every user that does not do image snapshot testing. We could potentially export a utility from the test-runner itself that essentially executes those lines, something like export const waitForPageReady = async (page: Page) => {
await page.waitForLoadState('domcontentloaded');
await page.waitForLoadState('load');
await page.waitForLoadState('networkidle');
await page.evaluate(() => document.fonts.ready);
} and then you can use that in the postRender function. I might experiment with that, release a canary and you can test it. How does that sound? Once I have feedback that it actually has helped, then I can merge it into a stable release! Edit: It's done! You can install the canary: yarn add --save-dev @storybook/[email protected] and use it like so: // .storybook/test-runner.ts
import { waitForPageReady } from '@storybook/test-runner'
import { toMatchImageSnapshot } from 'jest-image-snapshot'
const customSnapshotsDir = `${process.cwd()}/__snapshots__`
export const setup = () => {
expect.extend({ toMatchImageSnapshot })
}
export const postRender = async (page, context) => {
// make sure everything is properly loaded before taking an image snapshot
await waitForPageReady(page)
const image = await page.screenshot()
expect(image).toMatchImageSnapshot({
customSnapshotsDir,
customSnapshotIdentifier: context.id,
})
} |
I just tested the When I comment out the line Any idea why this might be? Is there something else I'm missing? |
Thanks a lot for trying it out @skaindl! You mention about a simple button component. Can you share a reproduction I can take a look at?
I'm not sure, I would love to take a look at a reproduction to understand it further. I tested this in a medium sized project with about 80 tests, from simple buttons to complex full page stories that fetch data, and even a story for the entire app with some internal navigation, and it all worked correctly. |
@skaindl Have you got Hot Module Reload running? If I remember correctly, I've worked on SB projects before where that was an issue with network idle. |
One element that still seems to struggle; HTML elements with a background image don't seem to be caught by networkidle or by checking for images on the page (obvs). Anyone know of a clean way we can detect those load events? EDIT: Also having issues with google fonts - seems like tiny changes in font rendering between test runs is causing issues that can amount to more than a 1% threshold. |
|
Adding #444 |
Works for me, thank you so much! Too bad this isn't included in |
We use MSW for image requests: export const XSSnapshot: Story = {
parameters: {
cookie: {},
viewport: {
defaultViewport: 'xs',
},
msw: {
handlers: [
rest.get('**/*.svg', async (req, res, ctx) => {
const buffer = await fetch(
`/fixtures/mock-image-1.png`
).then((response) => response.arrayBuffer())
return res(
ctx.set('Content-Length', buffer.byteLength.toString()),
ctx.set('Content-Type', 'image/png'),
ctx.body(buffer)
)
}),
],
},
},
render,
play: async (context) => {
await xsRendered.play?.(context)
},
} Snapshot tests are still flaky - images do/don't load - with this setup: async postVisit(page, story) {
// Run snapshot tests for stories with `snapshot` in the name.
if (story.id.includes('snapshot')) {
// Awaits for the page to be loaded and available including assets (e.g., fonts)
await waitForPageReady(page)
// Generates a snapshot file based on the story identifier
const image = await page.screenshot({
animations: 'disabled',
fullPage: true,
})
expect(image).toMatchImageSnapshot({
customSnapshotsDir,
customSnapshotIdentifier: story.id,
failureThreshold: 0.01, // 1% threshold for entire image.
failureThresholdType: 'percent', // Percent of image or number of pixels.
})
}
}, |
example: in CE project, some logos took time to load which resulted in flaky snapshot tests
The text was updated successfully, but these errors were encountered: