Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

speed up skipped build processing #239

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

speed up skipped build processing #239

wants to merge 2 commits into from

Conversation

Mic92
Copy link
Member

@Mic92 Mic92 commented Jul 26, 2024

No description provided.

@MagicRB
Copy link
Contributor

MagicRB commented Jul 26, 2024

Shouldnt workers be able to attach to multiple builds?

@Mic92
Copy link
Member Author

Mic92 commented Jul 26, 2024

I don't think so. Before I added one worker per CPU, it would only process one build at the time.

@MagicRB
Copy link
Contributor

MagicRB commented Jul 26, 2024

Then what's that?
Concurrencu must be blocked by locks.
Right do we have a lock in the skipped build builder?

@Mic92
Copy link
Member Author

Mic92 commented Jul 26, 2024

I checked now a couple of times and with this change the time reduces from 20s to 15s for "skipping" disko

@zowoq
Copy link
Contributor

zowoq commented Aug 4, 2024

I tried this branch on a fully cached nixvim eval a few days ago. There wasn't any other buildbot/hydra jobs running at the time.

With this branch: 1:06 https://buildbot.nix-community.org/#/builders/26/builds/20

Without: 28:23 https://buildbot.nix-community.org/#/builders/26/builds/21

I ran another one without today: 38:14 https://buildbot.nix-community.org/#/builders/26/builds/95

@Mic92
Copy link
Member Author

Mic92 commented Aug 12, 2024

That's quite significant. Before I had skipped builds not actually having to run any code i.e. build steps. I think they were faster back than. I wish there was a better way to insert a build without any involvement of the scheduler.

@MagicRB
Copy link
Contributor

MagicRB commented Aug 12, 2024

While this works, I don't like that we don't know why the worker won't accept more than one build at a time. I'll try to debug that.

@MagicRB
Copy link
Contributor

MagicRB commented Aug 12, 2024

buildbot/buildbot#7125 okay this is this, it was not obvious from the code, but yeah, its one builder one worker so it won't run multiple. We should do the same for the NixEval builder then

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants