Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(watcher): add support for poll watcher #21290

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

amribm
Copy link

@amribm amribm commented Sep 13, 2024

Closes #21111

@amribm amribm requested a review from a team as a code owner September 13, 2024 04:59
@bits-bot
Copy link

bits-bot commented Sep 13, 2024

CLA assistant check
All committers have signed the CLA.

@amribm amribm force-pushed the feature/support_for_poll_watcher branch from 3773b14 to ae66312 Compare September 13, 2024 05:01
@amribm amribm force-pushed the feature/support_for_poll_watcher branch from 4c926d0 to 1802ece Compare September 13, 2024 05:17
@amribm amribm changed the title feat(watcher): add support for poll watcher support feat(watcher): add support for poll watcher Sep 13, 2024
src/app.rs Outdated Show resolved Hide resolved
src/app.rs Outdated Show resolved Hide resolved
src/config/watcher.rs Outdated Show resolved Hide resolved
src/config/watcher.rs Outdated Show resolved Hide resolved
}
WatcherConfig::PollWatcher(interval) => {
let config =
notify::Config::default().with_poll_interval(Duration::from_secs(*interval));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we allow the user to specify compare_contents? https://docs.rs/notify/latest/notify/struct.Config.html#method.with_compare_contents
maybe another argument? Im afraid we are adding too much arguments for this though

@jszwedko

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd vote against it being user controllable and hope for it to just have a sane default. I can't think of a scenario where I'd wanna toggle it!

Copy link
Contributor

@jorgehermo9 jorgehermo9 Sep 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We found issues in openshift environments where the inotify watcher was constantly triggered and the file contents did not change at all. In this case, we would like to hash the contents to check if it really changed before reloading vector config. I think having sane defaults is good (for example, this would default to false), but there are situations where this is useful

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would there be a downside to always hashing the changes?

Copy link
Contributor

@jorgehermo9 jorgehermo9 Sep 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, as the docs I pointed states:

By enabling this feature, performance will be significantly impacted as all files will need to be read and hashed at each poll_interval.

I prefer that downside to be opt-in

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it is a reasonable opt-in option that we can include in this PR. To be clear, I don't feel strongly about adding it here and now.

src/config/watcher.rs Show resolved Hide resolved
src/config/watcher.rs Show resolved Hide resolved
src/config/watcher.rs Outdated Show resolved Hide resolved
src/config/watcher.rs Outdated Show resolved Hide resolved
src/config/watcher.rs Outdated Show resolved Hide resolved
src/config/watcher.rs Outdated Show resolved Hide resolved
src/config/watcher.rs Outdated Show resolved Hide resolved
@pront pront self-requested a review September 25, 2024 22:24
@pront
Copy link
Contributor

pront commented Sep 26, 2024

Hey @amribm, thank you for this PR!

@amribm
Copy link
Author

amribm commented Sep 26, 2024

okay, i will add changelog

  • I see there are a couple of open comment threads, is this ready for a review?

Yes This is ready for review , @pront i need some confirmation from vector team for those comments. that's why i didn't resolved them.

src/cli.rs Outdated Show resolved Hide resolved
src/cli.rs Outdated Show resolved Hide resolved
src/cli.rs Show resolved Hide resolved
src/config/watcher.rs Outdated Show resolved Hide resolved
}
WatcherConfig::PollWatcher(interval) => {
let config =
notify::Config::default().with_poll_interval(Duration::from_secs(*interval));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it is a reasonable opt-in option that we can include in this PR. To be clear, I don't feel strongly about adding it here and now.

amribm and others added 3 commits September 28, 2024 10:59
Co-authored-by: Pavlos Rontidis <[email protected]>
Co-authored-by: Pavlos Rontidis <[email protected]>
Co-authored-by: Pavlos Rontidis <[email protected]>
src/config/watcher.rs Outdated Show resolved Hide resolved
@amribm amribm requested review from a team as code owners October 1, 2024 15:34
@github-actions github-actions bot added the domain: external docs Anything related to Vector's external, public documentation label Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: external docs Anything related to Vector's external, public documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Polling based extension of the —watch-config flag that doesn’t rely on inotify/SIGHUP
6 participants