Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LuceneIndexMaintenanceTest sometimes take many minutes #2842

Open
ScottDugas opened this issue Jul 22, 2024 · 1 comment
Open

LuceneIndexMaintenanceTest sometimes take many minutes #2842

ScottDugas opened this issue Jul 22, 2024 · 1 comment
Assignees
@ScottDugas

Comments

@ScottDugas
Copy link
Contributor

I've seen test runs where it takes many minutes. Often on manyDocument or manyDocumentSlow or concurrentStoreTest
@ScottDugas ScottDugas self-assigned this Jul 22, 2024
ScottDugas added a commit to ScottDugas/fdb-record-layer that referenced this issue Jul 23, 2024
@ScottDugas
Add asyncToSyncTimeout for LuceneIndexMaintenanceTest
38bd624 
To help investigate FoundationDB#2842
I'm hoping that something will timeout that isn't the test, and
that will give us a good starting point.
@ScottDugas ScottDugas mentioned this issue Jul 23, 2024
Merged
@ScottDugas
Copy link
Contributor Author

After adding the timeout, it seems clear that one test will get a timeout, and then all remaining tests will either get an asyncToSync timeout at the first store open or timeout overall.

ScottDugas added a commit to ScottDugas/fdb-record-layer that referenced this issue Jul 31, 2024
@ScottDugas
Attempt to reduce FDB load during Lucene tests
3577e71 
Sleeping for a second between tests didn't seem to make the tests
behave well, trying to slow down the document generation used in
the most expensive tests.

Hopefully this will make FoundationDB#2842 no longer a problem.
@ScottDugas ScottDugas mentioned this issue Jul 31, 2024
Merged
ohadzeliger pushed a commit that referenced this issue Jul 31, 2024
@ScottDugas
Attempt to reduce FDB load during Lucene tests (#2860)
6a4b6ec 
Sleeping for a second between tests didn't seem to make the tests
behave well, trying to slow down the document generation used in
the most expensive tests.

Hopefully this will make #2842 no longer a problem.
ScottDugas added a commit to ScottDugas/fdb-record-layer that referenced this issue Aug 1, 2024
@ScottDugas
Log fdbcli status after failed test
d1075c1 
This is an attempt to investigate Issue FoundationDB#2842 in a remote environment.
The theory is that the cluster is in a state where it is rejecting
all transactions, but the hope is that it is in a state where
status works, and will tell us what is wrong.

This is resiliant to not having `fdbcli` on your path or if you
are using `fdb-environment.properties` instead of the default
cluster file, but it will just print errors that it could not get
the status.
@ScottDugas ScottDugas mentioned this issue Aug 1, 2024
Merged
ohadzeliger pushed a commit that referenced this issue Aug 1, 2024
@ScottDugas
Log fdbcli status after failed test (#2862)
2c8314b 
This is an attempt to investigate Issue #2842 in a remote environment.
The theory is that the cluster is in a state where it is rejecting
all transactions, but the hope is that it is in a state where
status works, and will tell us what is wrong.

This is resiliant to not having `fdbcli` on your path or if you
are using `fdb-environment.properties` instead of the default
cluster file, but it will just print errors that it could not get
the status.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ScottDugas ScottDugas

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ScottDugas