Optimize get_artifacts_for_build() with regex filtering #4297
Conversation
Introduces regex-based filtering directly in the API to improve performance and reduce number of calls to ab api server.
|
/gcbrun |
| @@ -118,10 +118,19 @@ def download_artifact(client, bid, target, attempt_id, name, output_directory, | |||
| return output_path | |||
|
|
|||
|
|
|||
| def get_artifacts_for_build(client, bid, target, attempt_id='latest'): | |||
| def get_artifacts_for_build(client, bid, target, attempt_id='latest', | |||
| regexp=''): | |||
There was a problem hiding this comment.
default to None please.
Also, please add type annotations to this function.
| target=target, | ||
| attemptId=attempt_id, | ||
| nameRegexp=regexp, | ||
| maxResults=100) |
There was a problem hiding this comment.
Why is maxResults passed here but not above?
| """Return list of artifacts for a given build.""" | ||
| request = client.buildartifact().list( | ||
| buildId=bid, target=target, attemptId=attempt_id) | ||
| if regexp == '': |
There was a problem hiding this comment.
nit: if not regexp: may read a little better.
There was a problem hiding this comment.
This change looks good.
@aditya-wazir it would be really interesting to see about adding a test for files located in the src/clusterfuzz/_internal/platforms/android/ directory. As far as I can tell, there are not any (at least, not in src/clusterfuzz/_internal/tests/core).
Could you please investigate the level of effort to add a new test file for android here? Maybe we could just start with a single fetch_artifact_test.py file, and build on the structure thereafter.
Introduces regex-based filtering directly in the API to improve performance and reduce number of calls to ab api server.