Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rerun_py: Improvements to dataframe and simple unit-test framework #7600

Merged
merged 5 commits into from
Oct 7, 2024

Conversation

jleibs
Copy link
Member

@jleibs jleibs commented Oct 4, 2024

What

  • Allow select() to support *args
  • Return a RecordBatchReader instead of list[RecordBatch]
  • Add a unit-test that creates a recording, opens it, and does some stuff.

We still aren't able to iterate over the result-set without collecting it, as we need to fix:

// TODO(cmc): This needs to be a refcounted type that can be easily be passed around: the ref has
// got to go. But for that we need to generally introduce `ChunkStoreHandle` and `QueryCacheHandle`
// first, and this is not as straightforward as it seems.
pub struct QueryEngine<'a> {

Future work:

  • Lots of fun stuff to do here with the zoo

Checklist

  • I have read and agree to Contributor Guide and the Code of Conduct
  • I've included a screenshot or gif (if applicable)
  • I have tested the web demo (if applicable):
  • The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG
  • If applicable, add a new check to the release checklist!
  • If have noted any breaking changes to the log API in CHANGELOG.md and the migration guide

To run all checks from main, comment on the PR with @rerun-bot full-check.

@jleibs jleibs added 🐍 Python API Python logging API exclude from changelog PRs with this won't show up in CHANGELOG.md labels Oct 4, 2024
@jleibs jleibs marked this pull request as ready for review October 4, 2024 20:41
@jleibs jleibs force-pushed the jleibs/dataframe_improvements branch from b5aa6e9 to 6977862 Compare October 4, 2024 20:41
@teh-cmc teh-cmc self-requested a review October 7, 2024 07:20
@teh-cmc teh-cmc merged commit 42cdfef into main Oct 7, 2024
34 of 35 checks passed
@teh-cmc teh-cmc deleted the jleibs/dataframe_improvements branch October 7, 2024 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
exclude from changelog PRs with this won't show up in CHANGELOG.md 🐍 Python API Python logging API
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants