Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: ak.num should always return a useful (non-unknown length) type #2785

Merged
merged 10 commits into from
Nov 1, 2023

Conversation

agoose77
Copy link
Collaborator

@agoose77 agoose77 commented Oct 30, 2023

Replaces #2774 by ensuring the return type of ak.num is always a valid "index type" (something that kernels produce).

It might be that this is an integer rather than an np.ndarray (or cp.ndarray, etc); this is up to the nplike. Right now, nplikes don't promote Python integers to 0D arrays.

@agoose77 agoose77 marked this pull request as ready for review October 30, 2023 11:14
@codecov
Copy link

codecov bot commented Oct 30, 2023

Codecov Report

Merging #2785 (b8ace29) into main (23f5322) will increase coverage by 0.00%.
The diff coverage is 66.66%.

Additional details and impacted files
Files Coverage Δ
src/awkward/operations/ak_num.py 93.75% <66.66%> (+0.20%) ⬆️

@agoose77 agoose77 temporarily deployed to docs October 30, 2023 11:31 — with GitHub Actions Inactive
Copy link
Member

@jpivarski jpivarski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one fix, and then it's ready to merge (i.e. no need for a second review).

tests/test_2785_ak_num_typetracer_axis_0.py Show resolved Hide resolved
@agoose77 agoose77 temporarily deployed to docs October 30, 2023 13:54 — with GitHub Actions Inactive
@agoose77 agoose77 enabled auto-merge (squash) October 31, 2023 12:25
@agoose77 agoose77 temporarily deployed to docs October 31, 2023 12:34 — with GitHub Actions Inactive
@agoose77
Copy link
Collaborator Author

@jpivarski to confirm — forcibly returning ArrayLike values from ak.num means that it's not possible to use ak.num to return concrete lengths for typetracers with known shapes. I assume you're OK with that, but I want to surface before we merge.

@agoose77 agoose77 temporarily deployed to docs October 31, 2023 12:58 — with GitHub Actions Inactive
@agoose77 agoose77 temporarily deployed to docs October 31, 2023 14:11 — with GitHub Actions Inactive
@jpivarski
Copy link
Member

@jpivarski to confirm — forcibly returning ArrayLike values from ak.num means that it's not possible to use ak.num to return concrete lengths for typetracers with known shapes. I assume you're OK with that, but I want to surface before we merge.

I don't see anything wrong with that—internally in the codebase, we should be using .length and users have len if they want to make a distinction between getting the length eagerly at DAG-building time (if the TypeTracer has a known shape) and getting the length as delayed data until it's taken up by a Dask worker.

In fact, if I had to make that distinction, I would expect the Awkward function to give me delayed data and the Python function to give me eager data, because Awkward knows about Dask and Python doesn't.

I guess we're losing a case of "give me the eager, numerical length if you have it and a delayed token if you don't." But if I wanted that case, I wouldn't have guessed that ak.num would provide it (and it would be a tricky thing to explain in the documentation—likely undiscoverable, anyway). If I needed this case, I'd write a try-except around len.

@agoose77 agoose77 enabled auto-merge (squash) October 31, 2023 20:41
@pre-commit-ci pre-commit-ci bot temporarily deployed to docs October 31, 2023 20:57 Inactive
@agoose77 agoose77 merged commit d3793f4 into main Nov 1, 2023
36 checks passed
@agoose77 agoose77 deleted the agoose77/fix-ak-num-return-type branch November 1, 2023 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants