handles ndarray type parsing for types in short notation with bytes - #9
Conversation
DType.fromLabel now accepts NumPy-style short forms (e.g. u2, f4, |u1, =c8) in addition to standard labels, consistent with appose-python. Explicit byte orders (< or >) are rejected, since array data is always in native byte order. Labels are now matched case-sensitively. Also add DType.FLOAT16. See also apposed/appose-python#9. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NDArray now accepts NumPy-style short forms (e.g. u2, f4, |u1, =c8) in addition to standard names, normalizing them to the standard name (e.g. uint16), so that only standard names cross the IPC channel. Only platform-independent types are supported, via a fixed lookup table, so that parsing behaves the same in every environment. Explicit byte orders (< or >) are rejected: array data is always native order. See also apposed/appose-java@655db12. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
7342403 to
8c00e30
Compare
|
@gletort Thanks for the detailed report on Zulip as well as this PR, and apologies for the delay in response. Your diagnosis was spot on: short-form dtypes like While digging in, I decided dtype handling needed a more thorough fix across both appose-python and appose-java, so Claude and I built on your branch:
One gotcha: |
It allocates shared memory matching the array's dtype and shape, then copies the data by assignment, so that NumPy converts values into the native byte order and C-ordered layout. This handles e.g. big-endian images from file readers, whose str(dtype) is '>u2' and thus rejected. Add numpy as a dev dependency, for testing it. As a consequence, the worker in test_crash_with_active_task now warns that numpy is installed but not imported, polluting the stderr under test; so import it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a dtype like '>u2' is rejected, the error now names the standard dtype to use instead (e.g. 'uint16'), and points to arr.dtype.name and NDArray.from_ndarray. Also document that str(arr.dtype) is not suitable for non-native arrays, whereas arr.dtype.name is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
On my use case, switching the |
Ahh, I forgot to push. 🤦 Done now! I'll merge it as soon as tests pass. |
Name the NumPy-to-Appose copy after ShmImg.copyOf in imglib2-appose. For Appose-to-NumPy, implement __array__ so numpy.asarray(nda) wraps the shared memory without copying, and deprecate ndarray() in favor of it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
OK, I lied: I decided to fix the horribly confusing naming of Copy a native array into new shared memory
View shared memory as a native array (no copy)
Get an
|
In NDArray() function, dtype str can be a short notation in bytes instead of bits (more standard). Handles that case by not dividing by 8 when it's not a standard notation.