Compare commits

...
70 Commits
Author SHA1 Message Date
Ken Museth c81d904ded fixed bug in VolumeToMesh.cuh (#2306)
Signed-off-by: Ken Museth <ken.museth@gmail.com>
2026-08-31 10:31:01 -07:00
Ken Museth a89012d569 improved the vdb option in the vdb_tool (#2294)
* improved the vdb option in the vdb_tool

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* vdb_tool: address review of the vdb/geo name-resolution option

Two issues raised in review of the name-resolution feature:

1. Resolution was written back into Option::value, destroying the parsed source
   expression. Because a loop body revisits the same Action every iteration, this
   pinned the option to whatever the first iteration resolved: the documented
   pattern "-for v=0,3 -render vdb={$v} -end" rendered age 0 on every pass. It
   would also have corrupted the source form that -write emits to a config file,
   which reads Option::value directly.

   Resolution now runs per-read on a copy instead of mutating the option:
   Parser::beforeActionRun is replaced by Parser::onGetOption, invoked from
   Parser::getStr() after expression expansion. Every accessor (get<T>, getVec<T>,
   getVec3<T>) funnels through getStr(), so coverage is unchanged, but the stored
   expression survives and is re-expanded and re-resolved each iteration.

2. Expanding a duplicated name to several ages was unsafe. Most actions read
   vdb/geo as a single age via get<int>() and cannot parse a list at all (29 such
   call sites versus 19 list consumers), and list consumers such as -clear erase
   by index while iterating, so a multi-age value could delete the wrong entry.
   A name must now identify exactly one entry; an ambiguous name is rejected with
   an error listing the matching ages so an explicit index can be used. Because
   this fires during option resolution, the action aborts before mutating anything.

Also exempts -soup2ls, whose "vdb" option selects an output resolution level of
the shrink-wrap hierarchy rather than an entry on the grid stack.

Tests: inverted the ambiguity case, added the requested loop regression using
vdb={$v}, a bare-name-in-loop case, a no-partial-deletion check, and a
singleton-consumer (-rename) ambiguity check.

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* vdb_tool: fix partial deletion in -clear with multiple ages

"-clear" erased entries one at a time inside its loop, but an age is a distance
from the back of the stack, so removing one shifted the ages of those still to be
processed. On a three-grid stack "-clear vdb=0,2" erased age 0, leaving two
grids, and then failed on age 2 -- having already destroyed a grid. An invalid
age behaved the same way, aborting only after earlier deletions had been applied.

Reachable with plain integer indices and unrelated to name resolution, though the
name-resolution work made it easy to hit.

Every requested age is now resolved to a list iterator, and duplicate ages
removed, before anything is erased. An out-of-range index therefore leaves both
stacks untouched, matching how "-copy" validates all of its indices up front, and
a repeated age such as "vdb=0,0" deletes a single entry instead of erasing the
same iterator twice (which is undefined behaviour). Resolving up front is safe
because mGrid/mGeom are std::lists, whose iterators remain valid when other
elements are erased.

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* vdb_tool: rename the -soup2ls "vdb" option to "levels"

"-soup2ls" used "vdb" for something no other action does: selecting which
resolution levels of the shrink-wrap hierarchy to output, rather than referring
to an entry on the VDB stack. That collision forced the grid-name resolution
introduced earlier in this branch to special-case the action, and it read badly
in help output, needing the caveat "unlike other actions, here vdb selects
outputs (not inputs)".

Renaming the option to "levels" removes both problems: the resolution hook no
longer needs an exemption, and every remaining "vdb"/"geo" option in the tool
now uniformly means "an entry on one of the two stacks". No other option on the
action begins with "l", so the name stays unambiguous under prefix matching.

Also drops the stale reference to "vdb" in the neighbouring "dim" description,
copy-pasted from actions where "vdb" supplies a transform; -soup2ls never read
it that way.

This option has not appeared in a release (it is still described only in
pendingchanges), so the rename is not a break for any published version and the
existing changelog entry is amended in place rather than adding a deprecation.

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* review comments

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
2026-08-27 12:43:32 -07:00
Nick Avramoussis a14a735de9 Merge pull request #2204 from Idclip/vectorize_pca_transfers
Vectorized weights and covariance transfers in points::pca methods
2026-08-28 03:14:05 +12:00
Jonathan Swartz 9df9ec060d NanoVDB ReadAccessor::getDimAndActive (#2220)
* Add fused ReadAccessor::getDimAndActive API

Introduce a fused getDim + isActive traversal that shares the
Root->Internal->Leaf descent, the accessor cache update, and the
terminating-level mask read between the two queries. Adds a
DimAndActive POD, getDimAndActiveAndCache on Root/Internal/Leaf,
and getDimAndActive on all four ReadAccessor specializations.

Two policy tags control how `active` is resolved when an
InternalNode skip-flag fires along the descent:

  - ActiveExact (default): preserves byte-for-byte semantics of
    separate getDim + getAndCache<GetState>. `active` is correct
    even when the skip-flag is set, at the cost of forcing the
    child descent. Required by callers that read `active` at
    upper levels (e.g. fvdb HDDA iterators).

  - ActiveOnLeafOnly: cheap skip-flag short-circuit; `active` is
    unspecified when the returned `dim` exceeds leaf-dim, and the
    caller must gate on `dim` first
    (`if (hdda.dim() > 1 || !active) continue;`).

HDDA.h ZeroCrossing is intentionally not migrated -- on the
register-tight sm_120 kernel, any fused shape grows reg/thread by
2-3 and drops occupancy from 4 to 3 blocks/SM. The kernel is
memory-latency bound (~14% achieved occupancy) and the L1 cache
absorbs the second descent at ~87% hit rate, so the occupancy hit
dominates the descent-fusion savings. Rationale recorded inline.

Also rework the ex_raytrace_level_set benchmark harness for honest
measurement: kernel-only cudaEvent timing on the CUDA path, 2
untimed warmup iters, min/median/mean/max reporting,
numIterations 50 -> 200, block size 512 -> 256.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb: pack DimAndActive into a single int32_t

The fused getDim+isActive traversal returns a {dim, active} pair. The
otherwise-natural {uint32_t, bool} struct layout passes through two
registers (or a packed 64-bit pair) at the function-return boundary.

NanoVDB's getDim return value is bounded by the largest internal-node
DIM in the default FloatTree hierarchy (4096 for an upper internal --
2^12, well below 2^31). The sign bit is unused, so we can pack:

  low 31 bits = dim
  sign bit    = active

The struct keeps its existing API surface (aggregate-init via a 2-arg
constructor, accessors dim() and active()), so callers that used
`return {dim, active}` continue to work; consumers that previously
read `.dim` / `.active` as members switch to `.dim()` / `.active()`.

Effect on consumers:

  - On the static ZeroCrossing kernel (which does not use the fused
    API; rationale comment updated below): no change.

  - On lean_hdda + fusion (a separate experiment) the renderOp kernel
    drops from 62 regs (struct encoding) to 61 regs (packed encoding).
    Doesn't cross an occupancy tier on sm_120 at b=256, so the wall-
    time delta is within noise. Still useful: any consumer holding
    DimAndActive live across other work gains 1 reg of headroom for
    no semantic change.

  - On fvdb's HDDA iterators (the productive consumer that reads
    active unconditionally at upper levels), the packing reduces
    return-boundary register pressure in a hot path.

Also expand the rationale comment in HDDA.h's ZeroCrossing. The
original justification framed the don't-migrate decision as a
register-cliff problem; testing against lean_hdda (which has 3 regs
of headroom for fusion) shows the same negative wall-time result
without an occupancy hit. The real reason is the second descent is
L1-warm at ~87 % hit rate, so eliminating it saves ~no cycles, and
the fused call's slightly heavier per-call overhead nets out slightly
worse. Verified both struct- and packed-encoded fusion across both
regimes.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb: drop ZeroCrossing rationale comment

The long NB block that explained why HDDA::ZeroCrossing keeps
separate getDim + isActive calls (instead of using the fused API
this branch adds) was useful during the iteration but doesn't earn
its keep in the source tree -- a future reader doesn't need to relive
the cliff/lean/descent-cost investigation that justified the decision.

The rationale is preserved in the PR description: ZeroCrossing's
`dim > 1 || !active` short-circuit makes the active read dead at upper
levels, and the L1 cache absorbs the leaf-level descent at ~87% hit
rate, so fusion saves essentially no cycles while adding a small per-
call overhead.  The fused API is retained for callers that read active
unconditionally (fvdb's HDDA iterators -- -27% to -62% wall time win).

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* comment cleanup

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* comment update

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb: fix DimAndActive packing UB and add getDimAndActive test

The DimAndActive constructor packed active via 'int32_t(a) << 31',
which shifts 1 into the sign bit of a signed int -- undefined behavior
under C++17 (the repo's minimum standard). Do the OR in uint32_t so the
shift is well-defined, and mask dim to 31 bits so a stray high bit can't
clobber the active bit.

Add TestNanoVDB.GetDimAndActive, which asserts the fused
getDimAndActive matches the separate getDim + isActive it replaces:

  - ActiveExact: {dim, active} equals getDim(ijk, ray) and isActive(ijk)
    byte-for-byte across a box spanning interior, surface and exterior.
  - ActiveOnLeafOnly: dim always matches getDim; active matches isActive
    whenever dim == 1 (the contract gate).

The sphere's narrow band is too thin to fill an InternalNode, so a
skip-flagged internal node above an active voxel cannot arise from the
geometry. The test forces that case by setting the skip-flag on the
lower node over a known-active voxel, then confirms ActiveExact descends
past it (active == true) while ActiveOnLeafOnly short-circuits
(active == false), both reporting the skipped dim.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* comment cleanup

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb: address PR review on DimAndActive and raytrace example

- DimAndActive: store the packed field as uint32_t instead of int32_t.
  The previous int32_t form converted a uint32_t with bit 31 set to a
  signed int (implementation-defined) and left the defaulted ctor's
  member uninitialized. uint32_t packs/decodes with no conversion, the
  default ctor zero-initializes, and the size/ABI is unchanged.

- ex_raytrace_level_set reportStats: take samples by const reference,
  early-return on an empty vector (front()/back() would otherwise be
  UB if numIterations were 0), and sort a local copy.

- ex_raytrace_level_set common.h: check the cudaEventDestroy return
  codes with NANOVDB_CUDA_CHECK_ERROR, consistent with the rest of the
  CUDA timing path.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb: assert dim fits in 31 bits when packing DimAndActive

The packed encoding masks dim to 31 bits (bit 31 holds active). dim is
1u << TOTAL, bounded by the deepest node dim (4096 for the default tree),
so it always fits -- but a pathological tree config with TOTAL >= 31 would
be silently truncated. Add a debug assert documenting the invariant, in
the same style as the version-packing asserts above. Release builds are
unaffected (assert compiles out; the mask still protects the active bit).

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Add pending changes entry

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Fix compile warnings

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Reduce comment verbosity

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-08-25 15:15:25 -07:00
Mark HarrisandClaude Fable 5 71da3a862e NanoVDB: add cuda::copyTo cross-space transfers and pinned-buffer GridHandles (CUDA) (#2292)
cuda::copyTo is the explicit, stream-carrying deep copy between grid
handles in different address spaces: a single-space device handle copies to
a host-readable handle (HostBuffer or a cuda::Buffer over a host-accessible
resource) and back, and device-to-device across different resources. An
optional prototype buffer routes the destination allocation through a
caller-supplied resource, which non-default-constructible resources (e.g.
ResourceRef) require. A host-readable destination synchronizes the transfer
stream before returning; a device destination parses and validates its
metadata on the transferred bytes. Dual-space buffers are rejected with a
named error: they keep deviceUpload/deviceDownload until their retirement.

GridHandle now accepts buffers over host-accessible resources (e.g.
PinnedResource) as host-readable single-space handles instead of rejecting
them: metadata parses on the host, the host accessors and write() apply,
and read() and copy() allocate through the buffer's resource via a shared
host-storage factory (BufferT::create for buffers that provide it, the
resource otherwise). The byte-element requirement now covers host-single
buffers, and splitGrids/mergeGrids name their create() requirement instead
of failing inside the template body.

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 09:48:10 -07:00
Jonathan SwartzandClaude Fable 5 c814fae1fe NanoVDB: chain DeviceBuffer::recordUse and expose orderAfterPriorUses (CUDA) (#2287)
* NanoVDB: chain DeviceBuffer::recordUse and expose orderAfterPriorUses (CUDA)

recordUse re-records the single per-device tracking event, and re-recording
moves an event: uses recorded on streams A then B left only B's coverage, so
the device free could run while A's work was in flight — violating the
invariant documented on mEvents ("every use waits on this event before
issuing work and re-records it afterwards"). Record now waits on the prior
capture first, so the event transitively covers every recorded use. Also make
orderAfterPriorUses public as the consume-side companion, so external
consumers (e.g. the Python bindings' zero-copy exports) can order their own
stream after the tracked uses. Adds a DeviceBufferChainedRecordUse regression
test that fails before the fix and passes after.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Address review: doc concurrent-reader trade-off, guard vacuous test pass, tidy access sections

- recordUse's doc note now states that concurrent READERS recording uses
  serialize behind each other (the single per-device event cannot distinguish
  read-read from write-read) and names the upgrade path (read/write-separated
  or per-record events) should a profile ever surface it.
- DeviceBufferChainedRecordUse now GTEST_SKIPs when the allocator did not
  recycle the freed block, making a vacuous run visible instead of green.
  Verified on sm_120 that stream-ordered pools recycle WITH the dependency
  attached even post-fix, so the skip does not fire on a working platform.
- orderAfterPriorUses moved into the public section next to recordUse instead
  of a public/private island inside the private section.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 22:42:22 -07:00
Jonathan SwartzandClaude Opus 5 2176f69923 NanoVDB: assign each upper-node tile to a single GPU in DistributedPointsToGrid (#2283)
* NanoVDB: assign each upper-node tile to a single GPU in DistributedPointsToGrid

The device segment boundaries produced by the initial even split were
rebalanced by adjusting only adjacent pairs of boundaries. That cannot
consolidate an upper-node tile spanning three or more GPUs, since a
fully-interior device lies entirely within the tile. Downstream
construction assumes each tile - and therefore each lower node, leaf node
and voxel - is owned by exactly one device, so multiple devices built the
same leaf concurrently and raced on its value mask, silently dropping
active voxels.

Replace the pairwise GPU rebalance with a global, monotonic snap performed
on the host over the (small) set of device boundaries. The keys are
globally sorted at that point, so a tile boundary is simply a position
where the key changes; fully-interior devices are left empty, which the
rest of the pipeline already handles.

Add two multi-GPU regression tests: SingleUpperNode, which places all
input in one upper-node tile and asserts the exact unique active-voxel
count, and MatchesSingleGpu, which cross-checks topology and occupancy
against the single-GPU builder on the same input.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Added a `snapBoundariesToRunsKernel` and running on device to avoid stream sync

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 17:16:36 +12:00
Ken Museth 83890bd879 fixed warnings that caused build to fail (#2297)
* fixed warnings that caused build to fail

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* reviewer comment

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
2026-08-24 21:13:17 -07:00
Ken MusethandClaude Opus 4.8 eb0f1bfd83 added build support on Ampere (#2267)
* added build support on Ampere

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* vdb_tool: link vdb_tool_test against GTest::gtest only

The unit test provides its own main() and uses only gtest (no gmock), so it
needs just GTest::gtest. Listing gmock / gtest_main / gmock_main as well pulled
libgtest.a and libgmock.a onto the link line multiple times, which the macOS
(Xcode 15+) linker flags as "ignoring duplicate libraries". Linking only
GTest::gtest removes the warning and also works against gtest-only GoogleTest
installs (BUILD_GMOCK=OFF).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Ken Museth <ken.museth@gmail.com>

* review comments

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-24 21:07:34 -07:00
Ken Museth c6353069c8 added histogram action to vdb_tool (#2296)
* added histogram action to vdb_tool

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* now -stats and -histogram handles vector grids

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* fixed various issues (see comment in PR)

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* addressed reviewer comments

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
2026-08-24 20:28:26 -07:00
Mark HarrisandClaude Fable 5 d19e483564 NanoVDB: migrate handle reset() onto destroy() and give NodeManager a single-space overload (CUDA) (#2293)
The handles (GridHandle, NodeManagerHandle, tools::VoxelBlockManager) now
release storage through the buffer's destroy() when it provides one and
fall back to the legacy clear() otherwise, detected by a BufferHasDestroy
trait companion. That lets the transitional cuda::Buffer::clear() and
cuda::BufferView::clear() finally carry their [[deprecated]] attribute
(BufferView gains destroy() with its detach semantics); the legacy
HostBuffer/DeviceBuffer clear() methods are unaffected.

cuda::createNodeManager gains a single-space overload taking a memory
resource: the returned handle owns a cuda::Buffer<std::byte,
ResourceRef<ResourceT>> whose storage and size scratch allocate through the
injected resource, which must outlive the handle -- the builders'
ownership contract. NodeManagerHandle supports single-space buffers
(deviceMgr maps onto the buffer, host accessors become compile-time
errors, constructors member-initialize so non-default-constructible
buffers work), and tools::cuda::GridStats builds its temporary NodeManager
through its injected resource, closing the last allocation that bypassed
ResourceT. The dual-space overload's raw size-scratch pair becomes an RAII
buffer, and both overloads now synchronize before the host reads the
device-computed size, which the old code consumed without ordering.

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:11:14 -07:00
Jonathan Swartz b2b4d30648 Fix out-of-date signature in TEACHME (#2295)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-08-23 15:45:30 -07:00
Mark HarrisandClaude Opus 4.8 2a301b1290 Add NanoVDB TEACHME interactive tutorial + docs-compile test harness (#2226)
* Add NanoVDB TEACHME interactive tutorial + docs-compile test harness

doc/nanovdb/TEACHME/ is an interactive, LLM-guided tutorial for the
NanoVDB user API: 10 modules (reading grids, ReadAccessor, math/HDDA
sampling, GPU kernels, GPU grid builders, topology operators + data
re-homing via Injection, IndexGrid/VoxelBlockManager, OpenVDB
conversion), a cheat sheet, and a GPU level-set ray-march capstone.

To keep the tutorial from drifting from the API, the test harness in
doc/nanovdb/TEACHME/test/ extracts every C++/CUDA code block from the
docs and compiles it against the real NanoVDB headers, and validates
that every #include and prose header reference resolves. ~64 blocks
compile green.

A CI workflow (.github/workflows/teachme.yml) runs the harness whenever
the NanoVDB headers or the tutorial change, so API drift is caught
automatically. It is compile-only and needs no GPU.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: add pending-changes note for the TEACHME tutorial

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: note the TEACHME tutorial in pendingchanges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-23 20:53:21 +00:00
Ken Museth e68ff4bc8e Added new actions to vdb tool (#2259)
* added a new -copy action to vdb_tool

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* added the actions: copy,diagnose,rename,stats,swap,multiply,divide to vdb_tool

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* fixed some issues

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* major cleanup

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* added line continuation to config files

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* removed duplicate headers

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* snapshot

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* addressed reviewer comments (see detailed below)

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* review comments

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* fix to change log

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
2026-08-23 11:29:29 -07:00
Mark HarrisandClaude Fable 5 e679862fca NanoVDB: support single-space device buffers in GridHandle (CUDA) (#2288)
* NanoVDB: support single-space device buffers in GridHandle (CUDA)

GridHandle can now own a nanovdb::cuda::Buffer<std::byte, R>: a new
hasDeviceSingle buffer trait selects a constructor (implemented in
cuda/GridHandle.cuh) that parses the grid metadata through the device,
with every operation -- the validated header walk, metadata scratch
allocation through the buffer's resource, the copy kernel and the
readback -- ordered on the buffer's retained stream.
deviceData()/deviceGrid() work on such handles. The host accessors
(data, grid, gridData, gridMetaData) are SFINAE-removed for them,
since the handle owns no host-readable bytes; the read/write I/O
members and splitGrids/mergeGrids stay addressable (the python
bindings take write's address via overload_cast) and fail instead
with an explanatory static_assert when instantiated for a
single-space handle.

The device parse validates the whole grid chain (per-header bounds
checks against the allocation) before launching the metadata kernel,
so truncated buffers and forged mGridCount/mGridSize headers are
rejected with an exception instead of an out-of-bounds device read;
the pre-existing dual-space parse had the same hole and now shares the
validation helper. Its metadata scratch uses MallocResource so that
long-standing path keeps working on devices without memory-pool
support, and the dirty-flag scratch in splitGridHandles and
mergeGridHandles is a resource-aware buffer as well -- together
replacing the file's six raw cudaMalloc/mallocAsync sites, one of
which was missing its cudaCheck.

copy() dispatches on the traits: host buffers keep the memcpy path,
single-space buffers deep-copy device-to-device through their own
resource on the retained stream and reuse the host-resident metadata
instead of re-parsing. Buffer gains ElementType/ResourceType typedefs,
a resource() accessor, and a no-argument copy() for stream-ordered
resources that orders the copy on the retained stream.

Host accessibility is a property of the resource, not the element
type: the new is_host_accessible_resource trait detects a
HOST_ACCESSIBLE marker (declared by PinnedResource and forwarded by
ResourceRef and AsyncFromSync). A cuda::Buffer over a host-accessible
resource is rejected at handle scope with a named error for now --
GridHandle's host paths require an allocation interface cuda::Buffer
does not yet provide -- and non-byte element types fail loudly in the
single-space constructor. Both hasDeviceSingle and hasHostSingle are
detected rather than required, so pre-existing BufferTraits
specializations in and out of tree compile unchanged.

Part of #2232 (step 3).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: validate and parse the device grid chain in one kernel pass

Replace the per-header device-to-host walk in the GridHandle constructors
with a single host read of the head header (which sizes the metadata
scratch) followed by one kernel that validates every header and fills the
metadata scratch in the same pass, plus one readback and one
synchronization regardless of the grid count. The previous walk issued one
synchronizing copy per header, so constructing the handle that
cuda::mergeGridHandles returns cost O(N) stream drains for N grids; it now
costs the same two round trips as a single-grid buffer, and the separate
metadata kernel launch, its scratch copy and final synchronization are
gone. Bounds are still checked before every header read, on the host for
the head and on the device for the rest, so a truncated buffer or forged
header can never read out of bounds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: normalize grid index/count in addBlindData and indexToGrid output

Both tools copy the source grid's header verbatim into a new single-grid
buffer, so a source grid taken from a multi-grid handle left a stale
mGridIndex/mGridCount in the output. The returned handle's metadata parse
then walked a grid chain that is not there: past the end of the buffer for
index 0 of a multi-grid source, or starting at a nonzero index. The output
header is now normalized to index 0 / count 1 in the same kernels that
already patch its other fields, before each tool's existing checksum
update. The other handle-returning GPU tools were audited: the topology
ops and MeshToGrid normalize in TopologyBuilder, and the point builders
initialize fresh headers via GridData::init.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: share the buffer trait detectors and deduplicate GridHandle lookups

BufferHasDeviceSingle/BufferHasHostSingle move next to the BufferTraits
primary in HostBuffer.h: they are companions to the trait protocol, not
GridHandle-specific, so other handle types can consume them without
including GridHandle.h. The three identical grid-lookup bodies behind
grid() and both deviceGrid() families collapse into one private gridAt()
helper, and the supported-combination guard for GridHandle::copy() now
lives solely in the no-arg overload, so the two spellings cannot drift
apart when cross-space transfers relax it. cuda::Buffer documents the
single-space device-buffer concept on its BufferTraits specialization --
naming exactly the interface a consumer of hasDeviceSingle may rely on --
and gains a rebind alias for generic code that constructs sibling buffers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-22 21:23:34 -07:00
Ken Museth 568538e9ec added vdb option to -soup2ls action and improved error handeling (#2266)
* added vdb option to -soup2ls action and improved error handeling

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* improved the documentation

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
2026-08-20 22:11:15 -07:00
Jonathan SwartzandClaude Fable 5 c3ee2009c0 VBM select-based inverse-map decode (#2258)
* NanoVDB CUDA: select-based VoxelBlockManager inverse-map decode

Replace the O(nLeaves x 512) cooperative sweep in decodeInverseMaps with an
O(1)-per-slot select: one thread per output slot ranks itself into its leaf via
the jumpMap popcount, then locates its voxel with the leaf's 9-bit mPrefixSum plus
an in-word __fns bit select. Same signature, byte-exact decode maps.

Verified byte-exact on the VBM goldens for all consumers (decode/box/lap/weno) at
widths 64/128/256/512, and faster for every consumer with no regression: decode-only
2.17x/1.97x/1.62x/1.42x (w6/w7/w8/w9), 7-pt Laplacian -8.5..-15.4%, box/weno -4..-9%.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* NanoVDB: add pending changes entry for the select-based VBM decode

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Additional explanatory comments and cleanup

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* NanoVDB CUDA: consolidate VBM decode to a thread-local per-slot API

Replace the cooperative shared-memory decodeInverseMaps with decodeInverseMap:
a per-slot decode into registers (no shared memory or synchronization; callable
from divergent threads), factored into jumpMapRank + selectVoxelInLeaf helpers.
computeBoxStencil now takes the decoded (leafIndex, voxelOffset) by value.
Block-level facts formerly read from the materialized maps derive directly from
the VBM metadata: the block's first leaf is firstLeafID, slot p starts a new
leaf iff jumpMap bit p is set, and the spanned-leaf count is 1 + the jumpMap
popcount.

Decode maps and all 27 box-stencil taps verified byte-exact against the prior
select decode at widths 64/128/512 across 16-86% leaf occupancy. Decode-only
+14..17% at width 128 and +8..13% at width 512 over the prior select, neutral
at width 64; vs master's sweep: 1.56-2.08x (w7) and 1.24-1.68x (w9) across the
sparsity range.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* NanoVDB CUDA: preserve decodeInverseMaps as a wrapper over the per-slot decode

Restore the cooperative decodeInverseMaps with its pre-existing signature as a
thin wrapper that materializes all BlockWidth slots via decodeInverseMap
(grid-striding over the slots, ending in __syncthreads), so existing callers
keep compiling and behaving unchanged. The per-slot decodeInverseMap remains
the preferred form for consumers that only read their own slot's result.

The unit test decodes through the wrapper (which exercises decodeInverseMap
internally) and resolves via the by-value computeBoxStencil; both
VoxelBlockManager tests pass.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 11:12:37 +12:00
Mark HarrisandClaude Fable 5 7946f17edb NanoVDB: own the small builders' device scratch with cuda::Buffer (CUDA) (#2286)
* NanoVDB: own the small builders' device scratch with cuda::Buffer (CUDA)

Route the last raw mallocAsync/freeAsync pairs in the GPU tools through
RAII cuda::Buffer scratch over an injectable resource, completing the
audit that PointsToGrid, TopologyBuilder, MeshToGrid and the topology-op
consumers already went through: IndexToGrid (node accessor + grid name),
addBlindData (byte-size scratch), GridStats (per-node statistics
scratch) and SignedFloodFill (node counts). Each gains a defaulted
ResourceT template parameter following the established conventions --
classes take a must-outlive resource instance in the constructor, free
functions take the template parameter and route through the per-type
default instance. Every replaced free keeps its position in stream
order via destroy(stream) at the old freeAsync site, and GridStats'
mode-dependent allocation becomes a zero-element buffer when the mode
has no average, so the resource sees identical traffic.

IndexToGrid's nested NodeAccessor is hoisted to the namespace-scope
IndexToGridNodeAccessor so the kernels that name it do not acquire a
spurious ResourceT dependency; a public alias preserves the nested
spelling.

Seam tests with exact allocation counts cover all four (including the
grid-name path, the Extrema zero-allocation path, and free-function
forwarding to the default instance).

Part of #2232 (step 2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: document GridStats' NodeManager allocation boundary

The temporary NodeManager in GridStats::update allocates through the
dual-space DeviceBuffer, which cannot yet take a resource; make that
visible on update() so users injecting a resource for accounting or
limits know this allocation bypasses it until the single-space handle
work lands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 10:12:11 +12:00
Nick Avramoussis 204fce09fb Merge pull request #2285 from Idclip/codeowners
Improve codeowners lists for more targetted reviewers
2026-08-19 10:57:14 +12:00
Nick Avramoussis 385584be80 Improve codeowners lists for more targetted reviewers
Signed-off-by: Nick Avramoussis <4256455+Idclip@users.noreply.github.com>
2026-08-19 10:36:13 +12:00
Eftychios Sifakis 2728de45a1 nanovdb: undef CALL_CUBS at the end of MeshToGrid.cuh (#2284)
TopologyBuilder.cuh and PointsToGrid.cuh both undef CALL_CUBS after
their last use so the macro doesn't leak into whatever gets included
next in the same translation unit. MeshToGrid.cuh was missing this.

Signed-off-by: Efty Sifakis <esifakis@nvidia.com>
2026-08-19 10:05:12 +12:00
Mark HarrisandClaude Fable 5 3f35101729 NanoVDB: run the builders on a synchronous memory resource (CUDA) (#2272)
* NanoVDB: align cuda::Buffer member names with cuda::buffer

Name the free operation destroy, as cuda::buffer does, and add the
stream-taking overload it also provides. clear stays but only as a
transitional delegate, marked as such: it exists because
GridHandle::reset still calls it, and goes away with the legacy dual
buffers that cuda::Buffer replaces.

Rename setStream to set_stream. Member names cannot be aliased, so
matching the standard spelling is the whole reason the resource concept
kept allocate_async; the same argument applies here. Note in passing that
ours deliberately does not synchronize, matching cuda::buffer's
set_stream_unsynchronized rather than its set_stream, whose own
documentation and implementation disagree (NVIDIA/cccl#10649).

Add swap. The generic std::swap already does the right thing through the
move operations, but both std::vector and rmm::device_buffer provide one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: allocate TopologyBuilder scratch from an injected resource

Eleven of the builder's buffers are device-only -- nothing reads them on
the host -- yet they were cuda::DeviceBuffer, whose host pointer and
per-device array they never use. Move them to the single-space
cuda::Buffer and give the builder a resource parameter, so its scratch is
injectable like PointsToGrid's already is, and freed by scope rather than
by hand. mProcessedRoot and mData are read on the host and stay dual.

This is not a speedup: DeviceBuffer::init allocates host or device memory
but never both, and these buffers always passed a real device id, so no
cudaMallocHost was on this path to begin with. Measured dilate on 1k/20k/
200k points, and the difference is within run-to-run noise.

Hoist the Data struct out of the class. It does not depend on the
resource, and leaving it nested would give every ResourceT its own
incompatible type for the device functors to name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: add SyncFromAsync, and give MeshToGrid a resource seam

A stream-ordered resource must also model the synchronous concept, which
means writing four methods where two would do. The synchronous pair is
not a bare delegate -- memory from allocate must be usable on any stream
when it returns, so the null-stream allocation has to be synchronized
first -- and omitting that yields memory which satisfies the concept but
is not actually synchronous. Put it in one place rather than leaving each
author to rediscover it.

The two resources in TestMemoryResource are the first users, and were
already wrong in exactly that way: they provide only the async pair, so
they never modelled is_async_resource. TempPool duck-typed and never
checked, so nothing caught it.

MeshToGrid was the last builder allocating from a hard-wired
DeviceResource, through TempDevicePool. Give it a ResourceT parameter and
thread it into both its TopologyBuilder and its pool. As with Data in the
builder, BoxTrianglePair is hoisted out of the class: it does not depend
on the resource, and leaving it nested would give every ResourceT its own
incompatible type for the device functors to name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: add ResourceRef and route TempPool's scratch through cuda::Buffer

Buffer holds its resource by value, matching cuda::buffer -- whose model
this completes: in CCCL the ownership semantics are selected by what is
placed in the by-value slot, an owning any_resource or a borrowing
resource_ref. We adopted the slot without the borrowing type, so a
container like TempPool, whose contract is a non-owning pointer to a
possibly stateful resource, had no way to hold a Buffer without copying
that resource and stranding its state. ResourceRef is the missing piece:
a non-owning reference that is itself a resource, so copying the ref
shares the underlying instance. Its async methods exist only when R
models AsyncResource, so a ref over a synchronous resource does not
misreport its tier, and two refs compare equal exactly when they
reference the same resource.

TempPool now keeps its bytes in a Buffer<std::byte, ResourceRef<R>>:
same resource contract, same stream retention, same discard-on-growth
reallocation, but the block is freed by ownership rather than by hand.
The TempPool unit tests, which assert traffic against the caller's own
resource instance, pass unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: note cuda::Buffer and cuda::BufferView in pendingchanges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: express PointsToGrid's density search as a loop

The bisection search over voxel size was written as a backward goto,
which made the lifetimes of the buffers it retries over non-lexical.
Rewrite it as while(true) with continue on retry and break on
convergence; the six hand-written frees before the jump are unchanged.
The change is easiest to review with whitespace ignored, since the loop
body re-indents: git diff -w shows 27 changed lines.

d_keys and d_node_count carry results past the loop, so their
declarations move above it, as does the copy event, which was created
inside the retried region on every iteration but destroyed only once at
the end -- each retry leaked the previous handle.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: own PointsToGrid's device arrays with cuda::Buffer

Every device array PointsToGrid allocates is now owned by a
Buffer<T, ResourceRef<ResourceT>> borrowing the injected resource --
members where the pipeline frees them in a later member function than
the one that allocated them (countNodes allocates; processUpperNodes,
processLeafNodes, processPoints and processBBox release), locals where
the lifetime is contained. The raw pointers survive only as views: the
device-visible fields inside mData, and the working pointers the cub
and kernel calls take. Every owner event -- assignment, swap, destroy
-- immediately refreshes its view, and released views are nulled so a
stale use faults instead of reading a freed block. The index ping-pong
becomes a swap of owners across the member/local boundary, replacing
the bare pointer swap whose safety depended on nothing reading the
device copy of d_indx between the two uploads.

The density-search retry keeps its free-before-reallocate order via
explicit destroy calls, so peak device memory is unchanged. The
hand-matched byte sizes at every free site disappear, and the arrays
released one line before scope exit now just leave scope. One behavior
change worth naming: the too-many-points-per-leaf throw previously
leaked the reduction scratch; ownership now releases it during unwind.

Verified against the previous commit with a counting resource over
three shapes (bulk segmented-sort branch, serial per-tile branch, and
the bisection retry engaged): allocation count, free count, and total
bytes are identical. Full CUDA and memory-resource suites unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: free TopologyBuilder scratch on the caller's stream

Each release site sits in a function that receives the stream, so pass
it to destroy rather than relying on the retained stream matching --
they are the same on every current path, but the explicit form does not
depend on that staying true. Also drop the cudaGetDevice calls whose
result the Buffer conversion left unused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: borrow TopologyBuilder scratch through ResourceRef

The scratch buffers held their resource by value, so each of the eight
carried its own copy -- fine for the stateless default, wrong for a
stateful resource, whose accounting would be split across copies while
the caller's instance saw nothing. Borrow through ResourceRef instead,
the same reconciliation TempPool uses.

Assert the stream-ordered requirement directly in TopologyBuilder and
MeshToGrid so a synchronous-only resource fails with a diagnostic that
names the builder, not just the pool inside it. Note SyncFromAsync's
synchronize cost on its allocate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: guard PointsToGrid's copy event with a scope owner

The too-many-points-per-leaf throw unwinds past the event's manual
destroy, leaking the handle. Own it with a small guard so unwinding
releases it, consistent with the buffer ownership in this function.
Also assert the stream-ordered resource requirement on the class, so a
synchronous-only resource fails naming PointsToGrid rather than the
pool inside it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: run the builders on a synchronous memory resource

The builders require a stream-ordered resource, which a device without
memory-pool support cannot provide through cudaMallocAsync. Two additions
close the gap. MallocResource is a synchronous cudaMalloc/cudaFree
resource that works on any device. AsyncFromSync presents a synchronous
resource as a stream-ordered one -- the mirror of SyncFromAsync, and the
analog of cuda::mr's synchronous_resource_adapter: allocate_async
forwards, since synchronously allocated memory is already valid on every
stream, and deallocate_async synchronizes the stream first, which makes
the synchronous contract's quiescence requirement hold. The serialization
cost of that synchronize is documented at the type, so a synchronous
backend is an explicit caller choice rather than a silent substitution.

The new test drives PointsToGrid end-to-end with an injected
AsyncFromSync over a stateful synchronous resource: every scratch
allocation routes through cudaMalloc and is freed through the caller's
instance, never touching stream-ordered allocation -- which is exactly
what a pool-less device requires of the scratch path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: assert the scratch alignment TopologyBuilder relies on

The byte scratch is reinterpreted as word-sized types, which is valid
for every resource whose DEFAULT_ALIGNMENT is at least word alignment --
all CUDA allocation paths give 256 -- but nothing said so. Assert it, so
a custom resource with a weaker guarantee fails at compile time instead
of misaligning on the device.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: terminate a bare cudaCheckError with a semicolon

Legal without one -- the macro expands to a braced block -- but every
other use in the file spells it as a statement.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: make AsyncFromSync's null deallocation a no-op

Null-free is a no-op everywhere else, so there is no quiescence to
establish and no reason to synchronize the stream for it. Also correct
the sync-resource test's comment: the grid handle's output buffer is
allocated through BufferT, not the injected resource, so "every builder
allocation" overstated what the test demonstrates -- it is the scratch
allocations that never touch stream-ordered allocation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: set_stream matches cuda::buffer's contract, not a divergence

cuda::buffer's set_stream is deliberately non-synchronizing -- its
documented synchronization is a stale note left behind when the
synchronization was removed -- so our non-synchronizing set_stream
matches it in both name and contract, and the comment claiming a
divergence was wrong.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: mark cuda::Buffer::clear deprecated in the documentation

Documentation-level only: the [[deprecated]] attribute would warn from
GridHandle::reset and NodeManager::reset, template members in our own
headers that must keep calling clear() until every buffer type provides
destroy() -- an unactionable diagnostic for callers of reset(), and a
build break under -Werror. The attribute lands when those callers
migrate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: exercise the concept-required members of the test resources

The trait checks detect allocate/deallocate through unevaluated contexts,
which never odr-use them, so nvcc warned that every synchronous pair and
stub member was declared but never referenced (#177-D). The attribute
route is closed -- nvcc's front end ignores [[maybe_unused]] for this
diagnostic -- so reference them the honest way: a test that verifies the
synchronous halves of the counting and stream-recording doubles behave
like their stream-ordered halves, and one that pins the trait probes'
stub behavior. The TU now builds with no warnings at all.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 12:36:46 -07:00
Jonathan Swartz 8a8ed7ca3e NanoVDB Python: restructure to mirror the C++ NanoVDB API (#2219)
* nanovdb python: Phase 0 foundation for C++ API mirror (#2209)

* nanovdb python: Phase 0 foundation for C++ API mirror

First slice of the Python bindings restructure tracked in #2208 and
laid out in nanovdb-python-plan.md. Phase 0 is the mechanical
groundwork the rest of the plan builds on:

- BuildTypes.def: single X-macro list of currently-bound BuildT types
  (scalar / vector / point / sampleable). NanoVDBModule.cc, PyMath.cc,
  PySampleFromVoxels.cc, PyCreateNanoGrid.cc, PyTools.cc, PyGridHandle.h
  and cuda/PyDeviceGridHandle.cu now drive their per-type instantiations
  from this one file. Adding a BuildT in Phase 2 becomes one line.

- CMakeLists.txt: under SKBUILD, ship the nanovdb/ headers inside the
  Python wheel at nanovdb/include/nanovdb/ so downstream extension
  authors can compile against the same headers the wheel was built with.
  Also wire nanobind_add_stub so a nanovdb.pyi (and py.typed marker) are
  emitted into the wheel for IDE / type-checker support; gated behind
  NANOVDB_BUILD_PYTHON_STUBS and silently skipped on older nanobind.

- __init__.py: add nanovdb.get_include() returning the bundled include
  dir. Also fix the Windows DLL shim, which referenced an undefined
  `directory` variable instead of the local `openvdb_dll_directory`.

- Relocate the batched sampleFromVoxels CUDA kernel binding from the
  phantom nanovdb.math.cuda submodule (which has no C++ counterpart) to
  nanovdb.tools.cuda, alongside the existing signedFloodFill and
  pointsToRGBA8Grid kernels. The math.cuda submodule is no longer
  registered. TestNanoVDB.py updated to match.

- Pre-existing observable bug fixes:
  * GridHandle.__bool__ returned None (lambda was missing return);
    it now returns !handle.empty() as intended.
  * Enable __repr__ on GridType, GridClass and io::Codec via the
    nanovdb::toStr / strlen<> helpers — previously commented out.

Build + pytest verified locally (CPU-only): 28 tests pass; the single
pre-existing test_read_write_grid BLOSC failure is unrelated (test
doesn't wrap the optional codec call in try/except like its sibling
does).

Part of #2208.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: align Phase 0 X-macros with codingstyle.txt

Audit of the new BuildTypes.def + consumers against
nanovdb/nanovdb/docs/codingstyle.txt:

- Rename the X-macro family NVDB_PY_FOR_EACH_*_BUILDT to
  NANOVDB_PY_FOR_EACH_*_BUILDT and the helper sentinels
  NVDB_PY_LOCAL_DEFINED_* to NANOVDB_PY_LOCAL_DEFINED_* to match the
  established NANOVDB_ macro prefix used everywhere else in the codebase
  (NANOVDB_USE_CUDA, NANOVDB_BUILD_PYTHON_MODULE, NANOVDB_HOSTDEV, ...).

- Bring lines under the 100-column limit:
    * Wrap the long VECTOR_BUILDT lines in BuildTypes.def across two
      lines each (cleaner alignment, no behavior change).
    * Rename the consumer macro parameter DeviceHandleMethod to
      DeviceMethod so the #define line itself fits (was 102 cols).
    * Reformat the GridHandle __bool__ lambda onto three lines instead
      of one 119-col line (made worse by the `return` fix in the
      previous commit).

- Add a top-of-file justification block in BuildTypes.def explaining why
  this file is a deliberate exception to the codingstyle "avoid macro
  functions" rule (templates can't emit top-level declarations and
  explicit instantiations across translation units from a single
  canonical list).

No functional change. Rebuild + pytest_nanovdb is identical to the
previous commit: 28 tests pass, 8 CUDA skips, 1 pre-existing BLOSC
failure.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: only generate .pyi stubs under SKBUILD by default

Fixes the macOS CI failure on #2209. The previous commit defaulted
NANOVDB_BUILD_PYTHON_STUBS to ON, which made the nanovdb_python_stub
target fire on every in-tree CI build. The macOS GitHub Actions runner
wraps the stubgen invocation with

  cmake -E env DYLD_INSERT_LIBRARIES=.../libclang_rt.tsan_osx_dynamic.dylib:
                                      .../libclang_rt.asan_osx_dynamic.dylib:
                                      .../libclang_rt.ubsan_osx_dynamic.dylib
  ASAN_OPTIONS=detect_leaks=0 python stubgen.py -m nanovdb ...

(this preamble is added by the runner / CMake env wrapper — nothing in
our nanobind_add_stub call sets it, and the .so itself is not built with
-fsanitize). ThreadSanitizer can only install its interceptors at
process start; Python loads first and then dlopen's the compiled .so,
which TSan considers "too late" and aborts:

  ==49907==ERROR: Interceptors are not working. This may be because
  ThreadSanitizer is loaded too late (e.g. via dlopen).

So the macOS build target failed with exit 2 after the .so itself built
fine. The other matrix legs (linux-nanovdb Debug/Release for clang/gcc)
all passed.

Stubs are only useful to wheel consumers — they ship next to the .so in
the installed package layout under SKBUILD. The in-source OpenVDB CI
build never consumes them, so making stub generation default ON only
when SKBUILD is set keeps the wheel build path unchanged and stops the
macOS CI from invoking stubgen under the sanitizer wrapper. The user can
still force generation with -DNANOVDB_BUILD_PYTHON_STUBS=ON for local
dev builds where it's useful.

Verified locally:
  - in-tree config (SKBUILD unset): no nanovdb_python_stub target
    defined; `make nanovdb_python_stub` errors with "no rule". Matches
    desired CI behavior.
  - SKBUILD=ON config: stub target exists, builds, emits nanovdb.pyi
    and py.typed into the install layout (28 kB stub file with all the
    expected GridHandle/GridType/GridClass symbols).

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: Phase 1 — polymorphic Grid, GridMetaData, blind data, PointAccessor (#2210)

* nanovdb python: Phase 1 — polymorphic Grid, GridMetaData, blind data, PointAccessor

Second slice of the Python bindings restructure tracked in #2208 and
laid out in nanovdb-python-plan.md. Phase 1 bundles the three sub-todos
(1a polymorphic API, 1b type-erased introspection + blind data, 1c
PointAccessor + handle utilities) into a single change on top of the
Phase 0 X-macro foundation.

API changes (pre-1.0 breaks called out in the plan):

- Polymorphic accessors. handle.grid(n=0) and handle.deviceGrid(n=0)
  replace the typed handle.floatGrid()/doubleGrid()/int32Grid()/
  vec3fGrid()/rgba8Grid() and their device equivalents. Dispatch is
  driven by gridType(n) through a switch generated from BuildTypes.def;
  unbound BuildTs route to None rather than throwing.

- Grid base class rename. The Python class previously bound as
  nanovdb.GridData is now nanovdb.Grid (matching the C++ user-facing
  class name Grid<TreeT>). All typed grid classes (FloatGrid, ...)
  inherit from Grid.

- Base-class method lift. version/gridSize/gridIndex/gridCount/voxelSize/
  map/gridType/gridClass/checksum/isLevelSet/isFogVolume/.../hasMinMax/
  hasBBox/.../isBreadthFirst/shortGridName move from per-BuildT
  defineNanoGrid<T> up to defineGrid via lambdas that read GridData data
  members directly. defineNanoGrid<T> now only binds getAccessor,
  activeVoxelCount, and isSequential — the BuildT-dependent slice.

Additive surface:

- GridMetaData. Bound as nanovdb.GridMetaData with constructor from a
  Grid and the full read-only accessor surface (gridType, gridClass,
  shortGridName, gridSize/Index/Count, map, worldBBox, indexBBox,
  voxelSize, blindDataCount, activeVoxelCount, activeTileCount(level),
  nodeCount(level), checksum, version, isValid, isLevelSet/...,
  hasMinMax/..., isBreadthFirst, rootTableSize, isEmpty). Type-erased
  introspector — answer "what's in this buffer?" without knowing BuildT.

- Blind data API on Grid. blindDataCount, blindMetaData(n),
  findBlindData(name), findBlindDataForSemantic(sem), getBlindData(n).
  getBlindData returns a zero-copy NumPy view typed by mDataType (Float
  -> 1D float32, Vec3f -> (N, 3) float32, RGBA8 -> (N, 4) uint8, etc.).
  Out-of-range and unknown-type paths return None / fall back to a flat
  uint8 byte view.

- Enums: GridBlindDataClass (Unknown/IndexArray/AttributeArray/GridName/
  ChannelArray/End) and GridBlindDataSemantic (Unknown/PointPosition/
  PointColor/PointNormal/PointRadius/PointVelocity/PointId/WorldCoords/
  GridCoords/VoxelCoords/LevelSet/FogVolume/Staggered/End). Bound as
  nb::enum_ with .export_values() so the names are also top-level
  attributes of the module.

- GridBlindMetaData struct. Read-only fields valueCount/valueSize/
  semantic/dataClass/dataType, name() accessor, isValid(),
  blindDataSize().

- PointAccessor variants. nanovdb.PointIndexAccessor (uint32 indices,
  used by PointIndex grids) and nanovdb.PointDataAccessor (Vec3f
  positions, used by PointData grids). Methods gridPoints(),
  leafPoints(ijk), voxelPoints(ijk) each return a zero-copy NumPy view
  onto the underlying blind-data buffer, anchored to the accessor
  lifetime via keep_alive.

- GridHandle utilities. handle.copy() does a deep copy into a freshly
  allocated buffer of the same buffer type. Module-scope splitGrids(h)
  -> list[GridHandle] and mergeGrids(handles) -> GridHandle are
  registered for both host and device handles via the existing
  defineGridHandleUtilities<BufferT> template (nanobind merges them as
  an overload set).

Mechanical X-macro changes:

- BuildTypes.def gains a GridTypeEnum column on each row so the
  polymorphic dispatch in pyHostGrid/pyDeviceGrid can `case
  nanovdb::GridType::<GridTypeEnum>:` on it. The Point row maps to
  GridType::PointIndex (there is no GridType::Point).
- HandleMethod/DeviceMethod columns dropped — the typed handle.fooGrid()
  accessors no longer exist.

NB_MODULE bind order:

- defineCheckMode + defineChecksum now bind BEFORE defineGrid because
  Grid.checksum() returns Checksum by value (registration must precede
  use).
- defineGridBlindData binds BEFORE defineGrid for the same reason
  (Grid.findBlindDataForSemantic / blindMetaData reference the new enum
  and class in their signatures).

Tests (TestNanoVDB.py): all typed-accessor call sites rewritten to
handle.grid(i)/handle.deviceGrid(i). New test classes cover the new
surface — TestPolymorphicGridAccess, TestGridBase, TestGridMetaData,
TestBlindDataEmpty, TestSplitMergeCopy — 11 new tests; the full suite
is now 48 tests, 39 pass on a minimal CPU build (8 CUDA skip, 1
pre-existing test_read_write_grid BLOSC failure unrelated to this PR).

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: don't expose splitGrids/mergeGrids for DeviceGridHandle

Caught during a real-CUDA verification pass of #2210: calling
nanovdb.mergeGrids([device_h1, device_h2]) raised std::bad_cast on
sm_120 (Blackwell, CUDA 13.2). Both the host and device overloads of
splitGrids/mergeGrids take nb::list, so nanobind's overload resolution
can't disambiguate by element type — it picks the first match and the
inner nb::cast<HostHandle&&>(device_h) fails.

The host-only variant is what the Phase 1 plan calls for. A properly
typed device variant (with its own name, or strongly-typed
std::vector<HandleT> args via nanobind/stl/vector.h) can land later if
it's actually needed. handle.copy() on a DeviceGridHandle continues to
work because copy() is a regular method, no overload resolution
involved.

Full CUDA pytest now reports 46/48 (the 2 failures are the pre-existing
test_read_write_grid BLOSC bug, host + device variants — both call
writeGrid(..., Codec.BLOSC) without try/except, identical to master).

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: address Copilot review on #2210

Three concrete fixes from Copilot's review of the Phase 1 PR
(https://github.com/AcademySoftwareFoundation/openvdb/pull/2210):

1) mergeGrids no longer consumes its input handles.

   The previous implementation built a std::vector<HandleT> via
   nb::cast<HandleT&&>(h), which move-constructs the C++ GridHandle
   out of the Python wrapper — leaving caller's h1/h2 silently
   emptied (gridCount went 1 -> 0, size went non-zero -> 0).
   Reproduced before fix; locked into a regression test
   (TestSplitMergeCopy.test_merge_does_not_consume_inputs).

   Rewrote the binding to read each handle by const reference and
   inline the merge concat directly. The nanovdb::mergeGrids C++
   helper's signature requires a std::vector<GridHandle> (a
   move-only type), so reusing it from Python without moving from
   the inputs would have meant deep-copying each handle twice; the
   inlined version is ~15 lines and does one memcpy per source grid
   with tools::updateGridCount fixing up the per-grid header.

2) getBlindData validates mValueSize against the implied dtype/shape
   before building a typed NumPy view.

   A blind-data channel with mDataType=Float but mValueSize != 4
   (corruption, version mismatch, or an unknown variant of a known
   tag) would previously be exposed as `count` float32 elements —
   i.e. count*4 bytes — even though the underlying region is only
   count*mValueSize bytes. That overruns the channel and returns a
   view onto unrelated bytes.

   Added a `valueSize == sizeof(...)` (or `dim*sizeof(scalar)` for
   vector cases) check on every handled GridType. On mismatch the
   binding falls back to a raw uint8 byte view of mValueCount *
   mValueSize, which is by definition the actual byte extent and
   therefore always safe.

3) GridMetaData ctor + safeCast guard against invalid grids before
   calling into NanoVDB, where NANOVDB_ASSERT(gridData->isValid())
   would abort debug builds and undefined-behave in release.

   nanobind's type system already rejects Python None at the bind-
   site (None can't bind to const GridData*), so the literal
   "GridMetaData(None)" case Copilot called out is a TypeError
   today — but the broader concern (an otherwise-valid Grid object
   wrapping a corrupted buffer) is real.

   __init__ now does an explicit `gd == nullptr || !gd->isValid()`
   check and raises nb::value_error with a descriptive message
   before calling into nanovdb::GridMetaData. safeCast does the
   same and returns False on bad input, matching the spirit of
   "is this safe to cast?".

   New tests: TestGridMetaDataGuards covers the rejection paths and
   the still-works happy path.

Build + test verified locally on both CPU (52 tests, 43 pass, 8 CUDA
skip, 1 pre-existing BLOSC) and full CUDA (52 tests, 50 pass, 2 pre-
existing BLOSC host+device).

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: Phase 2 — broaden BuildT coverage to 24 grid types (#2211)

* nanovdb python: Phase 2 — broaden BuildT coverage to 24 grid types

Third slice of the Python bindings restructure tracked in #2208 and
laid out in nanovdb-python-plan.md. Phase 2 expands the bound BuildT
list from the six surfaced in Phase 0/1 (float, double, int32_t, Vec3f,
Rgba8, Point) to seventeen new types, riding the Phase 0 X-macro so
each addition is a single row in BuildTypes.def plus a polymorphic
dispatch arm.

New BuildTs by category:

- SCALAR (+4): int16_t, int64_t, uint8_t, uint32_t.
  Bound exactly like the existing float/double/int32_t — full
  defineScalarAccessor (with setVoxel) plus defineNodeInfo.
  Class names follow the GridType enum: Int16Grid, Int64Grid,
  UInt8Grid, UInt32Grid.

- VECTOR (+5): Vec3d, Vec4f, Vec4d, Vec3u8, Vec3u16.
  Same shape as the existing Vec3f / Rgba8 — defineVectorAccessor
  with setVoxel, no NodeInfo. Accessor names use the consistent
  "<Suffix>ReadAccessor" form (Vec3dReadAccessor, ...); only the
  legacy Vec3fReadVectorAccessor / RGBA8ReadAccessor names from
  Phase 0 stay as-is.

- READONLY (new category, +8): bool, Fp4, Fp8, Fp16, FpN,
  ValueIndex, ValueOnIndex, ValueMask. These all have
  nanovdb::BuildTraits<T>::is_special == true, which means the C++
  SetVoxel<T> specialization static_asserts and won't compile.
  They get a bare defineAccessor<T> binding — getValue() only,
  no setVoxel, no NodeInfo. Class names: BooleanGrid, Fp4Grid /
  Fp8Grid / Fp16Grid / FpNGrid (quantized — getValue returns
  float), IndexGrid / OnIndexGrid (getValue returns uint64), and
  MaskGrid (getValue returns bool).

The accessor's value type now resolves through
nanovdb::BuildToValueMap<BuildT>::Type rather than
DefaultReadAccessor<BuildT>::ValueType. For ordinary types they're
identical, but for Half / Fp* the accessor decodes to float on read,
for ValueIndex / OnIndex it returns uint64, and for ValueMask / bool
it returns bool — the bound probeValue() out-parameter and the
Python-side return type both want the decoded form. (Without this
change, probeValue<Half>'s instantiation chain mismatches its own
ProbeValue<Half>::ValueT = float.)

nanovdb::Half is intentionally NOT bound. The source declares it as
`class Half{};` (an empty placeholder for IEEE 754 half-precision)
and the C++ ProbeValue<Half> chain is inconsistent — leaf storage
carries Half but ProbeValue expects float, so the template doesn't
instantiate. When the upstream Half implementation lands we can add
it via the same X-macro path.

Polymorphic dispatch in pyHostGrid / pyDeviceGrid gains a fourth
arm (NANOVDB_PY_FOR_EACH_READONLY_BUILDT) covering all eight new
GridType enumerators (Boolean, Fp4/Fp8/Fp16/FpN, Index, OnIndex,
Mask). handle.grid(n) / handle.deviceGrid(n) now return the right
typed subclass for these too.

Tests: a single new TestPhase2BuildTCoverage class with 5 methods
verifies every new BuildT registered, every accessor surface matches
its category (scalars have setVoxel + getNodeInfo; vectors have
setVoxel; read-only have neither), and all the new typed grids
inherit from the polymorphic Grid base. We can't host-construct
Int16Grid / Fp4Grid / IndexGrid / etc. yet because the C++
create*Grid factories for those types land in Phase 5 — but the
registration and the dispatch surface are locked in.

Build + test verified locally:
- CPU (no CUDA, no OpenVDB, no BLOSC): 57 tests, 48 pass, 8 CUDA
  skip, 1 pre-existing test_read_write_grid BLOSC failure unrelated
  to this PR.
- CUDA sm_120 (Blackwell, CUDA 13.2): 57 tests, 55 pass, 2 pre-
  existing BLOSC failures (host + device variants).

No new lines over 100 cols.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: address Copilot review on #2211

Three concrete fixes from Copilot's review of the Phase 2 PR
(https://github.com/AcademySoftwareFoundation/openvdb/pull/2211):

1) Bind GridType.UInt8 in the Python enum.

   Real bug — a Phase 0 oversight that was harmless until Phase 2
   surfaced UInt8Grid. The C++ enumerator nanovdb::GridType::UInt8
   existed (value 26), and the polymorphic dispatch routes it
   correctly internally, but the enum binding was missing the
   `.value("UInt8", GridType::UInt8)` line. Python users therefore
   couldn't write `handle.gridType(n) == nanovdb.GridType.UInt8` to
   discriminate UInt8 grids.

   New regression test TestPhase2BuildTCoverage.
   test_all_grid_type_enums_reachable walks every BuildT we bind
   and confirms its GridType enumerator is reachable from Python,
   so the next oversight gets caught at test time.

2) Update pyHostGrid() docstring in PyGridHandle.h. Was claiming
   "Boolean, Half, Fp16 land in Phase 2" as examples of types that
   weren't yet Python-visible — those examples are now stale (Phase
   2 binds Boolean and Fp16; Half stays unbound but for a different
   reason). Reworded to point at BuildTypes.def as the source of
   truth so the docstring can't go stale again as Phase 5+ adds
   more types.

3) Disambiguate "bool for ValueMask and bool" wording in the
   READONLY macro doc comment in BuildTypes.def. The second "bool"
   referred to the literal BuildT=bool grid; the phrasing read as
   redundant. Now: "bool for ValueMask and for the BuildT=bool
   (Boolean) grid". Same fix also enumerates the quantized types
   explicitly (Fp4/Fp8/Fp16/FpN) instead of writing "Fp*".

Build + test on CUDA sm_120 with BLOSC + ZLIB: 58/58 pass.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: Phase 3 — tree / nodes / NodeManager / leaf_values (#2212)

* nanovdb python: Phase 3 — tree / nodes / NodeManager / leaf_values

Fourth slice of the Python bindings restructure tracked in #2208 and
laid out in nanovdb-python-plan.md. Phase 3 makes the tree itself
walkable from Python: every BuildT now has a bound NanoTree, Root,
Upper / Lower internal node, Leaf, and a host-side NodeManager. Where
the leaf actually carries a contiguous T mValues[512] (regular
scalar BuildTs) we also expose zero-copy NumPy views, including a
high-level grid.leaf_values() bulk extractor.

Per BuildT we now register six new classes:

- <Suffix>Leaf — origin, bbox, dim, voxelCount, memUsage, flags,
  isActive(ijk|n), getValue(offset|ijk), getFirstValue, getLastValue,
  minimum / maximum / average / stdDeviation, valueMask, probeValue,
  and (for arithmetic non-special ValueTs) values() returning a
  zero-copy (512,) NumPy view of the leaf's mValues array.

- <Suffix>Upper, <Suffix>Lower — origin, bbox, dim, memUsage,
  minimum / maximum / average / stdDeviation, valueMask, childMask,
  getValue, getFirstValue, getLastValue, isActive, probeValue. The
  two internal node levels share a single defineInternalNodeBase
  helper since their C++ APIs are identical.

- <Suffix>Root — background, tileCount, getTableSize, isEmpty, bbox,
  minimum / maximum / average / stdDeviation, memUsage, getValue,
  isActive, probeValue.

- <Suffix>Tree — root, background, activeVoxelCount,
  activeTileCount(level), nodeCount(level), totalNodeCount,
  memUsage, getValue, isActive, probeValue, extrema() (returns
  (min, max) tuple), getFirstLeaf / getFirstLower / getFirstUpper.
  Grid.tree() is bound on NanoGrid<T> and returns the typed tree
  as reference_internal.

- <Suffix>NodeManager — isLinear, memUsage, nodeCount(level),
  leafCount, lowerCount, upperCount, leaf(i), lower(i), upper(i)
  returning typed node refs. Constructed via the module-scope
  nanovdb.createNodeManager(grid) which polymorphically picks the
  right BuildT and returns a NodeManagerHandle. Handle exposes
  size(), __bool__(), and mgr() — which itself dispatches by stored
  gridType to return the right typed NodeManager.

- grid.leaf_values() bulk extractor — for arithmetic non-special
  BuildTs (float, double, Int16/32/64, UInt8/UInt32), walks the
  breadth-first leaf array and returns a strided zero-copy
  (N_leaves, 512) NumPy view. Stride between leaves is sizeof(LeafT)
  / sizeof(ValueT), reflecting the leaf header between value blocks.
  Throws ValueError on non-breadth-first grids (createNanoGrid
  produces breadth-first by default).

Mechanical bits:

- All six bindings driven from BuildTypes.def via the existing
  X-macro. New file PyTree.h holds the templated definitions; PyTree.cc
  holds the non-templated NodeManagerHandle + createNodeManager bindings
  (the latter dispatches over every BuildT via the same X-macro).
- Tree / node classes are registered BEFORE defineNanoGrid because
  NanoGrid<T>.tree() returns NanoTree<T>& and nanobind needs the
  return type registered first.

Skipped / deferred:

- LeafT::variance() is NOT bound. NanoVDB.h line 4388 reads
  `Pow2(DataType::getDev())` unqualified, which fails ADL for
  non-float ValueTs (ValueIndex / ValueMask). Same for InternalNode.
  Users can compute variance from stdDeviation() in Python.
- VoxelBlockManager (OnIndexGrid-specific) deferred to a follow-up;
  surface is sizeable enough to merit its own PR.
- Vector leaf values() (Vec3f / Vec3d / Vec4f / Vec4d / Vec3u8 /
  Vec3u16 / Rgba8) deferred — these need a flattened
  (count, dim) component view since nanobind ndarray<Vec3f> isn't
  well-formed. A future PR can add float[N, 512, 3] views.
- Tree iterators (beginValueOn etc.) deferred — the bulk leaf_values
  view + per-leaf valueMask covers the most common use case (mask
  the bulk array and you have your active values).

Verified locally (BLOSC + ZLIB on so the optional-codec tests pass):
- CPU build: 58 tests + 8 new TestPhase3TreeNodes — **all 66 pass**.
- CUDA sm_120 build (RTX PRO 6000 Blackwell, CUDA 13.2):
  **all 66 pass**.
- numpy-backed shape/dtype assertions in the new tests confirm the
  per-leaf values() (512,) float32 and bulk leaf_values()
  (N_leaves, 512) float32 views. NodeManager.leaf(i).values() is
  byte-identical to tree.getFirstLeaf().values() for i==0.

No new lines over 100 cols.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: drop phase numbers + reviewer markers from tests and comments

User-facing strings in tests/comments that reference in-flight project
history (phase numbers, reviewer names) lose their meaning once the
project merges. Drop them and reword the surrounding text so each name
and comment self-describes the feature being tested.

- Renamed TestPhase2BuildTCoverage -> TestBuildTRegistrations.
- Renamed TestPhase3TreeNodes -> TestTreeNodeWalking.
- Reworded docstrings on TestPolymorphicGridAccess, TestGridBase,
  TestGridMetaData, TestBlindDataEmpty, TestSplitMergeCopy,
  TestGridMetaDataGuards to describe the feature instead of the phase.
- Inline comments referencing "Phase 0", "Phase 1a.3", "Copilot review",
  etc. reworded into prose about the actual behavior being asserted or
  the underlying bug being regression-tested.
- Same cleanup on the cuda/PyDeviceGridHandle.cu note about why we don't
  register splitGrids/mergeGrids on DeviceBuffer.

No behavior change. 66/66 tests still pass with BLOSC+ZLIB on.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: fix Clang Tree.nodeCount overload + drop exception-driven createNodeManager dispatch

Two real issues caught in CI / review on #2212:

1) Tree.nodeCount(int) binding doesn't compile on Clang.

   PyTree.h used `nb::overload_cast<int>(&TreeT::nodeCount, nb::const_)`
   to select the non-templated `uint32_t nodeCount(int) const` overload
   on nanovdb::Tree<RootT>. Tree<RootT> also has a templated
   `template<typename NodeT> uint32_t nodeCount() const` overload.
   GCC accepted the overload_cast under SFINAE rules; Clang (used by
   the linux-nanovdb:cxx:clang++-Debug CI leg) rejects it with
   `no matching function for call to object of type
    'const detail::overload_cast_impl<int>'`, repeated once per
   BuildT instantiation.

   Replaced with a direct static_cast to the function pointer type,
   which is unambiguous to both compilers:

       static_cast<uint32_t (TreeT::*)(int) const>(&TreeT::nodeCount)

2) createNodeManager dispatch was exception-driven.

   The previous binding tried nb::cast<NanoGrid<T>&>(py_grid) for every
   BuildT and caught nb::cast_error on each mismatch — so a single call
   to nanovdb.createNodeManager(grid) threw and caught 22 cast_error
   exceptions before landing on the matching BuildT. Replaced with an
   nb::isinstance<GridT>(py_grid) pre-check so the cast is only ever
   attempted on the matching BuildT.

Both linux-nanovdb:cxx:clang++-Debug should now build, and
createNodeManager() no longer pays per-call exception overhead.
Verified locally on CPU and CUDA sm_120 builds: 66/66 tests pass with
BLOSC + ZLIB on.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: bounds-check + lifetime fixes on #2212

Addresses Copilot review notes on PR #2212. Two real classes of bug.

(1) Out-of-range arguments fell through into raw memory access.

Several entry points on Leaf, Tree, and NodeManager rely on
NANOVDB_ASSERT in the underlying C++ to catch invalid indices. That
assertion is a no-op in release builds, so passing an OOB index from
Python would read off the end of mValueMask / mValues / mNodeOffset[]
arrays. Wrapped each with an explicit range check that raises a
Python IndexError or ValueError:

  - Leaf.isActive(n)            n must be < voxelCount()  (512)
  - Leaf.getValue(offset)       offset must be < voxelCount()
  - Tree.activeTileCount(level) level must be 1, 2, or 3
  - Tree.nodeCount(level)       level must be 0, 1, or 2
  - NodeManager.nodeCount(L)    same as Tree
  - NodeManager.leaf(i)         i must be < leafCount()
  - NodeManager.lower(i)        i must be < lowerCount()
  - NodeManager.upper(i)        i must be < upperCount()

(2) Returned pointers / NumPy views did not actually keep their
backing buffers alive.

The pattern `nb::cast(value, nb::rv_policy::reference, parent)` was
used at multiple sites under the assumption that the third argument
established a Python-level keep_alive linkage between the returned
object and `parent`. It doesn't — rv_policy::reference is "no
ownership, no keep_alive" by definition; the parent argument is only
a hint to the cast machinery, not a lifetime guarantee.

As a result, expressions that drop the intermediate handle would
silently free the underlying buffer:

    g = nanovdb.tools.createFogVolumeSphere(name='probe').grid()
    g.gridName()   # SEGFAULT — handle was GC'd

    vals = (nanovdb.tools.createFogVolumeSphere()
            .grid().tree().getFirstLeaf().values())
    vals.sum()     # SEGFAULT

    nm = nanovdb.createNodeManager(
        nanovdb.tools.createFogVolumeSphere().grid()).mgr()
    nm.leaf(0)     # SEGFAULT — NodeManager holds raw ptr to grid

Added explicit `nb::keep_alive<0, 1>()` to the .def for every
affected site so the returned value keeps its parent alive:

  - GridHandle.grid(n)              (PyGridHandle.h)
  - DeviceGridHandle.deviceGrid(n)  (cuda/PyDeviceGridHandle.cu)
  - NodeManagerHandle.mgr()         (PyTree.cc)
  - Grid.getBlindData(n)            (NanoVDBModule.cc)
  - Leaf.values()                   (PyTree.h)
  - Grid.leaf_values()              (PyTree.h)
  - PointAccessor.gridPoints / leafPoints / voxelPoints
                                    (NanoVDBModule.cc)
  - nanovdb.createNodeManager(grid) keep arg 1 (grid) alive as long
    as the returned NodeManagerHandle lives — because the underlying
    NodeManager stores a raw pointer to the grid.

These were pre-existing bugs introduced in Phase 1 (pyHostGrid) and
inherited by every subsequent zero-copy view; Phase 3 surfaced more
of them via the new Tree/Leaf/NodeManager bindings.

Two new test classes lock the fixes in:

  - TestBoundsChecks asserts each guarded entry point raises the
    correct exception type at every boundary and still accepts in-
    range inputs.
  - TestZeroCopyViewLifetimes invokes each fixed binding in the
    chained-temporary form (handle.grid().tree().getFirstLeaf()
    .values(), createNodeManager(temp).mgr().leaf(0).values(), ...),
    runs gc.collect(), then touches the returned value. Pre-fix
    these segfaulted; now they pass cleanly.

CPU + CUDA sm_120 (BLOSC + ZLIB on): 75/75 pass.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: address Copilot follow-up review on #2212

Three further Copilot notes on the bounds-check/lifetime commit
(f6735717), all valid.

(1) PyTree.h relied on transitive includes for std::is_arithmetic_v /
    std::enable_if. Works on GCC + libstdc++ because <type_traits> is
    pulled in by <nanobind/nanobind.h> -> standard headers, but MSVC /
    libc++ may break the chain. Added an explicit
        #include <type_traits>
    near the top.

(2) grid.leaf_values() returned None for empty grids (nLeaves == 0 or
    getFirstLeaf() == nullptr) while its docstring promised an
    (N_leaves, 512) NumPy view. Callers had to special-case the None
    sentinel before iterating. Now returns an empty (0, 512) ndarray
    of the right dtype, so the contract reads cleanly:

        for row in grid.leaf_values(): ...

    works on every grid, empty or not. When nLeaves == 0 we pass a
    dummy non-null aligned pointer (the grid itself) to nb::ndarray so
    nanobind has a valid base for the empty array — no data is read
    since the leading shape is 0. Docstring updated to call out the
    empty-grid behavior explicitly.

(3) The ValueError raised on a non-breadth-first grid said
        "rebuild via tools::createNanoGrid(...)"
    which reads like a C++ symbol. Reworded to the Python-API form
        "rebuild via nanovdb.tools.createNanoGrid(...)"
    so Python users see a Python entry point.

New test method TestTreeNodeWalking.test_bulk_leaf_values_empty_grid_returns_empty_array
constructs a grid with an empty bbox (nLeaves == 0) and asserts
leaf_values() is an (0, 512) float32 NumPy array (not None).

CPU + CUDA sm_120 (BLOSC + ZLIB on): 76/76 pass.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: VoxelBlockManager (host) — Phase 3 follow-up (#2213)

* nanovdb python: VoxelBlockManager (host) — Phase 3 follow-up

Phase 3 deferred the VoxelBlockManager surface; this picks it up before
moving on to Phase 4. Adds host-side bindings for everything under
nanovdb::tools::VoxelBlockManager*, plus a minimal createOnIndexGrid
scaffold needed to build OnIndex grids from Python (the broader
createNanoGrid<SrcGridT, DstBuildT> binding lands in Phase 5).

New module surface, all under nanovdb.tools:

- VoxelBlockManagerHandle (host) — owns the firstLeafID + jumpMap
  metadata buffers; exposes blockCount(), firstOffset(), lastOffset(),
  reset(), __bool__. Buffers exposed as zero-copy NumPy views:
    firstLeafID() -> (blockCount,) uint32
    jumpMap(jump_map_length=1) -> (blockCount, jump_map_length) uint64
  jumpMap takes a jump_map_length argument because the value depends on
  log2_block_width (= 1 << (log2_block_width - 6)) and isn't stored on
  the handle. handle.decodeBlock(grid, i, log2_block_width=6) is a
  convenience method that slices firstLeafID / jumpMap for block i and
  calls decodeInverseMaps internally.

- buildVoxelBlockManager(grid, log2_block_width=6, first_offset=0,
  last_offset=0, n_blocks=0) — runtime switch over Log2BlockWidth ∈
  {6, 7, 8, 9} (BlockWidth 64/128/256/512) dispatching to the right
  template instantiation. Rejects non-OnIndex grids with TypeError;
  rejects out-of-range log2_block_width with ValueError.

- decodeInverseMaps(grid, first_leaf_id, jump_map, block_first_offset,
  log2_block_width=6) — free function. jump_map is a uint64 NumPy
  array of length JumpMapLength (= 1 << (log2_block_width - 6));
  returns (leaf_index, voxel_offset) freshly-allocated NumPy arrays
  (uint32, uint16) of length BlockWidth.

- createOnIndexGrid(src_grid, channels=0, include_stats=True,
  include_tiles=True, verbose=0) — minimal test-scaffold factory
  that binds tools::createNanoGrid<SrcGridT, ValueOnIndex>. Accepts
  FloatGrid, DoubleGrid, Int32Grid, Vec3fGrid sources. Required for
  end-to-end VBM testing since no other path produces an OnIndex
  grid from Python today. The full createNanoGrid<SrcGridT, DstBuildT>
  surface remains scoped to Phase 5.

Defensive checks:

- decodeBlock validates firstLeafID[block_index] is in [0, leafCount)
  before passing it into the C++ decodeInverseMaps. The underlying
  NanoVDB algorithm doesn't always initialize firstLeafID — blocks
  that no leaf's iteration sweep reaches (e.g. on tile-compressed
  OnIndex grids where some sequential offsets correspond to tile
  values rather than leaf voxels) are left with uninitialized memory.
  Without the guard, decodeInverseMaps would read
  tree.getFirstNode<0>()[garbage] and crash; the guard converts that
  into a Python ValueError with a clear message pointing at the
  workaround (build the source grid voxel-by-voxel via build::Grid
  so it stays uncompressed).

- All entry points reject out-of-range indices, levels, and grid
  build types up front (TypeError / ValueError / IndexError).

Tests under TestVoxelBlockManager exercise:
  - createOnIndexGrid produces an OnIndex / IndexGrid / sequential grid
  - Buffer shapes and dtypes (firstLeafID, jumpMap default + reshape)
  - decodeBlock(0) returns (uint32, uint16) arrays of length BlockWidth
  - decodeInverseMaps free function agrees with handle.decodeBlock
  - Out-of-range block_index raises IndexError
  - Out-of-range log2_block_width (5, 10) raises ValueError
  - Non-OnIndex grid argument raises TypeError
  - createOnIndexGrid(None) raises TypeError

End-to-end decode verification across every block is intentionally
deferred until Phase 4's build::Grid bindings land — that's the only
host-side path to construct a tile-free OnIndex grid where every
block's firstLeafID is reliably initialized by the current upstream
algorithm.

Verified locally:
- CPU build: 84 tests, 75 pass + 8 new VBM tests, 1 pre-existing
  test_read_write_grid BLOSC env-dependent failure unrelated to this PR.
- CUDA sm_120 + BLOSC + ZLIB: 84/84 pass.

No new lines over 100 cols.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: skip ndarray tests cleanly when numpy is missing

Windows CI surfaced three test errors on #2213 from
TestZeroCopyViewLifetimes — the test methods invoked
.values() / .leaf_values() directly without an `import numpy` guard,
so on environments without numpy installed (e.g. the Windows runner)
the bindings raised:

    TypeError: could not export nanobind::ndarray:
        ModuleNotFoundError: No module named 'numpy'

The other ndarray-touching tests (test_leaf_values_zero_copy,
test_bulk_leaf_values, test_node_manager_round_trip, the VBM
test_decode_block_zero, etc.) already guard with `import numpy`
and call self.skipTest. The three lifetime tests just skipped
that guard.

Added the same try/except ImportError + skipTest pattern to:
  - TestZeroCopyViewLifetimes.test_handle_grid_tree_leaf_values_chain
  - TestZeroCopyViewLifetimes.test_grid_leaf_values_temporary
  - TestZeroCopyViewLifetimes.test_node_manager_temporary_grid

While I was here, also dropped an unnecessary numpy guard on
TestVoxelBlockManager.test_create_on_index_grid_rejects_unsupported_source
— that test only checks TypeError on a non-grid input and doesn't
touch numpy, so it can run unconditionally.

Verified locally with both a numpy-enabled venv (84/84 pass minus the
pre-existing BLOSC env failure) and a numpy-less venv (84 ran, 17
skipped, no new errors).

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: install numpy on Windows CI so ndarray tests run

The Windows job for the NanoVDB workflow was missing the install_numpy
step that the main openvdb build.yml and weekly.yml workflows already
run after install_windows.ps1. As a result the vcpkg-supplied Python
on the Windows runner had no numpy, so every test that touches
nb::ndarray<nb::numpy, ...> either errored (before #2213's skip
guards) or now skips silently.

Add the install_numpy step using the existing
ci/install_windows_numpy.ps1 helper, matching the pattern already in
build.yml. The Windows runner will then actually execute the
ndarray-touching VoxelBlockManager / Tree / GridHandle tests instead
of skipping them.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: address Copilot review on #2213

Four issues raised by Copilot on the open VBM follow-up PR, all real:

1. VoxelBlockManagerHandle.jumpMap(jump_map_length) accepted any
   caller-supplied length and used it to shape a zero-copy NumPy view
   over hostJumpMap(). Since the underlying buffer was sized as
   blockCount * JumpMapLength (where JumpMapLength is derived from
   the log2_block_width the handle was BUILT with), a caller passing
   a larger value produced an ndarray whose elements lived past the
   end of the allocated buffer — an OOB read on access.

   Fix: wrap VoxelBlockManagerHandle in a small PyVBMHandle struct
   that also stores the log2_block_width. jumpMap() takes no
   arguments and derives JumpMapLength from the stored width, so the
   returned view always matches the buffer exactly. decodeBlock() no
   longer accepts a log2_block_width either, removing the equivalent
   mismatch hazard there. The new PyVBMHandle exposes
   log2_block_width / block_width / jump_map_length as read-only
   properties for introspection.

2. decodeInverseMaps() did not validate first_leaf_id against
   grid.tree().nodeCount(0). VoxelBlockManager::decodeInverseMaps
   indexes tree.getFirstNode<0>()[first_leaf_id] unconditionally, so
   a stray ID produced an OOB read of the leaf array. Validate up
   front and raise IndexError.

3. buildVoxelBlockManager() did not enforce grid.isSequential() (a
   precondition guarded only by NANOVDB_ASSERT, which is a no-op in
   release) nor that first_offset == 1 (mod BlockWidth). Validate
   both at the Python boundary and raise ValueError; the zero
   first_offset default still flows through unchanged because the
   C++ helper normalizes it to 1 itself.

4. The zero-copy shape/dtype test exercised
   vbm.jumpMap(jump_map_length=2) on a handle built with
   log2_block_width=6 — exactly the OOB case (1) was about. Rewrite
   the test to build a second handle with log2_block_width=7 and
   verify its jumpMap shape is (blockCount, 2), confirming the
   shape now follows the handle. Drop the obsolete
   log2_block_width=6 kwarg from decodeBlock. Add two new tests:
   misaligned first_offset is rejected, and an out-of-range
   first_leaf_id on decodeInverseMaps raises IndexError.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: empty-handle guards + accurate ownership comment

Two follow-up items from Copilot on #2213:

1. firstLeafID() and jumpMap() called hostFirstLeafID() / hostJumpMap()
   on the underlying VoxelBlockManagerHandle and fed the result straight
   into nb::ndarray, but both accessors legally return nullptr on a
   default-constructed or reset() handle (blockCount() == 0). Passing
   nullptr into nb::ndarray is unsafe even with a zero leading shape.
   Mirror the pattern PyTree.h uses for the empty-grid leaf_values()
   case: when the buffer is null, substitute a non-null dummy pointer
   (the handle itself) so nanobind has something to base the empty
   ndarray on; nothing is read because the leading shape is 0. Add
   tests for both default-constructed and reset() handles.

2. The pyDecodeInverseMapsImpl comment described nanobind as allocating
   "fresh memory through numpy", which doesn't match the implementation
   (which uses new[] + a capsule with delete[] as deleter). Rewrite the
   comment to describe the actual ownership model so future maintainers
   aren't misled.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: VBM exception-safety + firstLeafID sentinel prefill

Two more items from Copilot on #2213:

1. pyDecodeInverseMapsImpl allocated leafIndex / voxelOffset with raw
   new[] and only wrapped them in nb::capsule after several intervening
   operations (a second new[], the decodeInverseMaps call itself, and
   the first capsule's own construction). If anything in that window
   threw, the half-built state leaked. Hold the raw arrays in
   std::unique_ptr until each capsule has been constructed, then
   release() so ownership transfers cleanly; any throw during that
   sequence now unwinds without leaking.

2. The C++ allocating overload of buildVoxelBlockManager uses
   HostBuffer::create (uninitialized malloc) for firstLeafID, then
   touches only the slots its iteration sweep reaches. Blocks the
   algorithm doesn't visit retain arbitrary values; the existing
   decodeBlock guard (firstLeafID >= nLeaves) catches values past the
   leaf array but cannot tell garbage that happens to be < nLeaves
   from a real leaf id, so a low garbage byte would silently decode
   against the wrong leaf.

   Switch the Python binding to allocate the metadata buffers itself,
   prefill every firstLeafID slot with the sentinel value `nLeaves`,
   then call the in-place buildVoxelBlockManager(grid, handle)
   overload. The in-place builder zeros the jumpMap and only writes
   firstLeafID slots it visits, so every untouched slot keeps the
   sentinel and deterministically trips the decodeBlock guard.

   Add test_untouched_blocks_trip_sentinel_guard, which sweeps every
   block of the cube VBM and asserts each firstLeafID slot is either
   a real leaf id (< nLeaves) or exactly the sentinel — never any
   other value.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: VBM popcount upper-bound + n_blocks validation

Two more items from Copilot on #2213:

1. The decodeBlock guard only checked firstLeafID < nLeaves, but the
   C++ decodeInverseMaps loops leafID = firstLeafID ..
   firstLeafID + nExtraLeaves where nExtraLeaves is the popcount of
   this block's jumpMap (each set bit marks an additional leaf
   boundary crossed within the block). With a corrupt jumpMap or a
   handle paired with a different grid, the loop could read past
   tree.getFirstNode<0>() even with a valid firstLeafID.

   Hoist a popcount-based upper-bound check into
   pyDecodeInverseMapsImpl: compute nExtraLeaves locally and raise
   ValueError if firstLeafID + nExtraLeaves >= grid.tree().nodeCount(0).
   Because both the handle.decodeBlock() and the free-function
   decodeInverseMaps() funnel through this impl, both paths are
   covered without per-caller duplication.

2. buildVoxelBlockManager accepted an explicit n_blocks but didn't
   validate it against the documented precondition
   n_blocks >= ceil((last_offset - first_offset + 1) / BlockWidth).
   A smaller value produced a handle whose blockCount() < the
   coverage implied by lastOffset, silently truncating later
   decodeBlock sweeps. Validate when nonzero and raise ValueError
   with the minimum required value in the message.

   New test_build_voxel_block_manager_rejects_undersized_n_blocks
   covers the n_blocks=1 case on the cube VBM.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: explicit std headers in PyVoxelBlockManager.cc

Copilot noted PyVoxelBlockManager.cc uses std::integral_constant,
std::string/std::to_string, and std::move but relied on transitive
includes from nanobind / NanoVDB headers to bring those in. That works
on the toolchains we currently test but is fragile against stricter
ones.

Add explicit <string>, <type_traits>, and <utility> includes; drop
<cstring> and <stdexcept>, neither of which the translation unit
actually uses now.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: Phase 4a — tools.build.Grid<T> mutable CPU builder (#2214)

* nanovdb python: Phase 4a — tools.build.Grid<T> mutable CPU builder

Bind nanovdb::tools::build::Grid<BuildT>, its ValueAccessor<BuildT>, and
Tree<BuildT>::WriteAccessor under a new nanovdb.tools.build submodule.
One set of classes per writable BuildT in BuildTypes.def — every scalar
(float, double, int16, int32, int64, uint8, uint32) and every vector
(Vec3f, Vec3d, Vec4f, Vec4d, Vec3u8, Vec3u16, Rgba8). Naming mirrors
the C++ namespace: nanovdb.tools.build.FloatGrid is the mutable
counterpart of the read-only nanovdb.FloatGrid.

The binding exposes:

- Grid<T>(background, name='', gridClass=Unknown) — constructor
- getValue / setValue / setValueOn / isActive (the last two convenience-
  wrap a fresh ValueAccessor under the hood, since C++ has no
  setValueOn/isActive on Grid itself)
- nodeCount, gridType, gridClass, getName/setName, setTransform,
  .background property
- getAccessor() / getWriteAccessor() returning the typed Value /
  WriteAccessor proxies
- .to_nanovdb(sMode, cMode, verbose) — bakes the build grid into a
  host NanoGrid<BuildT> handle by calling tools::createNanoGrid

ValueAccessor exposes getValue/setValue/setValueOn/isActive/isValueOn.
WriteAccessor exposes setValue/setValueOn/merge. Both are wired up so
the parent grid is kept alive by Python while the accessor lives.

Implementation notes:

- WriteAccessor's defaulted move constructor leaves its internal
  ValueAccessor::mRoot reference dangling (the reference points into
  the moved-from WriteAccessor's own mRoot field, which is per-object
  state — not the parent Tree's mRoot). The Python binding bypasses
  the move path by heap-allocating via nb::rv_policy::take_ownership
  so the C++ object's address is stable for its entire lifetime.
- ValueAccessor doesn't have this hazard because its mRoot reference
  points at the parent Tree's mRoot (a stable address external to the
  accessor), so move construction is safe.
- Read-only special BuildTs (Boolean, Fp4/8/16/N, ValueIndex,
  ValueOnIndex, ValueMask) and Point are deliberately excluded — they
  have no SetValue<T> specialization and can't be built voxel-by-voxel.

Test coverage in new TestBuildGrid: constructor defaults, setValue
marks active, setValueOn preserves background, accessor parity with
grid, WriteAccessor merge-on-destruction, .to_nanovdb() round-trip
with metadata preserved, .to_nanovdb() doesn't consume the source,
Int32Grid + Vec3fGrid spot-checks, and setTransform propagation to
the baked grid's voxelSize / map.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: address Copilot review on #2214

Four items from the Phase 4a review, all valid:

1. Grid<T>.background property read self.mRoot.mBackground directly,
   touching an internal field even though RootNode exposes a
   background() accessor. Switch to self.mRoot.background() so the
   binding doesn't depend on the underlying field name staying put.

2. .to_nanovdb() can be an expensive bake for large grids but held
   the GIL the whole time. Add nb::call_guard<nb::gil_scoped_release>()
   so other Python threads can run during the conversion (the lambda
   only touches C++ state, no Python object handling).

3. nodeCount()'s docstring said the tuple was "internal node counts",
   but the first element is the leaf (level-0) count — leaves are not
   internal nodes. Reword to just "(leaf_count, lower_count,
   upper_count)" and drop the "internal" mislabel.

4. The WriteAccessor merge test was named "merges_on_destruction" but
   actually called wa.merge() explicitly and never forced the
   destructor to run. Split into two cases:
   - test_write_accessor_explicit_merge — calls .merge() explicitly,
     matching what the original test actually checked
   - test_write_accessor_merges_on_destruction — drops the only
     reference (del wa) and runs gc.collect() so the C++ destructor
     fires, then asserts the change is visible

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: use Vec3f.__eq__ in build::Vec3fGrid test

Copilot noted the component-by-component comparison in
test_vec3f_build_grid was justified by an out-of-date claim that
Vec3f equality isn't bound — it is (PyMath.cc defineVec3 wires
nb::self == nb::self). Use self.assertEqual(got, v) directly.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: Phase 4b + 4c — stats, validation, checksum (#2215)

* nanovdb python: Phase 4b + 4c — stats, validation, checksum

Round out Phase 4 by binding the GridStats, GridValidator, and
GridChecksum surfaces that didn't ship with Phase 4a.

Phase 4b — Stats
================

* Per-BuildT `tools.<Suffix>Extrema` and `tools.<Suffix>Stats` classes
  for every scalar and vector BuildT in `BuildTypes.def`. Extrema
  exposes `min` / `max` / `add(value)` and a truthy `bool()` for "has
  at least one sample"; Stats inherits from Extrema and adds `size`,
  `avg` / `mean`, `var` / `variance`, `std` / `stdDev`. Static
  predicates `hasMinMax` / `hasAverage` / `hasStdDeviation` /
  `hasStats` mirror the C++ trait queries.

* `tools.updateGridStats(grid, mode=Default)` — polymorphic dispatch
  via `callNanoGrid` over an `UpdateGridStatsOp`. Scalar and vector
  BuildTs route to `tools::updateGridStats<BuildT>`; the special /
  quantized / index / mask types (`Fp4/8/16/N`, `ValueIndex`,
  `ValueOnIndex`, `ValueMask`, `Point`) raise `ValueError` because
  `Stats<ValueT>` isn't meaningful for them. `bool` falls through to
  the C++ `NoopStats` arm.

* Per-BuildT `tools.getExtrema(grid, bbox)` returning the matching
  `Extrema`. Restricted to scalar + vector BuildTs (the only ones
  with an arithmetic ValueType).

Both `updateGridStats` and `getExtrema` release the GIL during the
traversal.

Phase 4c — Validation & checksum
================================

* `tools.validateGrid(handle, gridID, mode=Default, verbose=False)`
  for `GridHandle<HostBuffer>` — single-grid complement of the
  existing `validateGrids`. Returns `False` (without raising) when
  the gridID is out of range.

* `tools.checkGrid(grid, mode=Full)` — polymorphic via `callNanoGrid`.
  Returns `(ok, error_message)`. The 256-byte char buffer the C++
  helper writes into is hidden inside the binding so Python callers
  see a `(bool, str)` tuple.

* `tools.isValid(grid, mode=Default, verbose=False)` — polymorphic via
  `callNanoGrid<IsValidOp>`, equivalent to `checkGrid` + checksum
  verification rolled into a single bool.

* `tools.evalChecksum(grid, mode=Default)` and
  `tools.validateChecksum(grid, mode=Default)` — accept any bound
  NanoGrid via the existing `GridData` Python upcast (every
  `NanoGrid<T>` Python class is registered with `GridData` as its
  base, so nanobind handles the dispatch transparently). All three
  release the GIL.

Test coverage in new `TestGridStats`, `TestGridValidate`, and
`TestGridChecksum` (11 cases total): Extrema/Stats default-and-add,
polymorphic `updateGridStats` on a float grid, `updateGridStats`
rejection on an OnIndex grid, `getExtrema` over a sub-bbox,
`checkGrid` / `isValid` / `validateGrid` happy paths plus
out-of-range `gridID` and `CheckMode.Disable` short-circuit, and
`evalChecksum` → `updateChecksum` → `validateChecksum` round-trip.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: address Copilot review on #2215

Four items from the Phase 4b/4c review, all valid:

1. UpdateGridStatsOp rejected every BuildTraits<BuildT>::is_special
   type unconditionally (except bool), but tools::updateGridStats
   actually supports StatsMode::BBox on any ValueT via the NoopStats
   path. Drop directly into NoopStats for special BuildTs when mode
   is Disable or BBox; only MinMax / All raise (because those would
   instantiate Stats / Extrema over a non-arithmetic ValueT and the
   semantics are ill-defined). Update the binding docstring to
   describe the actual matrix.

2. The test_update_grid_stats_rejects_index_grid case was renamed to
   test_update_grid_stats_on_index_grid and extended to cover all
   four StatsMode arms on an OnIndexGrid: MinMax and All raise,
   BBox and Disable now succeed.

3. validateGrid's docstring didn't mention the CheckMode.Disable
   short-circuit (which returns True without inspecting the handle
   or gridID). The Python tests already exercised the short-circuit
   via test_validateGrid_disable_mode_always_true, so the behavior
   was correct — just the docstring was missing it.

4. IsValidOp::unknown ignored verbose=True and silently returned
   false, but the C++ callNanoGrid::unknown arm writes an
   "Unsupported GridType" message to std::cerr when verbose is set.
   Mirror that — pull in <iostream> in the binding TU and emit the
   same diagnostic from C++.

5. test_get_extrema_over_active_bbox used a bbox that exactly equals
   the root's active bbox, which triggers C++ getExtrema's
   "bbox contains root.bbox()" short-circuit that unconditionally
   folds the grid background into the extrema — so the min came
   back as 0.0 (background), not 1.0 (the smallest active voxel).
   The test passed but the semantics were muddy: it was really
   asserting "background gets added in this branch" rather than
   anything about getExtrema's bbox restriction. Replace with
   test_get_extrema_strictly_inside_active_region, which uses a
   bbox strictly inside the active region (so the recursive branch
   runs) and asserts min/max are exactly the smallest/largest
   sampled active values.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: bind validateGrid for device handles too

Copilot noted tools.validateGrids was registered for both
GridHandle<HostBuffer> and GridHandle<cuda::DeviceBuffer> (the
latter behind NANOVDB_USE_CUDA), but my Phase 4c addition of the
single-grid tools.validateGrid only covered HostBuffer — leaving
device handles able to validate the whole bundle but not an
individual grid.

Add the matching #ifdef NANOVDB_USE_CUDA overload binding
tools::validateGrid<GridHandle<cuda::DeviceBuffer>>. The C++
helper does host-side dispatch via callNanoGrid on the
host-resident gridData() pointer that DeviceGridHandle exposes, so
the same Python overload pair is appropriate.

New test_validateGrid_on_device_handle exercises the device path
end-to-end (build a CUDA level-set sphere, validate it, confirm
the same out-of-range and Disable-mode short-circuits as the host
overload). The test is gated on cuda module availability so it's
skipped cleanly on CPU-only builds.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: address more Copilot review on #2215

Two more items from the review, both valid:

1. CheckGridOp wrote tools::checkGrid's error message into a 256-byte
   stack buffer. tools::checkGrid (and its util::sprint / util::strcpy
   helpers) trust the caller's buffer is large enough — there is no
   bounds-checked variant. The current error messages are at most ~80
   characters once GridType and GridClass enumerator names are
   stringified, so 256 wasn't actually overflowing today, but the
   margin is uncomfortable and a future error string addition could
   push past it. Bump to a 4096-byte buffer (named via a constexpr
   kErrorBufSize) and zero-init the first byte before the call so the
   ok-detection still works if the helper bails out before writing
   anything.

2. updateGridStats's docstring claimed special (quantized / index /
   mask) BuildTs only accept Disable / BBox, but Boolean grids are
   special yet fall through the if-constexpr filter to the regular
   tools::updateGridStats path (which routes any mode to NoopStats
   for ValueT=bool inside the C++ helper). Reword the docstring to
   call out Boolean as the exception that accepts every StatsMode
   via the NoopStats internal path.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: Phase 5 — primitives + quantized/index createNanoGrid (#2216)

* nanovdb python: Phase 5 — primitives + quantized/index createNanoGrid

Closes out Phase 5 of the NanoVDB Python bindings restructure (#2208).
Lands all three sub-phases (5a primitives, 5b quantized createNanoGrid
overloads, 5c generic createNanoGrid from build::Grid) in one PR.

Phase 5a — Host primitives
==========================

Bind the nine primitives that didn't ship with Phase 0:

* tools.createLevelSetBox(gridType, width, height, depth, ...) — narrow-
  band level set of a solid axis-aligned box.
* tools.createLevelSetBBox(gridType, width, height, depth, thickness, ...) —
  narrow-band level set of a hollow box wireframe.
* tools.createLevelSetOctahedron(gridType, scale, ...) — narrow-band
  level set of an octahedron.
* tools.createFogVolumeBox(...) and tools.createFogVolumeOctahedron(...) —
  the fog-volume counterparts of the above.
* tools.createPointSphere / createPointTorus / createPointBox(gridType,
  pointsPerVoxel, ...) — PointDataGrids scattered on the surface of each
  primitive shape.
* tools.createPointScatter(srcGrid, pointsPerVoxel, ...) — scatter a
  PointDataGrid into the active voxels of a NanoGrid<float> level set or
  fog volume. The C++ template also accepts double sources; the binding
  is float-only for simplicity (the runtime grid pointer carries the
  source BuildT and adding a per-type dispatch is mechanical follow-up).

Each non-point primitive is instantiated for float and double via the
same runtime-GridType switch the existing createLevelSetSphere et al.
already use. The FpN overloads of these primitives are deliberately
not bound here — quantization is reachable via Phase 5b's generic
createNanoGrid* path with explicit oracle and dither parameters.

Phase 5b — Quantized createNanoGrid overloads
=============================================

* tools.AbsDiff(tolerance=-1.0) and tools.RelDiff(tolerance=-1.0) —
  compression-oracle classes used by FpN. Both expose getTolerance /
  setTolerance and a truthy __bool__ that returns True iff the
  tolerance has been initialized (>= 0). The default tolerance of -1
  matches the C++ "uninitialized — fill in via init()" sentinel.

* tools.createNanoGridFp4 / Fp8 / Fp16(src, sMode, cMode, ditherOn,
  verbose) — quantize a float source into a fixed-bit-width grid.
  ditherOn adds sub-quantum noise to break up banding.

* tools.createNanoGridFpN(src, oracle, sMode, cMode, ditherOn, verbose) —
  variable-bit-width quantization. Two overloads: one accepting an
  AbsDiff oracle (the default), one accepting a RelDiff oracle. Python
  picks the right overload from the oracle argument's type.

The C++ Fp{4,8,16,N} preProcess templates static_assert SrcValueT ==
float, so the binding rejects double sources with a TypeError instead
of letting the compile-time assertion trip a hard abort.

Phase 5c — Generic createNanoGrid from build::Grid
==================================================

Each conversion entry accepts both NanoGrid<SrcBuildT> and
tools::build::Grid<SrcBuildT> as its source. Internally a small
template helper tries each source-side BuildT in turn (matching the
existing tryCreateOnIndexGrid pattern from the Phase 3 follow-up) and
dispatches via nb::isinstance.

* tools.createNanoGridIndex(src, channels=0, includeStats=True,
  includeTiles=True, verbose=0) — NEW. Bake any supported source into
  a NanoGrid<ValueIndex> with all voxels (active and inactive) given a
  uint64 sequential index. Original values can be carried as blind
  data when channels > 0.

* tools.createNanoGridOnIndex(src, ...) — same as Index but only the
  active voxels get a sequential index. Supersedes the Phase 3
  follow-up's tools.createOnIndexGrid test scaffold (which is kept
  alive in PyVoxelBlockManager.cc for backwards compatibility with
  the VBM tests; new code should prefer createNanoGridOnIndex).

Source types accepted by the index path are NanoGrid<float | double |
int32 | Vec3f> and tools::build::Grid<float | double | int32 | Vec3f>.
The same-type bake of build::Grid<T> -> NanoGrid<T> remains served by
the Phase 4a .to_nanovdb() method.

Test plan
=========

Three new TestCase classes covering: every primitive (TestNewPrimitives,
11 cases), every quantization path including double-source rejection
and oracle defaults (TestCreateNanoGridQuantized, 8 cases), and every
index/onindex source type including the build::Grid path
(TestCreateNanoGridIndex, 6 cases). All 25 new cases pass locally; the
full pytest_nanovdb suite stays green except for the two pre-existing
BLOSC-disabled errors in my local build.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: address Copilot review on #2216

Four items from the Phase 5 review, plus a follow-on segfault fix
caught while addressing the third item:

1. openToNanoVDB binding accidentally dropped the sMode argument and
   gave `base` a default of tools::StatsMode::Default — nonsense both
   semantically (base is an OpenVDB GridBase::Ptr) and structurally
   (the C++ template expects 4 args, the binding declared 3 keyword
   args). The CI build broke for openvdb-enabled configs with a
   "number of nb::arg annotations must match the argument count"
   static_assert from nanobind. Restore the correct signature: base
   has no default, sMode/cMode/verbose carry their defaults.

2. Octahedron primitive default name strings carried the upstream C++
   typo "octadedron" (sic). The C++ side keeps the misspelling for
   binary compatibility with old grids; correct it on the Python
   side since the default propagates into help() output and grid
   metadata where end-users see it.

3. The point primitives took a `gridType` argument that was
   misleading — it controls the intermediate level-set's precision
   the scatter starts from, not the returned PointDataGrid's type
   (which is always UInt32). While investigating the rename, I found
   that the createPointSphere<double> / createPointTorus<double> /
   createPointBox<double> code paths actually segfault during scatter
   with the current C++ implementation (only the float path is
   exercised by the C++ unit tests). Take Copilot's "remove it and
   always use float" alternative: drop the argument entirely. The
   binding now always uses the float intermediate level-set.
   createPointSphere / Torus / Box no longer have the parameter;
   their docstrings explicitly note "always returns a UInt32
   PointDataGrid".

4. The defineCreateNanoGridConversions doc-comment claimed sources
   were accepted for "every scalar/vector BuildT in BuildTypes.def
   that tools::createNanoGrid supports", which was wishful — the
   implementation actually tries a narrower set per destination kind.
   Spell out the actual matrix: quantized Fp{4,8,16,N} paths accept
   float only (the C++ preProcess static-asserts SrcValueT == float);
   the createNanoGridIndex / OnIndex paths accept float / double /
   int32 / Vec3f. Adding more source types is a one-line extension
   of the explicit try-each-SrcBuildT chains.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: address more Copilot review on #2216 + CI GPU gating

Four items in this round, three from Copilot plus a CI failure caught
on the linux-clang++ runner.

1. test_validateGrid_on_device_handle was gated only on the cuda
   submodule being importable, but the linux-clang++ CI runner has a
   CUDA toolkit installed (so the submodule loads) but no GPU
   driver. The test then crashed inside the C++ helper with
   "CUDA error 35: CUDA driver version is insufficient for CUDA
   runtime version". Switch the gate to the project's existing pair
   of helpers — nanovdb.isCudaAvailable() (build-time CUDA support)
   AND nanovdb.isGpuAvailable() (runtime device probe) — matching
   how TestPointsToGrid / TestSignedFloodFill / TestSampleFromPoints
   already gate. The test skips cleanly on CPU-only runners and
   driverless CUDA runners alike.

2. The Phase 5b/5c conversion helpers (tryQuantizeFpX, tryQuantizeFpN,
   tryIndexify) ran the heavy tools::createNanoGrid traversal while
   holding the Python GIL. Restructure each helper so the GIL is
   held only for the isinstance / cast dispatch, then released
   around the conversion itself (the source data lives in stable
   C++ storage anchored by the Python wrapper passed in via py_src,
   so it's safe to read without the GIL). Other Python threads can
   now make progress during large-grid quantization / indexing.

3. AbsDiff's class docstring said "pass an explicit positive value"
   for the tolerance, but the C++ operator bool() actually treats
   any non-negative value (including 0.0) as initialized. Reword to
   match what the API actually does.

4. The createNanoGridIndex/OnIndex source-rejection test was named
   test_index_rejects_unsupported_source with a comment about
   "bool / Boolean grids" but really only exercised the None case.
   Split into two cases with single, accurate purposes:
   - test_index_rejects_none — passes None; matches neither the
     NanoGrid nor the build::Grid isinstance arms.
   - test_index_rejects_unsupported_buildt — builds a
     tools.build.Vec3dGrid (structurally valid but a BuildT outside
     the float/double/int32/Vec3f source set) and confirms the
     try-each-SrcBuildT chain falls through to a TypeError.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: correct createPointScatter docs + add fog-rejection test

Copilot noted the createPointScatter binding docstring claimed the
source could be a "level set or fog volume", but the C++ implementation
explicitly checks srcGrid.isLevelSet() at line 1679 of
tools/CreatePrimitives.h and throws std::runtime_error("Expected a
level set grid") otherwise. Match the actual behavior: the docstring
now says the source must satisfy srcGrid.isLevelSet() and that
non-level-set sources (e.g. fog volumes) raise RuntimeError.

Add test_create_point_scatter_rejects_fog_volume to lock in the
behavior (build a fog volume sphere, confirm createPointScatter raises
RuntimeError).

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: fully-qualify nanovdb.tools.build.* in 5b/5c docs

Copilot noted the Phase 5b/5c TypeError messages and per-function
docstrings referred to the mutable build grids as "tools.build.FloatGrid",
but the actual Python import path is "nanovdb.tools.build.FloatGrid".
Replace every occurrence so help() / autocomplete / error messages
point at a path users can actually import.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: examples + docstring coverage sweep (#2218)

* nanovdb python: Phase 6 — docs, migration guide, examples

Closes Phase 6 of the NanoVDB Python bindings restructure (#2208).
Ships:

* doc/nanovdb/PythonMigration.md — porting guide for users on the
  pre-Phase-1 typed-accessor API. Covers the polymorphic
  handle.grid(n) replacement of floatGrid() / doubleGrid() / etc.,
  the removal of the nanovdb.math.cuda submodule (sampleFromVoxels
  is now in nanovdb.tools.cuda), the GridHandle.__bool__ + enum
  __repr__ behavior changes, and a survey of every new surface that
  used to require dropping to C++ (GridMetaData, blind data,
  PointAccessor, tree/node walking, NodeManager, leaf_values,
  VoxelBlockManager, tools.build.Grid, stats, validation,
  primitives, quantized createNanoGrid). Closes with a mechanical
  before/after replacement table.

* doc/nanovdb/PythonAPI.md — narrative API reference for the final
  surface. One section per submodule (nanovdb root, math, io,
  tools, tools.build, tools.cuda, cuda) listing the bound classes
  and functions with one-line descriptions. Points readers at
  help(x) for full signatures.

* nanovdb/nanovdb/python/examples/ — five runnable .py scripts:
  - load_inspect.py: polymorphic handle.grid(n), GridMetaData
    introspection, mixed-type handle via mergeGrids.
  - build_grid.py: tools.build.FloatGrid setValue / ValueAccessor /
    WriteAccessor / .to_nanovdb() round-trip.
  - bulk_leaf_numpy.py: zero-copy (N_leaves, 512) NumPy view via
    grid.leaf_values(), with global-stats reduction and in-place
    mutation demonstrating the no-copy semantics.
  - quantize.py: createNanoGridFp{4,8,16} fixed-width + createNanoGridFpN
    with AbsDiff and RelDiff oracles. Prints the per-format size
    so users see the compression tradeoff at a glance.
  - validate.py: validateGrid / validateGrids / checkGrid / isValid
    plus evalChecksum / updateChecksum / validateChecksum.

* nanovdb/nanovdb/python/examples/README.md — index of the above
  with one-line summaries.

* nanovdb/nanovdb/Readme.md — added a "Python bindings" section
  linking the API reference, migration guide, and examples.

Each example is self-contained (builds its own input data,
prints a small stdout summary) and was smoke-tested locally
against the post-Phase-5 binding. The bulk_leaf_numpy example is
the only one requiring NumPy; it prints a friendly skip message
otherwise.

API reference docs via Sphinx / mkdocs are deferred to a follow-up;
the existing docstrings already make help() useful and the .pyi
type stubs (from Phase 0) cover IDE / type-checker integration.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: drop end-user docs, scrub refactor framing from examples

Two changes in response to review feedback:

* Remove doc/nanovdb/PythonAPI.md and doc/nanovdb/PythonMigration.md.
  End users don't need a separate narrative reference (help() and
  the .pyi stubs cover the same ground) and don't need a migration
  guide framed around the multi-phase refactor history.

* Rewrite the docstring blurb at the top of each example
  (load_inspect.py, build_grid.py, bulk_leaf_numpy.py, quantize.py,
  validate.py) so the prose describes what the example demonstrates
  about the current API, with no mention of which phase of the
  refactor added the feature. Same scrub on
  python/examples/README.md (drop the "bindings restructure" framing
  and the now-dead links to the deleted docs) and on
  nanovdb/nanovdb/Readme.md (drop the new "Python bindings" section
  header — just leave the single examples link alongside the existing
  Examples link).

Examples themselves remain unchanged in behavior; all five still
run cleanly against the current binding.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: docstring coverage sweep across the binding surface

Add a short prose docstring to every nanobind .def() and nb::class_
call that didn't already carry one. The audit dropped from 374
distinct undocumented call sites (3278 raw, counting per-BuildT
template duplicates) to 0.

Style:
* One short imperative sentence per method, focused on what it does
  in Python terms.
* For methods that exactly mirror a C++ member of the same name, a
  "See nanovdb::<Class>::<method> in NanoVDB.h." tail so users can
  find the canonical reference without us re-stating it.
* Class-level docstrings on every bound class (Grid, GridHandle,
  GridMetaData, Tree, Root, Upper, Lower, Leaf, NodeManager, the
  per-BuildT accessor subclasses, Map, Coord, BBox*, samplers,
  build::Grid + ValueAccessor + WriteAccessor, Extrema / Stats,
  oracle classes, Checksum, etc.).
* Two-to-three sentence blurbs on the entry-point classes that
  genuinely need orientation.

Files touched: every Py*.cc / Py*.h binding file under
nanovdb/nanovdb/python/ plus the three cuda/ binding files. No code
changes — only the trailing const char* docstring argument on .def
calls and the third arg of nb::class_. The auto-generated .pyi stub
output picks the new prose up automatically; help() shows the same
text at runtime.

Also drop the few remaining "Phase N" / "from Phase N follow-up"
references that survived in internal C++ comments in
PyCreateNanoGrid.cc, PyPrimitives.cc, and PyVoxelBlockManager.cc,
keeping the source consistent with the no-refactor-history policy
already applied to the public docstrings, tests, and examples.

Test plan: full pytest_nanovdb green locally (138/140; the two
errors are pre-existing BLOSC-disabled errors). Module compiles
cleanly with CUDA + nanobind stub generation enabled.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: address Copilot review on #2218

Two items from the review:

1. The Readme link to the Python examples directory used the wrong
   relative path. From nanovdb/nanovdb/Readme.md, the examples live
   at python/examples/ (sibling), not nanovdb/python/examples/
   (which would resolve to nanovdb/nanovdb/nanovdb/python/examples/).

2. The Map.applyMap / applyJacobian / applyInverseMap /
   applyInverseJacobian docstrings used the words "double precision"
   and "single precision" to describe the variant suffix. That
   wording is misleading: the F-suffixed variants use 32-bit math
   internally, the unsuffixed variants use 64-bit math, but BOTH
   return a vector whose dtype matches the input (Vec3f -> Vec3f,
   Vec3d -> Vec3d). Switching to "uses 64-bit math" / "uses 32-bit
   math" with an explicit "returns a vector of the same dtype as
   the input" tail makes the semantics unambiguous. Same fix on the
   Jacobian variants (which previously didn't mention precision at
   all on the 64-bit overloads, only on the 32-bit ones — also
   confusing).

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: clarify Grid.gridName docstring

Copilot caught that the Grid.gridName docstring described it as "the
in-header buffer; truncated if very long" — which is actually the
semantics of shortGridName(). gridName() reads the LONG-form name
from blind data when the HasLongGridName flag is set, falling back
to the in-header buffer otherwise. Reword the docstring to match
and explicitly point at shortGridName() for the truncated form.

(The other comment in the same review pass flagged the Examples
table in python/examples/README.md as having "double leading pipes"
but the actual file uses single-pipe GitHub-flavored markdown
syntax — verified via per-character inspection. The table renders
correctly on GitHub; no change needed there.)

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: clarify VBM offset and Root.bbox docstrings

Two more Copilot review items:

* VoxelBlockManagerHandle.firstOffset / lastOffset docstrings said
  "Linear leaf offset of the first/last block..." but these are
  sequential VOXEL indices, not leaf IDs. Switch the wording to
  "Sequential voxel index of the first/last active voxel covered
  by this handle" so users aren't confused into thinking the value
  is a leaf number.

* RootT.bbox docstring said "Index-space bounding box of every
  active tile" but RootNode::bbox() returns the bounding box of
  every active VALUE in the tree, not just the tile entries in the
  root table. Drop the misleading "tile" wording.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: remove dead code in bulk_leaf_numpy example

Copilot caught that bulk_leaf_numpy.py computed n_active_after via a
per-voxel sum over leaf.isActive() but never used the result — dead
code that linters would flag. Replace with leaf.getFirstValue(),
which is actually informative for what the surrounding comment is
trying to show (the values changed in place via the zero-copy
NumPy write) and prints in the same line.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: bounds-check GridHandle grid accessors; fix stale docstring

Addresses review feedback on PR #2219.

GridHandle::gridSize(n) and gridType(n) index mMetaData[n] without a
bounds check (the C++ API documents "assumed to be less than
gridCount()"), so calling handle.gridSize()/gridType()/gridData() with
an out-of-range n — including any n on an empty handle — was undefined
behaviour reachable straight from Python. Wrap all three in
bounds-checked lambdas that raise IndexError. gridData() is guarded too:
gridData(n) itself returns nullptr for bad n, but the binding passes
gridSize(n) alongside it, and that call is the one that reads
mMetaData[n] out of bounds.

Also refresh the createOnIndexGrid docstring: it claimed the broader
createNanoGrid<SrcGridT, DstBuildT> surface "lands in a later phase",
but tools.createNanoGridOnIndex already ships in this PR
(PyCreateNanoGrid.cc). Point users at the canonical binding instead.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: harden mergeGrids for empty input and use gridData(n)

Addresses review feedback on PR #2219.

Empty input: mergeGrids([]) — or a sequence of only-empty handles —
produced totalGrids == 0 and called BufferT::create(0). For HostBuffer
that yields a buffer whose data() is non-null over a zero-byte region,
so the GridHandle(buffer&&) ctor then reads a full GridData header out
of it (heap-overflow read; in practice an opaque "invalid host buffer"
throw). Return an empty handle up front when there's nothing to merge.

Per-grid source pointer: copy from h->gridData(n) — the authoritative
start pointer that applies mMetaData[n].offset — instead of walking a
raw data() pointer advanced by gridSize(n). The two are equivalent for
the current tightly-packed layout (offsets are a running sum of grid
sizes), but the accessor form doesn't bake in that assumption and drops
the manual pointer arithmetic.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: release the GIL around VoxelBlockManager compute kernels

Addresses review feedback on PR #2219.

buildVoxelBlockManager, decodeBlock, and decodeInverseMaps all run
non-trivial C++ (the build may parallelize internally via
util::forEach; the decode sweeps multiple leaves) while holding the
GIL, blocking unrelated Python threads.

Release the GIL around the pure-C++ kernels only, matching the
established pattern in PyCreateNanoGrid.cc (tryIndexify / tryQuantizeFpX):

- pyDecodeInverseMapsImpl (shared by decodeBlock and the free
  decodeInverseMaps): scope a gil_scoped_release around the
  VBM::decodeInverseMaps call. The surrounding grid cast, bounds checks
  (which throw Python exceptions), and the NumPy array / capsule / tuple
  construction stay under the GIL.
- buildVoxelBlockManager: scope a gil_scoped_release around the
  buildVoxelBlockManager<LBW> call only.

Deliberately not a blanket nb::call_guard<nb::gil_scoped_release>() as
suggested: these entry points take an nb::handle and call
castOnIndexGrid (isinstance/cast) plus allocate NumPy arrays inside the
function body, so dropping the GIL across the whole call would touch the
Python C API without the GIL. The scoped release covers exactly the
GIL-free compute.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: index-grid completeness, transform aliases, sampler gradients

Close the residual gaps between the Python bindings and the C++ host API:

- Bind Grid.valueCount() on Index/OnIndex grids and Grid.pointCount() on
  PointGrid, gated per BuildT like the C++ SFINAE.
- Add tools.CreateNanoGrid mirroring the C++ converter class, exposing
  addBlindData() so blind-data channels can be authored from Python and
  filled through the writable getBlindData() NumPy view. Authored
  channels are zero-filled (the C++ path leaves them uninitialized).
- Bind ChannelAccessor<ChannelT, IndexT> for Index/OnIndex x
  float/double/int32/Vec3f, plus a polymorphic createChannelAccessor()
  dispatching on the channel's recorded dataType.
- Add the worldToIndex/indexToWorld convenience names (Dir/Grad/F
  variants included) as aliases on the type-erased Grid base.
- Bind gradient() and zeroCrossing() on the host samplers for the
  orders/BuildTs where the C++ templates compile (trilinear gradient,
  trilinear+triquadratic zeroCrossing, floating-point only).
- Fix a pre-existing bug: getBlindData() and the PointAccessor views
  built ndarrays without the nb::numpy framework tag, so the def-site
  keep_alive raised 'could not create a weak reference' whenever a
  non-empty view was returned.
- Paper cuts: restore the empty error messages in the sphere/torus
  primitive dispatchers, drop the unused baseName parameter from
  defineStats, and bind Checksum.isEmpty/isHalf/isFull/mode.

TestNanoVDB.py grows 139 -> 161 cases, covering the new surface plus the
previously untested populated blind-data and channel-accessor paths.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* nanovdb python: port host C++ examples to Python, smoke-test all examples

Nine new scripts under python/examples/, each a Python equivalent of a
host-capable C++ example (or its host code path):

- io_roundtrip.py: ex_write_nanovdb_grids + ex_read_nanovdb_sphere(_accessor)
  as a self-contained write/read round trip, plus splitGrids and zero-copy
  point positions via getBlindData.
- make_funny_nanovdb.py: ex_make_funny_nanovdb via the functor-based
  createFloatGrid factory (reduced domain — one Python call per voxel).
- make_typed_grids.py: ex_make_typed_grids across seven tools.build grid
  types with polymorphic re-read.
- raytrace_level_set.py / raytrace_fog_volume.py: the host render paths,
  re-expressed with the bound samplers (sphere tracing + zeroCrossing +
  gradient shading) and accessor marching respectively; PGM output.
- collide_level_set.py: the host particle-collision path using
  worldToIndexF, narrow-band isActive, and sampler.gradient normals.
- index_grid_channels.py: host half of ex_index_grid_cuda extended with
  ChannelAccessor read-back and CreateNanoGrid blind-data authoring.
- node_manager.py: host half of ex_nodemanager_cuda.
- openvdb_interop.py: ex_openvdb_to_nanovdb(_accessor); self-skips unless
  built with NANOVDB_USE_OPENVDB and openvdb imports.

Not ported: the pool-buffer examples (HostBuffer pool API unbound), the
MagicaVoxel converter (external parser/asset), device-only examples, and
a dedicated PointAccessor example (host point primitives bake UInt32
PointData grids; Point{Index,Data}Accessor require GridType.PointIndex).

test/TestExamples.py smoke-runs all 14 example scripts in subprocesses;
wired as ctest pytest_nanovdb_examples (~5 s) next to pytest_nanovdb,
gated on NANOVDB_BUILD_PYTHON_UNITTESTS.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* fix example DLL lookup on Windows

  Run example smoke tests through a Windows-specific child launcher that
  registers dependent DLL directories before importing nanovdb. Keep the
  DLL directory handles alive while preserving subprocess isolation and
  normal script execution behavior.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* fix Windows example test startup

Register dependent DLL directories through a test-only sitecustomize module so example smoke tests can run as normal Python scripts. This avoids the native access violations caused by executing them through runpy while preserving subprocess isolation.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* diagnose Windows example crashes

Enable Python's fault handler and unbuffered output for example subprocesses so Windows access violations report the active Python frame and preserve output leading up to the crash.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* fix Windows example DLL search

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* add CUDA DLL path to Windows examples

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Rename to_nanovdb->toNanoVDB to match convention

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Added comments on nb::object returned sentinel return in `try*` functions

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Fix `toNanoVDB` renames that were not caught.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* NanoVDB: add non-owning mergeGrids overload and use it from Python

Address review feedback on the Python bindings, which had reimplemented
the grid-merge logic instead of reusing nanovdb::mergeGrids. The
existing overload takes a const std::vector<GridHandle>&, which cannot
be built from Python-owned handles without moving from them (GridHandle
is move-only, and moving would silently empty the caller's objects) or
deep-copying every input buffer.

- Add mergeGrids(const GridHandle<BufferT>* const* handles, size_t
  count, pool) to GridHandle.h, which borrows the inputs; the vector
  overload now forwards to it, so there is a single implementation.
- While consolidating, fold in two fixes the Python binding had
  accumulated: empty input returns an empty handle instead of
  constructing from BufferT::create(0) (whose zero-byte region the
  GridHandle constructor would read a header out of), and the source
  walk uses gridData(n) — which applies the per-grid offset — instead
  of assuming grids are contiguous from data(), so padded handles
  merge correctly.
- Shrink the Python mergeGrids binding to collecting pointers and
  calling the new overload; drop the now-unused <cstring> and
  GridChecksum.h includes.

Verified with a standalone C++ test of both overloads (inputs left
intact, grid index/count rewritten, empty input, split round-trip) and
by running TestNanoVDB.py before/after: identical results, with all
merge/split tests passing.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-08-18 10:33:20 -07:00
Mark HarrisandClaude Fable 5 0377b2202d NanoVDB: fix nvcc warnings and treat NVCC diagnostics as errors by default (#2280)
* NanoVDB: use constructor-free shared-memory storage in GridStats kernels

CUDA does not support dynamic initialization of function-scope __shared__
variables, so nvcc skips the StatsT/CoordBBox constructors in processLeaf
and processInternal and emits warning #20054-D -- twelve copies in every
TU that instantiates the kernels, propagated to downstream builds since
GridStats.cuh is a header. The skipped initialization was harmless
(every shared slot is fully assigned before it is read), but the noise
is not.

Wrap the scratch arrays in cub::Uninitialized, whose backing store is a
trivially-constructible word array, and bind array references to the
aliased element type. The reduction code is unchanged.

Verified warning-free (0 vs 12) and performance-neutral: per-kernel SASS
comparison on sm_89 shows identical instruction counts, registers,
shared-memory bytes, and zero spills for all 14 kernels; timing 200
iterations of updateGridStats on an 8M-voxel grid shows matched-round
deltas within +/-0.1% with alternating sign.

Closes #2279

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: make GridChecksum's header-defined CRC32 kernels static

nvcc ignores the inline qualifier on __global__ functions and emits
warning #20050-D for crc32SlicedKernel and crc32CombineKernel. The
inline was doing ODR duty for a non-template kernel defined in a
header, so removing it outright would invite multiple-definition link
errors; static provides internal linkage per TU instead, matching the
effect of the anonymous namespace used by GridStats.cuh. Both launch
sites live in the same header, so nothing needs external linkage.

Note these warnings are invisible in CMake builds: CMake passes the
CUDA toolkit include directory via -isystem, and the diagnostic is
attributed through the __global__ macro machinery in those system
headers, so it is suppressed. Plain nvcc consumers of the headers see
it. With this change, TestNanoVDB.cu -- which includes essentially
every CUDA tool header -- compiles warning-free even in the
warning-visible (non -isystem) configuration.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: treat NVCC diagnostics as errors by default

Add the CMake option NANOVDB_CUDA_WERROR (default ON, also implied by
OPENVDB_CXX_STRICT), which prepends --Werror=all-warnings to
CMAKE_CUDA_FLAGS so every nvcc front-end diagnostic fails the build of
the NanoVDB tests, tools and examples. This locks in the warning-free
state: the #177-D, #20050-D and #20054-D classes fixed recently cannot
silently reappear. The option only affects NanoVDB's own build tree,
not projects that merely include the headers, and can be disabled if a
newer CUDA toolkit introduces new diagnostics.

Verified locally at CMAKE_CUDA_ARCHITECTURES=80 (matching CI) in both
Release and Debug with unit tests, tools and examples enabled: zero
diagnostics, full builds succeed, 7/7 ctests pass on a GPU.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: fix Windows fallout from NVCC warnings-as-errors

The first Windows CI run under --Werror=all-warnings surfaced two
issues:

Remove an unreferenced local in DistributedPointsToGrid's stripe
setup. deviceCoords became dead when the cudaMemAdvise call on the
external coords was removed; only the MSVC-host front end reports it
(#177-D), but the variable is dead on every platform.

Suppress the EDG dll-interface diagnostics (#1394, #1388) for CUDA
sources on Windows. This is the NVCC analog of the /wd4251 and /wd4275
suppressions in OpenVDBCXX.cmake: it's not possible to use STL types
in DLL interfaces in a portable and reliable way, so these fire on
every include of the OpenVDB core headers from a .cu file (404 + 14
instances in the CI log). Uses the top-level --diag-suppress option;
the -Xcudafe --diag_suppress spelling does not reach the CUDA
front-end phase that emits these.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: fix remaining Windows warnings-as-errors fallout

Clearing the dll-interface diagnostics unblocked the Windows projects
to compile further, exposing two more pre-existing warning classes that
NANOVDB_CUDA_WERROR now promotes to errors:

Guard the _USE_MATH_DEFINES definition in the four examples that
define it. The Windows build also passes -D_USE_MATH_DEFINES on the
command line (as 1), so the bare #define (empty) triggered MSVC C4005
macro-redefinition warnings, which the flag turns into C2220 errors.

Suppress EDG #177 for the CUDA gtest targets on Windows. gtest's TEST
macro declares each test's static test_info_ member with an
unused-attribute on GCC/Clang but not under MSVC, so for tests defined
inside an anonymous namespace (TestBuffer.cu, TestMemoryResource.cu,
TestUtilCuda.cu) the front end proves it unreferenced and reports #177
-- 76 errors, all from the macro machinery, none actionable in our
code. TestNanoVDB.cu is unaffected because its tests are declared at
namespace scope. The suppression is per-target and Windows-only, so
#177 stays enforced for all library headers, tools and examples on
every platform, and for everything on Linux.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: guard _USE_MATH_DEFINES in the example headers and CXX sources

The previous commit guarded only the nanovdb.cu files that the first
Windows log reported; each of these examples defines the macro in up to
three files, and common.h is included by the .cu sources, so the C4005
redefinition warnings persisted. Every definition site in the examples
is now guarded, verified by a tree-wide sweep for unguarded instances.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 20:40:24 +12:00
Nick Avramoussis 5eb1ec0c4d Updated pendingchanges
Signed-off-by: Nick Avramoussis <4256455+Idclip@users.noreply.github.com>
2026-08-18 20:18:52 +12:00
Nick Avramoussis 157c413b73 Avoid hadds in pca stamp kernels with intermediate buffers
Signed-off-by: Nick Avramoussis <4256455+Idclip@users.noreply.github.com>
2026-08-18 20:18:45 +12:00
Nick Avramoussis 20615f1b31 Added ostream methods for vcl types and scalar/simd trait selector
Signed-off-by: Nick Avramoussis <4256455+Idclip@users.noreply.github.com>
2026-08-18 20:16:32 +12:00
Nick Avramoussis e88f2f832e Added pca code example to ellipsoid raster
Signed-off-by: Nick Avramoussis <4256455+Idclip@users.noreply.github.com>
2026-08-18 20:00:09 +12:00
Jonathan Swartz 63d1878f30 NanoVDB: expose ResourceT on the five TopologyBuilder consumers (CUDA) (#2273)
Part of #2232 (B5), following #2268 which gave
tools::cuda::TopologyBuilder an injectable-resource seam and #2269 which
gave one to MeshToGrid.

The five classes that instantiate TopologyBuilder -- DilateGrid,
MergeGrids, PruneGrid, RefineGrid and CoarsenGrid, all in
nanovdb/tools/cuda/ -- were left behind as unaffected callers: each held
a TopologyBuilder<BuildT> constructed as mBuilder(stream), binding
default_resource<DeviceResource>() with no injection path. Downstream,
fvdb-core needs to route these operators' scratch through PyTorch's
c10::cuda::CUDACachingAllocator, and cannot delete its forked headers
until the seam exists.

Each of the five headers receives the conversion established by
MeshToGrid:

  - the class template gains typename ResourceT =
    nanovdb::cuda::DeviceResource, with an is_async_resource guard
    static_assert first in the class body;
  - the builder member becomes TopologyBuilder<BuildT, ResourceT>
    mBuilder;
  - the constructor(s) gain a trailing defaulted ResourceT& resource
    argument, forwarded as mBuilder(stream, resource). Trailing position
    keeps every existing call source-compatible. MergeGrids takes the
    argument on both its N-ary and its binary-convenience constructor,
    the delegating one forwarding;
  - every out-of-class member definition moves to the two-parameter
    form, including the dual-template getHandle.

No nested-type hoisting is needed: TopologyBuilder::Data was hoisted to
TopologyBuilderData in #2268, and none of the five classes declares its
own nested struct. The change is additive in behavior -- ResourceT
defaults everywhere, so existing callers compile unchanged with
identical allocation behavior.

Left out of scope: each operator's mBuilder.mProcessedRoot and
TopologyBuilder's internal mData are still host+device dual-space
nanovdb::cuda::DeviceBuffer, and the grid handle's output buffer still
goes through BufferT::create. None of these is observed by the injected
resource, which is expected and is noted in the new tests' comments; the
dual-space buffers are Step 3's problem.

Adds one injected-resource test per operator to TestMemoryResource.cu
(DilateGrid_InjectedResourceSeam, MergeGrids_InjectedResourceSeam,
PruneGrid_InjectedResourceSeam, RefineGrid_InjectedResourceSeam,
CoarsenGrid_InjectedResourceSeam), mirroring the existing PointsToGrid
seam tests: build a small ValueOnIndex source grid, run the operator
with a CountingResource passed as the new trailing constructor argument,
synchronize, and assert allocs > 0 && allocs == deallocs. MergeGrids
merges two disjoint grids, PruneGrid supplies a single all-on Mask<3>
sidecar over a one-leaf grid, and CoarsenGrid spreads voxels over a
2x2x2 leaf block so coarsening is non-degenerate.

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-08-18 18:13:22 +12:00
Mark HarrisandClaude Fable 5 d422d856d8 NanoVDB: own PointsToGrid's device arrays with cuda::Buffer (CUDA) (#2270)
* NanoVDB: align cuda::Buffer member names with cuda::buffer

Name the free operation destroy, as cuda::buffer does, and add the
stream-taking overload it also provides. clear stays but only as a
transitional delegate, marked as such: it exists because
GridHandle::reset still calls it, and goes away with the legacy dual
buffers that cuda::Buffer replaces.

Rename setStream to set_stream. Member names cannot be aliased, so
matching the standard spelling is the whole reason the resource concept
kept allocate_async; the same argument applies here. Note in passing that
ours deliberately does not synchronize, matching cuda::buffer's
set_stream_unsynchronized rather than its set_stream, whose own
documentation and implementation disagree (NVIDIA/cccl#10649).

Add swap. The generic std::swap already does the right thing through the
move operations, but both std::vector and rmm::device_buffer provide one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: allocate TopologyBuilder scratch from an injected resource

Eleven of the builder's buffers are device-only -- nothing reads them on
the host -- yet they were cuda::DeviceBuffer, whose host pointer and
per-device array they never use. Move them to the single-space
cuda::Buffer and give the builder a resource parameter, so its scratch is
injectable like PointsToGrid's already is, and freed by scope rather than
by hand. mProcessedRoot and mData are read on the host and stay dual.

This is not a speedup: DeviceBuffer::init allocates host or device memory
but never both, and these buffers always passed a real device id, so no
cudaMallocHost was on this path to begin with. Measured dilate on 1k/20k/
200k points, and the difference is within run-to-run noise.

Hoist the Data struct out of the class. It does not depend on the
resource, and leaving it nested would give every ResourceT its own
incompatible type for the device functors to name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: add SyncFromAsync, and give MeshToGrid a resource seam

A stream-ordered resource must also model the synchronous concept, which
means writing four methods where two would do. The synchronous pair is
not a bare delegate -- memory from allocate must be usable on any stream
when it returns, so the null-stream allocation has to be synchronized
first -- and omitting that yields memory which satisfies the concept but
is not actually synchronous. Put it in one place rather than leaving each
author to rediscover it.

The two resources in TestMemoryResource are the first users, and were
already wrong in exactly that way: they provide only the async pair, so
they never modelled is_async_resource. TempPool duck-typed and never
checked, so nothing caught it.

MeshToGrid was the last builder allocating from a hard-wired
DeviceResource, through TempDevicePool. Give it a ResourceT parameter and
thread it into both its TopologyBuilder and its pool. As with Data in the
builder, BoxTrianglePair is hoisted out of the class: it does not depend
on the resource, and leaving it nested would give every ResourceT its own
incompatible type for the device functors to name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: add ResourceRef and route TempPool's scratch through cuda::Buffer

Buffer holds its resource by value, matching cuda::buffer -- whose model
this completes: in CCCL the ownership semantics are selected by what is
placed in the by-value slot, an owning any_resource or a borrowing
resource_ref. We adopted the slot without the borrowing type, so a
container like TempPool, whose contract is a non-owning pointer to a
possibly stateful resource, had no way to hold a Buffer without copying
that resource and stranding its state. ResourceRef is the missing piece:
a non-owning reference that is itself a resource, so copying the ref
shares the underlying instance. Its async methods exist only when R
models AsyncResource, so a ref over a synchronous resource does not
misreport its tier, and two refs compare equal exactly when they
reference the same resource.

TempPool now keeps its bytes in a Buffer<std::byte, ResourceRef<R>>:
same resource contract, same stream retention, same discard-on-growth
reallocation, but the block is freed by ownership rather than by hand.
The TempPool unit tests, which assert traffic against the caller's own
resource instance, pass unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: note cuda::Buffer and cuda::BufferView in pendingchanges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: express PointsToGrid's density search as a loop

The bisection search over voxel size was written as a backward goto,
which made the lifetimes of the buffers it retries over non-lexical.
Rewrite it as while(true) with continue on retry and break on
convergence; the six hand-written frees before the jump are unchanged.
The change is easiest to review with whitespace ignored, since the loop
body re-indents: git diff -w shows 27 changed lines.

d_keys and d_node_count carry results past the loop, so their
declarations move above it, as does the copy event, which was created
inside the retried region on every iteration but destroyed only once at
the end -- each retry leaked the previous handle.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: own PointsToGrid's device arrays with cuda::Buffer

Every device array PointsToGrid allocates is now owned by a
Buffer<T, ResourceRef<ResourceT>> borrowing the injected resource --
members where the pipeline frees them in a later member function than
the one that allocated them (countNodes allocates; processUpperNodes,
processLeafNodes, processPoints and processBBox release), locals where
the lifetime is contained. The raw pointers survive only as views: the
device-visible fields inside mData, and the working pointers the cub
and kernel calls take. Every owner event -- assignment, swap, destroy
-- immediately refreshes its view, and released views are nulled so a
stale use faults instead of reading a freed block. The index ping-pong
becomes a swap of owners across the member/local boundary, replacing
the bare pointer swap whose safety depended on nothing reading the
device copy of d_indx between the two uploads.

The density-search retry keeps its free-before-reallocate order via
explicit destroy calls, so peak device memory is unchanged. The
hand-matched byte sizes at every free site disappear, and the arrays
released one line before scope exit now just leave scope. One behavior
change worth naming: the too-many-points-per-leaf throw previously
leaked the reduction scratch; ownership now releases it during unwind.

Verified against the previous commit with a counting resource over
three shapes (bulk segmented-sort branch, serial per-tile branch, and
the bisection retry engaged): allocation count, free count, and total
bytes are identical. Full CUDA and memory-resource suites unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: free TopologyBuilder scratch on the caller's stream

Each release site sits in a function that receives the stream, so pass
it to destroy rather than relying on the retained stream matching --
they are the same on every current path, but the explicit form does not
depend on that staying true. Also drop the cudaGetDevice calls whose
result the Buffer conversion left unused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: borrow TopologyBuilder scratch through ResourceRef

The scratch buffers held their resource by value, so each of the eight
carried its own copy -- fine for the stateless default, wrong for a
stateful resource, whose accounting would be split across copies while
the caller's instance saw nothing. Borrow through ResourceRef instead,
the same reconciliation TempPool uses.

Assert the stream-ordered requirement directly in TopologyBuilder and
MeshToGrid so a synchronous-only resource fails with a diagnostic that
names the builder, not just the pool inside it. Note SyncFromAsync's
synchronize cost on its allocate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: guard PointsToGrid's copy event with a scope owner

The too-many-points-per-leaf throw unwinds past the event's manual
destroy, leaking the handle. Own it with a small guard so unwinding
releases it, consistent with the buffer ownership in this function.
Also assert the stream-ordered resource requirement on the class, so a
synchronous-only resource fails naming PointsToGrid rather than the
pool inside it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: assert the scratch alignment TopologyBuilder relies on

The byte scratch is reinterpreted as word-sized types, which is valid
for every resource whose DEFAULT_ALIGNMENT is at least word alignment --
all CUDA allocation paths give 256 -- but nothing said so. Assert it, so
a custom resource with a weaker guarantee fails at compile time instead
of misaligning on the device.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: terminate a bare cudaCheckError with a semicolon

Legal without one -- the macro expands to a braced block -- but every
other use in the file spells it as a statement.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: set_stream matches cuda::buffer's contract, not a divergence

cuda::buffer's set_stream is deliberately non-synchronizing -- its
documented synchronization is a stale note left behind when the
synchronization was removed -- so our non-synchronizing set_stream
matches it in both name and contract, and the comment claiming a
divergence was wrong.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: mark cuda::Buffer::clear deprecated in the documentation

Documentation-level only: the [[deprecated]] attribute would warn from
GridHandle::reset and NodeManager::reset, template members in our own
headers that must keep calling clear() until every buffer type provides
destroy() -- an unactionable diagnostic for callers of reset(), and a
build break under -Werror. The attribute lands when those callers
migrate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: exercise the concept-required members of the test resources

The trait checks detect allocate/deallocate through unevaluated contexts,
which never odr-use them, so nvcc warned that every synchronous pair and
stub member was declared but never referenced (#177-D). The attribute
route is closed -- nvcc's front end ignores [[maybe_unused]] for this
diagnostic -- so reference them the honest way: a test that verifies the
synchronous halves of the counting and stream-recording doubles behave
like their stream-ordered halves, and one that pins the trait probes'
stub behavior. The TU now builds with no warnings at all.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 10:32:20 -07:00
Nick Avramoussis 25715e5c18 Vectorized weights and covariance transfers in points::pca methods
Signed-off-by: Nick Avramoussis <4256455+Idclip@users.noreply.github.com>
2026-08-15 18:33:27 +12:00
Nick Avramoussis 6b05f50a67 Merge pull request #2254 from mmalecki/include-tbb-version
Fix builds with newer TBB versions
2026-08-15 18:29:40 +12:00
Mark HarrisandClaude Fable 5 ba9a2c0974 NanoVDB: SyncFromAsync, ResourceRef, and resource seams for MeshToGrid and TempPool (CUDA) (#2269)
* NanoVDB: align cuda::Buffer member names with cuda::buffer

Name the free operation destroy, as cuda::buffer does, and add the
stream-taking overload it also provides. clear stays but only as a
transitional delegate, marked as such: it exists because
GridHandle::reset still calls it, and goes away with the legacy dual
buffers that cuda::Buffer replaces.

Rename setStream to set_stream. Member names cannot be aliased, so
matching the standard spelling is the whole reason the resource concept
kept allocate_async; the same argument applies here. Note in passing that
ours deliberately does not synchronize, matching cuda::buffer's
set_stream_unsynchronized rather than its set_stream, whose own
documentation and implementation disagree (NVIDIA/cccl#10649).

Add swap. The generic std::swap already does the right thing through the
move operations, but both std::vector and rmm::device_buffer provide one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: allocate TopologyBuilder scratch from an injected resource

Eleven of the builder's buffers are device-only -- nothing reads them on
the host -- yet they were cuda::DeviceBuffer, whose host pointer and
per-device array they never use. Move them to the single-space
cuda::Buffer and give the builder a resource parameter, so its scratch is
injectable like PointsToGrid's already is, and freed by scope rather than
by hand. mProcessedRoot and mData are read on the host and stay dual.

This is not a speedup: DeviceBuffer::init allocates host or device memory
but never both, and these buffers always passed a real device id, so no
cudaMallocHost was on this path to begin with. Measured dilate on 1k/20k/
200k points, and the difference is within run-to-run noise.

Hoist the Data struct out of the class. It does not depend on the
resource, and leaving it nested would give every ResourceT its own
incompatible type for the device functors to name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: add SyncFromAsync, and give MeshToGrid a resource seam

A stream-ordered resource must also model the synchronous concept, which
means writing four methods where two would do. The synchronous pair is
not a bare delegate -- memory from allocate must be usable on any stream
when it returns, so the null-stream allocation has to be synchronized
first -- and omitting that yields memory which satisfies the concept but
is not actually synchronous. Put it in one place rather than leaving each
author to rediscover it.

The two resources in TestMemoryResource are the first users, and were
already wrong in exactly that way: they provide only the async pair, so
they never modelled is_async_resource. TempPool duck-typed and never
checked, so nothing caught it.

MeshToGrid was the last builder allocating from a hard-wired
DeviceResource, through TempDevicePool. Give it a ResourceT parameter and
thread it into both its TopologyBuilder and its pool. As with Data in the
builder, BoxTrianglePair is hoisted out of the class: it does not depend
on the resource, and leaving it nested would give every ResourceT its own
incompatible type for the device functors to name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: add ResourceRef and route TempPool's scratch through cuda::Buffer

Buffer holds its resource by value, matching cuda::buffer -- whose model
this completes: in CCCL the ownership semantics are selected by what is
placed in the by-value slot, an owning any_resource or a borrowing
resource_ref. We adopted the slot without the borrowing type, so a
container like TempPool, whose contract is a non-owning pointer to a
possibly stateful resource, had no way to hold a Buffer without copying
that resource and stranding its state. ResourceRef is the missing piece:
a non-owning reference that is itself a resource, so copying the ref
shares the underlying instance. Its async methods exist only when R
models AsyncResource, so a ref over a synchronous resource does not
misreport its tier, and two refs compare equal exactly when they
reference the same resource.

TempPool now keeps its bytes in a Buffer<std::byte, ResourceRef<R>>:
same resource contract, same stream retention, same discard-on-growth
reallocation, but the block is freed by ownership rather than by hand.
The TempPool unit tests, which assert traffic against the caller's own
resource instance, pass unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: note cuda::Buffer and cuda::BufferView in pendingchanges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: free TopologyBuilder scratch on the caller's stream

Each release site sits in a function that receives the stream, so pass
it to destroy rather than relying on the retained stream matching --
they are the same on every current path, but the explicit form does not
depend on that staying true. Also drop the cudaGetDevice calls whose
result the Buffer conversion left unused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: borrow TopologyBuilder scratch through ResourceRef

The scratch buffers held their resource by value, so each of the eight
carried its own copy -- fine for the stateless default, wrong for a
stateful resource, whose accounting would be split across copies while
the caller's instance saw nothing. Borrow through ResourceRef instead,
the same reconciliation TempPool uses.

Assert the stream-ordered requirement directly in TopologyBuilder and
MeshToGrid so a synchronous-only resource fails with a diagnostic that
names the builder, not just the pool inside it. Note SyncFromAsync's
synchronize cost on its allocate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: assert the scratch alignment TopologyBuilder relies on

The byte scratch is reinterpreted as word-sized types, which is valid
for every resource whose DEFAULT_ALIGNMENT is at least word alignment --
all CUDA allocation paths give 256 -- but nothing said so. Assert it, so
a custom resource with a weaker guarantee fails at compile time instead
of misaligning on the device.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: set_stream matches cuda::buffer's contract, not a divergence

cuda::buffer's set_stream is deliberately non-synchronizing -- its
documented synchronization is a stale note left behind when the
synchronization was removed -- so our non-synchronizing set_stream
matches it in both name and contract, and the comment claiming a
divergence was wrong.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: mark cuda::Buffer::clear deprecated in the documentation

Documentation-level only: the [[deprecated]] attribute would warn from
GridHandle::reset and NodeManager::reset, template members in our own
headers that must keep calling clear() until every buffer type provides
destroy() -- an unactionable diagnostic for callers of reset(), and a
build break under -Werror. The attribute lands when those callers
migrate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: exercise the concept-required members of the test resources

The trait checks detect allocate/deallocate through unevaluated contexts,
which never odr-use them, so nvcc warned that every synchronous pair and
stub member was declared but never referenced (#177-D). The attribute
route is closed -- nvcc's front end ignores [[maybe_unused]] for this
diagnostic -- so reference them the honest way: a test that verifies the
synchronous halves of the counting and stream-recording doubles behave
like their stream-ordered halves, and one that pins the trait probes'
stub behavior. The TU now builds with no warnings at all.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 10:31:12 -07:00
Matthew Cong 6717cca14a Improve load balancing in DistributedPointsToGrid (#2276)
* Balance the multi-GPU merge tree across every device

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Generalize merge to non power-of-two GPU counts

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Remove prefetches causing performance regressions

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
2026-08-12 14:46:03 -07:00
Mark HarrisandClaude Opus 4.8 e538a0646b NanoVDB: handle empty partitions in the distributed merge path search (#2248)
* NanoVDB: handle empty partitions in the distributed merge path search

mergePath's bounds checks compare against (keys1Count - 1) and
(keys2Count - 1) on size_t counts, so a zero count underflows to
SIZE_MAX and the guards never fire; the binary search then reads the
empty array (and past either array) and produces wrong or garbage
intervals. An empty partition is reachable from radixSortAsync's
ceil-split whenever a device receives no items, e.g. sorting a single
element across two devices.

Return the trivial split before the search: with an empty side, every
element up to the diagonal comes from the non-empty side.

New single-GPU test MergePathEmptyPartition covers the empty-right,
empty-left, both-empty, and non-empty control cases; the empty cases
fail deterministically without the fix (inverted split / underflow
garbage) and run clean under compute-sanitizer with it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: compute the empty-partition merge split on the host

An empty side needs no merge path search, so skip the kernel launch:
the trivial split (everything up to the diagonal comes from the
non-empty side) is computed on the host by mergePathTrivial at the
launch site. mergePath reverts to the unguarded search and documents
that both inputs must be non-empty.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-11 10:32:02 -07:00
Mark HarrisandClaude Fable 5 586dde77a0 NanoVDB: inject a resource into TopologyBuilder scratch, and align cuda::Buffer with cuda::buffer (CUDA) (#2268)
* NanoVDB: align cuda::Buffer member names with cuda::buffer

Name the free operation destroy, as cuda::buffer does, and add the
stream-taking overload it also provides. clear stays but only as a
transitional delegate, marked as such: it exists because
GridHandle::reset still calls it, and goes away with the legacy dual
buffers that cuda::Buffer replaces.

Rename setStream to set_stream. Member names cannot be aliased, so
matching the standard spelling is the whole reason the resource concept
kept allocate_async; the same argument applies here. Note in passing that
ours deliberately does not synchronize, matching cuda::buffer's
set_stream_unsynchronized rather than its set_stream, whose own
documentation and implementation disagree (NVIDIA/cccl#10649).

Add swap. The generic std::swap already does the right thing through the
move operations, but both std::vector and rmm::device_buffer provide one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: allocate TopologyBuilder scratch from an injected resource

Eleven of the builder's buffers are device-only -- nothing reads them on
the host -- yet they were cuda::DeviceBuffer, whose host pointer and
per-device array they never use. Move them to the single-space
cuda::Buffer and give the builder a resource parameter, so its scratch is
injectable like PointsToGrid's already is, and freed by scope rather than
by hand. mProcessedRoot and mData are read on the host and stay dual.

This is not a speedup: DeviceBuffer::init allocates host or device memory
but never both, and these buffers always passed a real device id, so no
cudaMallocHost was on this path to begin with. Measured dilate on 1k/20k/
200k points, and the difference is within run-to-run noise.

Hoist the Data struct out of the class. It does not depend on the
resource, and leaving it nested would give every ResourceT its own
incompatible type for the device functors to name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: note cuda::Buffer and cuda::BufferView in pendingchanges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: free TopologyBuilder scratch on the caller's stream

Each release site sits in a function that receives the stream, so pass
it to destroy rather than relying on the retained stream matching --
they are the same on every current path, but the explicit form does not
depend on that staying true. Also drop the cudaGetDevice calls whose
result the Buffer conversion left unused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: set_stream matches cuda::buffer's contract, not a divergence

cuda::buffer's set_stream is deliberately non-synchronizing -- its
documented synchronization is a stale note left behind when the
synchronization was removed -- so our non-synchronizing set_stream
matches it in both name and contract, and the comment claiming a
divergence was wrong.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: mark cuda::Buffer::clear deprecated in the documentation

Documentation-level only: the [[deprecated]] attribute would warn from
GridHandle::reset and NodeManager::reset, template members in our own
headers that must keep calling clear() until every buffer type provides
destroy() -- an unactionable diagnostic for callers of reset(), and a
build break under -Werror. The attribute lands when those callers
migrate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

* NanoVDB: exercise the concept-required members of the test resources

The trait checks detect allocate/deallocate through unevaluated contexts,
which never odr-use them, so nvcc warned that every synchronous pair and
stub member was declared but never referenced (#177-D). The attribute
route is closed -- nvcc's front end ignores [[maybe_unused]] for this
diagnostic -- so reference them the honest way: a test that verifies the
synchronous halves of the counting and stream-recording doubles behave
like their stream-ordered halves, and one that pins the trait probes'
stub behavior. The TU now builds with no warnings at all.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:27:40 -07:00
Jonathan Swartz bb597a2109 Merge pull request #2250 from swahtz/tranche-2-maintenance-monoids
NanoVDB: parallel checksum, cooperative GridStats, coalesced IndexToGrid optimizations
2026-08-12 05:04:11 +12:00
Jonathan Swartz b795681895 Merge pull request #2277 from matthewdcong/cu128_cccl_fix
Fix CUDA 12.8 build with bundled CCCL
2026-08-11 14:50:40 +12:00
Matthew Cong 230d640768 Fix CUDA 12.8 build with bundled CCCL
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2026-08-10 12:02:55 -07:00
Matthew Cong 2ad18b6aeb Fix race conditions and empty launches in DistributedPointsToGrid (#2275)
* Fix empty chunks in the multi-GPU points-to-grid pipeline

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix race condition with prefix update and incorrect stream in process root

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Remove unsupported cudaMemAdvise call on memory pool allocations

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
2026-08-10 08:31:58 -07:00
Jonathan Swartz b082b5ec7e Merge remote-tracking branch 'upstream/master' into tranche-2-maintenance-monoids 2026-08-08 00:47:40 +00:00
Jonathan Swartz 8a91e1e096 Merge pull request #2249 from swahtz/tranche-1-correctness-determinism
NanoVDB: correctness and determinism fixes
2026-08-08 12:38:49 +12:00
Jonathan Swartz 457b671c34 NanoVDB: public DeviceBuffer::recordUse, per-device frees, and stronger stream tests
Review follow-ups from the Copilot pass on this PR.

The lifetime event only covers uses issued through deviceUpload/deviceDownload; work enqueued
against the raw pointer from deviceData() is invisible to it, so on a non-blocking stream such
work could still outlive the buffer (as on master, where no API could express the dependency).
Make recordUse public so raw-pointer callers can register their stream, and document the
contract on deviceData().

freeDeviceBuffers issued every device's free on one stream, but a stream belongs to a single
device; free each allocation on its own device instead (caller's stream for the current device,
the owning device's default stream otherwise, switching devices as needed). Same pre-existing
shape as master's loop; correctness on multi-GPU is by inspection - this machine has one GPU.

Split the free-ordering regression test into the two scenarios that discriminate the two
historical bugs: a blocking stream with an unregistered write (fails if frees move off the
default stream: the reviewed revision) and a non-blocking stream with a recordUse-registered
write (fails without the event: master). Verified per revision: current passes both, the
reviewed revision fails both, master fails the non-blocking one.

Add NonBlockingStreamSignedFloodFill, mirroring the dilation regression: identical output on
the default stream and on a non-blocking stream while the default stream is occupied, with the
readback on the producing stream. This locks the cross-stream contract; it cannot discriminate
processRoot's internal ordering on constructible inputs (verified: the pre-fix header passes
it) because that path only does work when interior root-level tiles exist and pre-fix code was
accidentally host-synchronous otherwise - noted in the test.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-08-07 23:35:23 +00:00
Jonathan Swartz 111b12f9c1 NanoVDB: guard DeviceBuffer self-move and document clear()'s stream argument
Two non-blocking review points from harrism.

Self-move assignment freed the buffers and then read its own (now dangling) members. Verified:
without the guard 'buf = std::move(buf)' aborts on the deviceData assertion because mDeviceCount
has been zeroed; with it the buffer is left intact. Note UnifiedBuffer::operator=(&&) has the
same flaw and is left for a separate change.

Also document clear()'s stream argument. The review noted it had become dead, which was true of
the revision reviewed - the per-device stream overrode it. With frees now ordered by the tracking
event the argument is honoured again and simply selects where the free is enqueued, so any stream
is safe to pass regardless of where the buffer was used.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-08-07 22:31:52 +00:00
Jonathan Swartz f95115304e NanoVDB: add a regression test for DeviceBuffer multi-stream free ordering
The existing DeviceBufferNonBlockingStreamLifetime test only asserts that repeated
allocate/use/free returns cudaSuccess, so it passes with or without a correctly ordered free.
Its comment also described the superseded 'free on the stream it was allocated on' mechanism.
Reword it as the smoke test it is, and add a test that actually discriminates.

DeviceBufferMultiStreamFreeOrdering uses a device-only buffer on purpose - it owns no pinned
host memory, so clear()'s cudaFreeHost (which implicitly synchronizes) cannot mask the problem.
A second stream is parked behind a busy-wait with a write to the buffer queued behind it, the
buffer is destroyed, and the next allocation is checked for corruption. Verified to fail against
the pre-fix DeviceBuffer (64 MB clobbered) and pass with the fix.

A warm-up pass is required: on a cold context the first launches serialize and the race never
forms. The recycled/still-pending state is reported in the failure message but deliberately not
used as a preconditon - with a correctly ordered free the allocator cannot hand the block out
again until the other stream drains, so those observations are the fix working rather than a
reason to skip.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-08-07 22:20:16 +00:00
Jonathan Swartz 5b5e936723 NanoVDB: update the DeviceBuffer entry in pending changes
The entry described the mechanism as freeing on the stream the buffer was last used on,
which was superseded: frees are now ordered after every stream the buffer was used on via a
per-device event.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-08-07 22:04:07 +00:00
Jonathan Swartz e3351803dc NanoVDB: order DeviceBuffer device frees after every stream that used the buffer
Tracking only the stream a device buffer was LAST used on is not sufficient: two streams can
both have work in flight, and freeing on the later one leaves the earlier one's work running
against memory the allocator has already recycled. Reported in review by matthewdcong.

Reproduced: a device-only DeviceBuffer plus a user kernel on a second blocking stream, with a
warm-up pass so the launches actually overlap. The freed block is recycled and all 64 MB of the
next allocation are overwritten by the late kernel. Freeing on the default stream (the previous
behaviour) passes, because the legacy stream is implicitly ordered after every blocking stream;
freeing on the last-used stream fails.

Replace the per-device stream with a per-device event. Each use waits on the event before
issuing work and re-records it afterwards, so one event transitively covers every stream the
buffer has been used on, and the frees wait on it. This is correct for any mix of blocking and
non-blocking streams and for any number of them, and it removes the requirement that callers
keep their streams alive until the buffer is destroyed, since the event is owned by the buffer.

Verified: both the multi-blocking-stream case and the non-blocking case pass, and the VBM,
topology and maintenance goldens are unchanged (46/46 on dragon and emu).

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-08-07 21:55:45 +00:00
Nick Avramoussis 92a4f48f8f Use __has_include to decide on tbb/version.h
Signed-off-by: Nick Avramoussis <4256455+Idclip@users.noreply.github.com>
2026-07-30 09:19:36 +12:00
Jonathan Swartz b0b99d6ce7 Merge branch 'master' into tranche-2-maintenance-monoids 2026-07-23 03:34:37 +00:00
Maciej Małecki 1ca9bb066e Include file containing TBB version information
Such as `TBB_INTERFACE_VERSION`.
Without it, the `#else` statement always triggers, breaking build for
newer TBB version (albeit rare at this point).

Signed-off-by: Maciej Małecki <me@mmalecki.com>
2026-07-21 22:44:55 +02:00
Jonathan Swartz 59fcab450a Comment fix
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-20 00:43:38 +00:00
Jonathan Swartz 65c9d4884c Update pendingchanges
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-20 00:38:44 +00:00
Jonathan Swartz ad251341d0 Comment cleanup and swap some constants for semantically defined values
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-20 00:28:00 +00:00
Jonathan Swartz 06ad21bf70 Improve comments for processInternal/Leaf
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-19 03:53:42 +00:00
Jonathan SwartzandClaude Opus 4.8 2e30d61283 NanoVDB CUDA: precompute GridChecksum combine operators on the host
crc32CombineKernel rebuilt its two GF(2) shift operators by binary
exponentiation on a single device thread every call - a fixed ~480 us that
dominated the checksum for small and medium grids (profiling showed it as large
as the entire tail-streaming pass). The operators depend only on the chunk
length, so they are now built on the host in microseconds and uploaded, and the
device kernel is reduced to the O(chunkCount) fold.

Bit-identical: all checksum goldens pass on dragon/emu/crawler/wdas_cloud.

A/B vs the parallel-checksum base (RTX PRO 6000, 3 rounds):
  dragon +40.7% (1.69x)  emu +15.3%  crawler +9.8%  wdas_cloud +1.9%
The gain shrinks with grid size because it removes a fixed cost; transient
memory unchanged.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 01:53:03 +00:00
Jonathan SwartzandClaude Opus 4.8 f20dd2a50e NanoVDB: add pending CHANGES entries for the CUDA correctness fixes
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-16 23:00:35 +00:00
Jonathan SwartzandClaude Opus 4.8 72a4bfe64e NanoVDB CUDA: GridStats review fixes - guard leaf launch + unique per-node stats slot
Addresses PR review on the cooperative GridStats:
- processLeaf is now launched only when nodeCount[0] > 0, matching the guards on
  the internal-node launches (a leaf-less grid would otherwise be a <<<0,...>>>
  launch).
- The average path now gives each node its own d_stats slot instead of reusing
  one of its children's. A fully-tiled internal node (active value tiles, no
  children) never claimed a child slot, so the old code wrote d_stats[-1] (out
  of bounds) and propagated an invalid slot to its parent. d_stats is now sized
  for all nodes (leaves, then lower, then upper) and each node writes slot
  leafCount + levelOffset + nodeID; the sSlot/atomicMax machinery is removed.

stats_minmax is unaffected (it never touches d_stats) and stays byte-exact;
stats_all output is unchanged for grids without childless internal nodes and
correct for those that have them. compute-sanitizer memcheck clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-15 00:02:52 +00:00
Jonathan SwartzandClaude Fable 5 5fcaa5a347 NanoVDB CUDA: coalesced IndexToGrid node/leaf remap
Port of Agent A's exp/20. Consecutive threads now process consecutive table
entries and leaf values - the former mapping gave each thread a consecutive RUN
(a 32-entry stride across the warp for internal nodes, 8 for leaves) on every
iteration. The 4 KB upper-node value/child masks are copied cooperatively across
the block instead of member-wise by thread 0 while the rest idled.

Combines with the Tranche-1 scoped zero-init already on this file: the leaf
kernel's stats/padding-gap zeroing is preserved inside the thread-0 block, and
the coalesced value loop still writes every mValues[i], so output stays
bit-identical - all indextogrid goldens pass on dragon/emu/crawler/wdas_cloud.

A/B vs the Tranche-1 base (RTX PRO 6000, 3 rounds): dragon +23%, emu +21%,
crawler +18%, wdas_cloud +14%; transient memory unchanged. The remap gain is on
top of Tranche 1's scoped init, so absolute indextogrid is now faster than both
master and A's exp/20.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-15 00:02:52 +00:00
Jonathan SwartzandClaude Fable 5 44d5e657a7 NanoVDB CUDA: cooperative GridStats reductions (warp-per-leaf, block-per-node)
Port of Agent A's exp/13. Replaces the thread-per-node serial reductions:
- processLeaf: one warp per leaf - lanes stride the 512 voxel slots (mask-gated)
  and merge partial statistics through shared memory; lane 0 keeps the bbox
  update and the final store.
- processInternal: one 128-thread block per internal node striding the
  4,096/32,768-entry child table with a shared-memory tree reduction of Stats +
  bbox. The former kernel gave each thread up to 32,768 serial child visits and
  ran an entire lower level of a 181M-voxel grid on ~9 warps of a 188-SM GPU.

min/max are order-independent, so stats_minmax output is bit-identical - all
stats_minmax goldens pass on dragon/emu/crawler/wdas_cloud. avg/stddev
(stats_all) use a Welford-correct parallel merge but are NOT bit-identical to
the serial result (floating-point reduction order differs); stats_all stays a
bench-only measurement, never a byte-exact claim.

A/B vs the Tranche-1 base (RTX PRO 6000, 3 rounds):
  stats_minmax  dragon +87.3%  emu +82.5%  crawler +80.3%  wdas +83.5%  (5-8x)
  stats_all     dragon +71.9%  emu +51.7%  crawler +29.8%  wdas +35.1%
transient memory unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-15 00:02:52 +00:00
Jonathan SwartzandClaude Fable 5 9e6d47cb88 NanoVDB CUDA: parallel bit-identical GridChecksum (slicing-by-4 + GF(2) combine)
Port of Agent A's exp/14. Replaces the two serial anti-patterns in blockedCRC32
with parallel equivalents:
- Per-block CRC via slicing-by-4: the 256-entry base LUT is staged into shared
  memory and three derived slice tables are built in place, so the dependent CRC
  chain advances four bytes per step with four shared-memory lookups instead of
  one byte per L2 lookup. (Not __constant__: divergent indices serialize there.)
- The final fold (a single thread CRCing the whole block-CRC array - megabytes
  for multi-GB grids) becomes parallel per-chunk CRCs plus a GF(2) crc32_combine
  (zlib construction: one shift operator per fixed chunk length, applied per
  fold). Small inputs keep the single-thread path.

Checksum VALUES are bit-identical (slicing-by-4 and the GF(2) combine are both
value-preserving); all checksum_full goldens pass on dragon/emu/crawler/
wdas_cloud. The pre-v32.6.0 crc32TailOld path is untouched.

A/B vs the Tranche-1 base (RTX PRO 6000, 3 rounds): dragon +84.7%, emu +92.0%,
crawler +93.2%, wdas_cloud +94.1% (~6.5x / 12.5x / 15x / 17x); transient memory
unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-15 00:02:52 +00:00
Jonathan SwartzandClaude Opus 4.8 bbb19a9c27 NanoVDB CUDA: include <algorithm> for std::min; fix DeviceBuffer stream comments
Addresses PR review:
- PointsToGrid/MeshToGrid/DistributedPointsToGrid use std::min in the grid-name
  copy but did not include <algorithm> (it compiled only via transitive
  includes). Add the include so each header is self-sufficient.
- DeviceBuffer comments still described mStreams as the "allocation stream", but
  it is now updated on every deviceUpload/deviceDownload; reword them to
  "last-used stream" to match the actual behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-15 00:02:23 +00:00
Jonathan SwartzandClaude Opus 4.8 04c16f94ad NanoVDB CUDA: update DeviceBuffer free-stream on every up/download
Addresses PR review: mStreams[device] was set only when the device allocation
was first created. If the buffer is later re-uploaded (or downloaded) on a
different stream, the destructor/move-assign/clear would still free on the old
stream, which does not necessarily order after the most recent work on the new
stream (notably for cudaStreamNonBlocking). Update mStreams[device] to the
current stream on every host<->device copy in deviceUpload/deviceDownload, not
just on first allocation. Guarded on mStreams (null for externally-managed
buffers, which are never freed here).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-14 22:54:59 +00:00
Jonathan SwartzandClaude Opus 4.8 3cdaa03ba8 NanoVDB CUDA: drain default stream after NonBlockingStreamDilate busy-wait (test)
Addresses PR review: the test launches streamBusyWaitKernel on the default
stream to occupy it during the candidate dilation, but only synchronized the
non-blocking stream afterward. The busy-wait could remain queued on stream 0
and leak into subsequent tests, causing timing-dependent flakes. Drain it with
a cudaDeviceSynchronize() after the candidate run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-14 05:52:51 +00:00
Jonathan SwartzandClaude Opus 4.8 5bb2eba69d NanoVDB CUDA: track DeviceBuffer free stream per device (multi-GPU fix)
Addresses PR review: the Tranche-1 stream-ordered-free change stored a single
mStream, overwritten by every DeviceBuffer::deviceUpload. A DeviceBuffer can
hold an allocation on each device (mGpuData is per-device), and the multi-GPU
example (ex_make_mgpu_nanovdb) uploads one handle to every device with that
device's own stream. The destructor/move-assign then freed *every* device's
allocation on the last device's stream - a cross-device cudaFreeAsync(ptr@devA,
stream@devB), which is invalid (the free stream must belong to the allocation's
device). Pre-Tranche-1 code freed on stream 0 uniformly, so this was a
regression for the multi-GPU path.

Track the allocation stream per device in a cudaStream_t array parallel to
mGpuData, set in init()/deviceUpload() at mStreams[device], and free each
mGpuData[i] on its own mStreams[i] in clear()/move-assignment. The single-GPU
path is unchanged (one device, one stream); the multi-GPU path now frees each
device's allocation on its own device-bound stream.

Single-GPU verified (DeviceBuffer lifetime gtest + byte-exact validation); the
multi-GPU free path is correct by construction but not runtime-tested here
(single-GPU environment).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-14 04:44:24 +00:00
Jonathan SwartzandClaude Opus 4.8 0e5fbf3c19 NanoVDB CUDA: only sync SignedFloodFill root pass on a non-default stream
The Tranche-1 stream-correctness fix added an unconditional
cudaStreamSynchronize(stream) before processRoot's synchronous default-stream
cudaMemcpy of the tree. That sync is only needed when the caller's node passes
ran on a non-default stream; on the default stream (0) the blocking copy already
orders after prior stream-0 work, so the extra sync there was pure overhead - a
measured ~4-11% regression on floodfill/dragon vs master. Guard it with
`if (stream != 0)`. Correctness for non-blocking-stream callers is unchanged.

floodfill is now NEUTRAL vs master on dragon/emu/crawler (0 regressions), and
byte-exact validation is unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-14 04:23:13 +00:00
Jonathan SwartzandClaude Opus 4.8 465599ddab NanoVDB CUDA: null-terminate over-long grid names; query current device in test
Addresses PR review:
- The grid-name copy could leave GridData::mGridName without a null terminator
  when the name length >= MaxNameSize: min(size+1, MaxNameSize) == MaxNameSize
  overwrote the entire zeroed field, and gridName() (a C-string accessor) would
  then read past it. Copy at most MaxNameSize-1 bytes and rely on the preceding
  memset for termination, truncating over-long names instead. Fixed in
  PointsToGrid, MeshToGrid and DistributedPointsToGrid.
- GridName_CudaPointsToGrid gains an over-long (>= MaxNameSize) case asserting
  the result is truncated to MaxNameSize-1 chars and stays null-terminated.
- NonBlockingStreamDilate_ValueOnIndex queried device 0 for the clock rate;
  query the current device via cudaGetDevice() instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-14 04:23:13 +00:00
Jonathan SwartzandClaude Opus 4.8 992aace6cf NanoVDB CUDA: scope IndexToGrid zero-init to the output it actually needs
The Tranche-1 determinism fix zero-initialized the *entire* IndexToGrid output
buffer so that bytes the build kernels never write - stats fields absent from
the source grid, and struct alignment padding - are deterministic instead of
carrying recycled pool bytes (which otherwise leak into written .nvdb files).
Measured against master this cost 10-22% on indextogrid across dragon/emu/
crawler/wdas_cloud: the leaf array dominates the buffer and was memset to zero
and then immediately overwritten value-by-value - a redundant multi-GB pass.

Scope the initialization to the bytes that are genuinely left unwritten:

  * getBuffer() memsets only the non-leaf region [0, node[0]) - grid, tree,
    root, root tiles and the internal nodes - a small fraction of the buffer.
  * processLeafsKernel zeroes each leaf's own gap [&mMinimum, mValues): the
    stats fields (skipped when the source has no stats) and any padding before
    the 32-aligned mValues array. Every mValues[i] is written anyway, so this
    reproduces the former full zero-init byte-for-byte.

Output is bit-identical to the full zero-init - all indextogrid goldens pass on
dragon/emu/crawler/wdas_cloud, compute-sanitizer memcheck is clean and initcheck
shows no new reports - and the regression vs master is gone (NEUTRAL on all four
datasets, RTX PRO 6000 / sm_120).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-14 04:08:04 +00:00
Jonathan Swartz fb1c692d05 comment fix
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-14 03:18:17 +00:00
Jonathan Swartz 42afea5d40 Fix up comments
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2026-07-14 03:02:43 +00:00
Jonathan SwartzandClaude Opus 4.8 c3cde0fea6 NanoVDB CUDA: correctness and determinism fixes
Five correctness/determinism fixes to the NanoVDB CUDA tools:

* GridChecksum: evalChecksum copied sizeof(GridData)+sizeof(TreeData) (736 B)
  from a device buffer holding a single uint32 CRC into the 4-byte head/tail of
  a stack Checksum -- an out-of-bounds device read plus host stack corruption
  (compute-sanitizer visible; a hard CUDA "invalid argument" on some configs).
  Copy sizeof(uint32_t) at all four sites, matching validateChecksum.

* Stream propagation: the NN_FACE_EDGE_VERTEX leaf-dilation kernel and
  SignedFloodFill's root pass ran on the default stream instead of the caller's
  stream, silently corrupting output for callers using a non-blocking stream.
  Launch and synchronize on the operation's stream.

* Bounded grid-name copy: the grid-name copy read a fixed MaxNameSize bytes from
  a possibly shorter std::string (whose .data() is never null, making the memset
  branch dead), over-reading up to 255 bytes of host heap into the grid and any
  written .nvdb file. Zero the field, then copy only min(name.size()+1,
  MaxNameSize) bytes. Applied to PointsToGrid, DistributedPointsToGrid, MeshToGrid.

* Deterministic output initialization: PointsToGrid and IndexToGrid left
  alignment padding and stats fields carrying recycled pool bytes, making output
  nondeterministic and leaking host heap into files. Zero the whole grid buffer
  once after allocation. For PointsToGrid this makes the three dense
  background-fill passes (upper/lower value tables, and the inactive-leaf-values
  pass for all but the Point build type) redundant, so they are dropped.

* DeviceBuffer stream-ordered lifetime: the destructor, move-assignment and
  clear() freed device allocations on the default stream regardless of the
  stream the buffer was last used on -- a latent use-after-free for
  non-blocking-stream callers, masked today by legacy-stream semantics. Track the
  allocation stream and free on it, matching the stream-ordered free TempPool
  now performs.

Adds gtest coverage in unittest/TestNanoVDB.cu: cross-stream dilation
determinism, grid-name round-trip (empty/short/max-length), PointsToGrid output
determinism, and DeviceBuffer non-blocking-stream lifetime. No performance claims.

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 02:42:23 +00:00
148 changed files with 18086 additions and 1977 deletions
+3
View File
@@ -120,6 +120,9 @@ jobs:
- name: install
shell: powershell
run: .\ci\install_windows.ps1
- name: install_numpy
shell: powershell
run: .\ci\install_windows_numpy.ps1
- name: build
# nvcc doesn't set _WIN32 when run in bash so we need to set it manually
shell: bash
+55
View File
@@ -0,0 +1,55 @@
name: TEACHME
# Compile-checks the NanoVDB TEACHME tutorial against the real headers. Runs
# when EITHER the NanoVDB headers or the tutorial change, so an API change that
# breaks a lesson code block is caught here (the whole point of the harness).
# Compile-only: no GPU is required.
on:
push:
branches:
- 'master'
- 'feature/**'
- 'pr/**'
paths:
- 'nanovdb/nanovdb/**'
- 'doc/nanovdb/TEACHME/**'
- '.github/workflows/teachme.yml'
pull_request:
branches:
- '**'
paths:
- 'nanovdb/nanovdb/**'
- 'doc/nanovdb/TEACHME/**'
- '.github/workflows/teachme.yml'
workflow_dispatch:
# Allow subsequent pushes to the same PR or REF to cancel any previous jobs.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
teachme-docs-compile:
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
name: teachme-docs-compile
container:
image: aswf/ci-openvdb:2024-clang17.2
env:
# Compile-only target; matches the arch used by the other NanoVDB jobs.
CUDAARCHS: "80"
# Each TU compiles the kitchen-sink fixture (most of NanoVDB's heavy
# headers), so nvcc is memory-hungry. Cap parallel compiles to stay
# within the runner's RAM — unbounded -j OOMs the 8-core/32 GB runner.
TEACHME_JOBS: "4"
steps:
- uses: actions/checkout@v3
- name: setup_cuda_12
run: |
echo "/usr/local/cuda-12/bin" >> $GITHUB_PATH
echo "LD_LIBRARY_PATH=/usr/local/cuda-12/lib64:$LD_LIBRARY_PATH" >> $GITHUB_ENV
- name: run_harness
# Reference-checks every #include / prose header path, then compiles
# every tagged code block. The OpenVDB-dependent blocks are skipped
# (still reference-checked) when OpenVDB is not installed in the image.
run: bash doc/nanovdb/TEACHME/test/run.sh
+1 -1
View File
@@ -6,10 +6,10 @@
# Admin/General
/.github/ @kmuseth @apradhana @jmlait @idclip @danrbailey @richhones
/.github/workflows/nanovdb.yml @kmuseth @apradhana
/ci/ @kmuseth @apradhana @jmlait @idclip @danrbailey @richhones
/cmake/ @kmuseth @apradhana @jmlait @idclip @danrbailey @richhones
/doc/ @kmuseth @apradhana @jmlait @idclip @danrbailey @richhones
/pendingchanges/ @kmuseth @apradhana @jmlait @idclip @danrbailey @richhones
/tsc/ @kmuseth @apradhana @jmlait @idclip @danrbailey @richhones
CODEOWNERS @kmuseth @apradhana @jmlait @idclip @danrbailey @richhones
LICENSE @kmuseth @apradhana @jmlait @idclip @danrbailey @richhones
+67
View File
@@ -0,0 +1,67 @@
# TEACHME
This directory contains interactive lesson documents designed to be loaded by
an LLM coding agent (Claude Code, Cursor, or similar) to teach a user NanoVDB
interactively.
The lesson teaches how to *use* NanoVDB to read, build, transform, and render
sparse volumetric data on CPU and GPU. It is a single self-contained markdown
file the agent reads at the start of a session.
## Getting started
The easiest way to use this lesson is with [Claude Code](https://docs.anthropic.com/en/docs/claude-code)
(CLI or IDE extension) from the root of an OpenVDB checkout. The agent can
read the lesson files *and* the NanoVDB source code, so it can verify API
details and help you debug exercises in real time.
**Example prompts:**
To learn how to use NanoVDB:
```
Read doc/nanovdb/TEACHME/nanovdb_user_lesson.md and teach me how to use NanoVDB.
```
You can also give the agent context about your background so it can tailor
the lesson:
```
Read doc/nanovdb/TEACHME/nanovdb_user_lesson.md and teach me NanoVDB. I have
C++ and CUDA experience but I've never used OpenVDB or any sparse 3D data
structure before.
```
Keep the matching cheatsheet (`*_cheatsheet.md`) open in your editor while
working through exercises — it's a quick reference for the APIs and
invariants covered in the lesson.
## How it works
The lesson is a self-contained markdown file that serves as both a
curriculum and an instructor prompt. The LLM acts as an interactive
instructor: teaching concepts module by module, quizzing the student, and
adapting to their responses.
The lesson includes:
- Teacher instructions (persona, pacing, scope)
- Module-by-module curriculum with embedded concepts and code examples
- Quiz questions and an answer key
- Exercises with progressive difficulty
- A capstone project
- A reference table at the end
## Available lessons
| Lesson | Cheat sheet | Covers |
|---|---|---|
| [nanovdb_user_lesson.md](nanovdb_user_lesson.md) | [nanovdb_user_cheatsheet.md](nanovdb_user_cheatsheet.md) | Reading `.nvdb` files, `ReadAccessor`, `NodeManager`, math + sampling, HDDA ray-march, GPU kernels with `Grid`, GPU topology builders (`PointsToGrid`, `MeshToGrid`, `DilateGrid`, etc.), `IndexGrid` + `VoxelBlockManager`, OpenVDB ↔ NanoVDB conversion |
## Not covered
- The OpenVDB side (the dynamic, mutable tree used at film/sim time) — this
lesson treats OpenVDB only at the conversion boundary.
- Houdini / Maya integration — outside the scope.
- Gaussian splatting and fVDB-specific topics — see the fVDB repo's own
TEACHME at <https://github.com/openvdb/fvdb-core/tree/main/docs/TEACHME>.
+6
View File
@@ -0,0 +1,6 @@
# Build + editor + render artifacts (not source)
build/
compile_commands.json
.clangd
*.ppm
sphere.png
@@ -0,0 +1,27 @@
cmake_minimum_required(VERSION 3.20)
project(nanovdb_capstone LANGUAGES CXX CUDA)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CUDA_STANDARD 17)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON) # emit build/compile_commands.json
if(NOT DEFINED CMAKE_CUDA_ARCHITECTURES)
set(CMAKE_CUDA_ARCHITECTURES 89) # RTX 6000 Ada; override with -DCMAKE_CUDA_ARCHITECTURES=...
endif()
# NanoVDB headers pull in CUDA lambdas / relaxed-constexpr paths.
string(APPEND CMAKE_CUDA_FLAGS " --extended-lambda --expt-relaxed-constexpr")
# This file is at doc/nanovdb/TEACHME/capstone/CMakeLists.txt; the NanoVDB
# include root (the dir containing the nanovdb/ header subdir) is the repo's
# `nanovdb/` subdir — repo root is four dirs up.
get_filename_component(_teachme "${CMAKE_CURRENT_SOURCE_DIR}" DIRECTORY) # TEACHME
get_filename_component(_docnv "${_teachme}" DIRECTORY) # doc/nanovdb
get_filename_component(_doc "${_docnv}" DIRECTORY) # doc
get_filename_component(REPO_ROOT "${_doc}" DIRECTORY) # repo root
set(NANOVDB_INCLUDE_ROOT "${REPO_ROOT}/nanovdb")
message(STATUS "NanoVDB include root: ${NANOVDB_INCLUDE_ROOT}")
add_executable(raymarch raymarch.cu)
target_include_directories(raymarch PRIVATE "${NANOVDB_INCLUDE_ROOT}")
# Binary lands in the build dir (conventional). raymarch writes sphere.ppm to
# the current working directory, so run it from wherever you want the output.
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env python3
"""Convert a binary P6 PPM to PNG using only the Python standard library
(the container has no ImageMagick / PIL). Usage: python3 ppm2png.py in.ppm out.png"""
import sys, struct, zlib, binascii
def main(src, dst):
data = open(src, "rb").read()
i = 0
def tok(i):
n = len(data)
while i < n and data[i] in b" \t\n\r": i += 1
s = i
while i < n and data[i] not in b" \t\n\r": i += 1
if s == i: raise ValueError("truncated or non-binary PPM")
return data[s:i], i
magic, i = tok(i)
assert magic == b"P6", "not a binary PPM"
w, i = tok(i); h, i = tok(i); mx, i = tok(i)
i += 1 # single whitespace after maxval
W, H = int(w), int(h)
px = data[i:i + W * H * 3]
def chunk(typ, payload):
return (struct.pack(">I", len(payload)) + typ + payload +
struct.pack(">I", binascii.crc32(typ + payload) & 0xffffffff))
raw = bytearray()
for y in range(H):
raw.append(0) # filter: none
raw += px[y * W * 3:(y + 1) * W * 3]
png = (b"\x89PNG\r\n\x1a\n"
+ chunk(b"IHDR", struct.pack(">IIBBBBB", W, H, 8, 2, 0, 0, 0))
+ chunk(b"IDAT", zlib.compress(bytes(raw), 9))
+ chunk(b"IEND", b""))
open(dst, "wb").write(png)
print(f"wrote {dst} ({W}x{H})")
if __name__ == "__main__":
main(sys.argv[1], sys.argv[2])
+111
View File
@@ -0,0 +1,111 @@
// Capstone scaffold: GPU level-set ray-march renderer.
//
// The host plumbing is DONE (build SDF, upload to device, launch, read back,
// write PPM) and the light direction + ambient level are handed to the kernel
// as parameters. Your job is the kernel body — the per-pixel ray-march + shade.
// As shipped it compiles and runs but only writes a placeholder gradient.
//
// Build: cmake -S . -B build && cmake --build build
// Run: ./build/raymarch [ambient] [lx ly lz] (writes sphere.ppm)
// View: python3 ppm2png.py sphere.ppm sphere.png (then open sphere.png)
#include <nanovdb/NanoVDB.h>
#include <nanovdb/tools/CreatePrimitives.h>
#include <nanovdb/cuda/DeviceBuffer.h>
#include <nanovdb/cuda/GridHandle.cuh>
#include <nanovdb/math/Ray.h>
#include <nanovdb/math/HDDA.h> // nanovdb::math::ZeroCrossing
#include <nanovdb/math/Stencils.h> // nanovdb::math::GradStencil
#include <cstdio>
#include <cstdlib>
#include <cmath>
#include <vector>
using GridT = nanovdb::NanoGrid<float>;
__global__ void render(const GridT* grid, unsigned char* img, int W, int H,
nanovdb::Vec3f lightDir, float ambient)
{
const int x = blockIdx.x * blockDim.x + threadIdx.x;
const int y = blockIdx.y * blockDim.y + threadIdx.y;
if (x >= W || y >= H) return;
const int pid = (y * W + x) * 3;
// ===================== YOUR WORK STARTS HERE =====================
// Placeholder so it compiles & runs: a horizontal grey gradient.
unsigned char shade = (unsigned char)(255.f * x / W);
// The aesthetic inputs are given to you as parameters: `lightDir` and
// `ambient` (set in main, overridable on the command line). You write the
// ray-march and the shading math. Signatures are in the cheat sheet.
//
// 1. Camera ray (WORLD space). Pinhole: an eye point and, per pixel, a
// direction through the image plane. You decide the math — remember
// aspect ratio and that image y grows downward. Wrap it in a Ray.
//
// 2. The SDF lives in INDEX space, so convert your world ray to index
// space before you trace it. (Ray has a method for this.)
//
// 3. Find where the ray first crosses the surface (the SDF sign change).
// Module 5 gave you a helper in math/HDDA.h that drives an HDDA and
// reports the hit voxel, the value, and the ray parameter.
//
// 4. On a hit, the surface normal is the (normalized) SDF gradient at the
// hit voxel. Module 5's gradient stencil computes it. (Watch its
// template parameter.)
//
// 5. Shade: a Lambertian intensity from the normal and the provided
// `lightDir`, lifted off the floor by `ambient`; write it as greyscale.
// On a MISS, leave the background.
// ====================== YOUR WORK ENDS HERE ======================
img[pid] = shade; img[pid + 1] = shade; img[pid + 2] = shade;
}
int main(int argc, char** argv)
{
// Shading parameters passed into the kernel. Defaults, overridable on the
// command line: ./raymarch [ambient] [lx ly lz]
float ambient = 0.15f;
nanovdb::Vec3f lightDir(-0.577f, 0.577f, -0.577f); // (-1,1,-1) normalized
if (argc >= 2) ambient = (float)std::atof(argv[1]);
if (argc >= 5) lightDir = nanovdb::Vec3f((float)std::atof(argv[2]),
(float)std::atof(argv[3]),
(float)std::atof(argv[4]));
lightDir.normalize();
// Build a level-set sphere SDF on the host (radius 100, voxel size 1).
auto handle = nanovdb::tools::createLevelSetSphere<float>(
/*radius=*/100.0, /*center=*/nanovdb::Vec3d(0.0),
/*voxelSize=*/1.0, /*halfWidth=*/3.0);
// Move to the device. copy<DeviceBuffer>() fills the HOST side of the dual
// buffer; deviceUpload() pushes it to the GPU — skip it and deviceGrid()
// returns nullptr.
auto devHandle = handle.copy<nanovdb::cuda::DeviceBuffer>();
devHandle.deviceUpload();
const GridT* dGrid = devHandle.deviceGrid<float>();
if (!dGrid) { std::printf("no device grid\n"); return 1; }
const int W = 512, H = 512;
unsigned char* dImg = nullptr;
cudaMalloc(&dImg, size_t(W) * H * 3);
const dim3 block(16, 16), gridDim((W + 15) / 16, (H + 15) / 16);
render<<<gridDim, block>>>(dGrid, dImg, W, H, lightDir, ambient);
cudaDeviceSynchronize();
if (auto e = cudaGetLastError(); e != cudaSuccess) {
std::printf("CUDA error: %s\n", cudaGetErrorString(e));
return 1;
}
std::vector<unsigned char> img(size_t(W) * H * 3);
cudaMemcpy(img.data(), dImg, img.size(), cudaMemcpyDeviceToHost);
cudaFree(dImg);
FILE* f = std::fopen("sphere.ppm", "wb");
std::fprintf(f, "P6\n%d %d\n255\n", W, H);
std::fwrite(img.data(), 1, img.size(), f);
std::fclose(f);
std::printf("wrote sphere.ppm (%dx%d)\n", W, H);
return 0;
}
@@ -0,0 +1,105 @@
// Capstone: GPU level-set ray-march renderer.
// Build an SDF sphere -> move to device -> one thread per pixel ->
// ZeroCrossing to find the surface -> Lambertian shade via SDF gradient
// normal -> write PPM.
#include <nanovdb/NanoVDB.h>
#include <nanovdb/tools/CreatePrimitives.h>
#include <nanovdb/cuda/DeviceBuffer.h>
#include <nanovdb/cuda/GridHandle.cuh>
#include <nanovdb/math/Ray.h>
#include <nanovdb/math/HDDA.h> // ZeroCrossing lives here
#include <nanovdb/math/Stencils.h> // GradStencil
#include <cstdio>
#include <cstdlib>
#include <cmath>
#include <vector>
using GridT = nanovdb::NanoGrid<float>;
__global__ void render(const GridT* grid, unsigned char* img, int W, int H,
nanovdb::Vec3f lightDir, float ambient)
{
const int x = blockIdx.x * blockDim.x + threadIdx.x;
const int y = blockIdx.y * blockDim.y + threadIdx.y;
if (x >= W || y >= H) return;
const int pid = (y * W + x) * 3;
// --- pinhole camera in WORLD space ---
const nanovdb::Vec3f eye(0.f, 0.f, -300.f);
const float aspect = float(W) / float(H);
const float tanHalfFov = tanf(0.5f * 45.f * 3.14159265f / 180.f);
const float u = (2.f * ((x + 0.5f) / W) - 1.f) * tanHalfFov * aspect;
const float v = (1.f - 2.f * ((y + 0.5f) / H)) * tanHalfFov; // flip y
nanovdb::Vec3f dir(u, v, 1.f);
dir.normalize();
nanovdb::math::Ray<float> wRay(eye, dir);
auto iRay = wRay.worldToIndexF(*grid); // ray in index space
auto acc = grid->getAccessor();
nanovdb::Coord ijk;
float t = 0.f, val = 0.f;
unsigned char shade = 0; // background (black)
if (nanovdb::math::ZeroCrossing(iRay, acc, ijk, val, t)) {
nanovdb::math::GradStencil<GridT> stencil(*grid);
stencil.moveTo(ijk);
nanovdb::Vec3f n = stencil.gradient(); // SDF gradient = outward normal
n.normalize();
const float diff = fmaxf(0.f, n.dot(lightDir));
const float I = fminf(1.f, ambient + (1.f - ambient) * diff);
shade = (unsigned char)(255.f * I); // greyscale
}
img[pid] = shade; img[pid + 1] = shade; img[pid + 2] = shade;
}
int main(int argc, char** argv)
{
// Shading parameters (passed into the kernel). Defaults, overridable on the
// command line: ./raymarch [ambient] [lx ly lz]
float ambient = 0.15f;
nanovdb::Vec3f lightDir(-0.577f, 0.577f, -0.577f); // (-1,1,-1) normalized
if (argc >= 2) ambient = (float)std::atof(argv[1]);
if (argc >= 5) lightDir = nanovdb::Vec3f((float)std::atof(argv[2]),
(float)std::atof(argv[3]),
(float)std::atof(argv[4]));
lightDir.normalize();
// 1. Build a level-set sphere SDF on the host (radius 100, voxel size 1).
auto handle = nanovdb::tools::createLevelSetSphere<float>(
/*radius=*/100.0, /*center=*/nanovdb::Vec3d(0.0),
/*voxelSize=*/1.0, /*halfWidth=*/3.0);
// 2. Move it to the device. copy<DeviceBuffer>() fills the HOST side of the
// dual buffer; deviceUpload() pushes the bytes to the device. Only then
// is deviceGrid() non-null.
auto devHandle = handle.copy<nanovdb::cuda::DeviceBuffer>();
devHandle.deviceUpload();
const GridT* dGrid = devHandle.deviceGrid<float>();
if (!dGrid) { std::printf("no device grid\n"); return 1; }
// 3. Render.
const int W = 512, H = 512;
unsigned char* dImg = nullptr;
cudaMalloc(&dImg, size_t(W) * H * 3);
const dim3 block(16, 16), grid((W + 15) / 16, (H + 15) / 16);
render<<<grid, block>>>(dGrid, dImg, W, H, lightDir, ambient);
cudaDeviceSynchronize();
if (auto e = cudaGetLastError(); e != cudaSuccess) {
std::printf("CUDA error: %s\n", cudaGetErrorString(e));
return 1;
}
// 4. Copy back and write PPM.
std::vector<unsigned char> img(size_t(W) * H * 3);
cudaMemcpy(img.data(), dImg, img.size(), cudaMemcpyDeviceToHost);
cudaFree(dImg);
FILE* f = std::fopen("sphere.ppm", "wb");
std::fprintf(f, "P6\n%d %d\n255\n", W, H);
std::fwrite(img.data(), 1, img.size(), f);
std::fclose(f);
std::printf("wrote sphere.ppm (%dx%d)\n", W, H);
return 0;
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

@@ -0,0 +1,592 @@
# NanoVDB User Cheat Sheet
A one-page reference for the APIs covered in `nanovdb_user_lesson.md`. Keep
this open while working through exercises.
---
## In one paragraph
NanoVDB is a single-header, read-only, GPU-friendly sparse 3D data
structure. The whole grid (one or more) lives in one contiguous,
32-byte-aligned byte buffer. `cudaMemcpy` it to the device and it works
as-is. Topology is *frozen* after construction; values can change, the set
of active voxels cannot. Every primitive is `__hostdev__` (callable from
both host and device with identical code).
---
## Includes & CMake
```cpp openvdb
#include <nanovdb/NanoVDB.h> // grid, accessor, math
#include <nanovdb/GridHandle.h> // owns the buffer
#include <nanovdb/HostBuffer.h> // CPU allocator
#include <nanovdb/NodeManager.h> // sequential iteration
#include <nanovdb/io/IO.h> // file IO
#include <nanovdb/math/HDDA.h> // ray-march
#include <nanovdb/math/SampleFromVoxels.h> // sampling
// GPU
#include <nanovdb/cuda/DeviceBuffer.h>
#include <nanovdb/cuda/GridHandle.cuh>
#include <nanovdb/cuda/NodeManager.cuh>
// Builders & operators (GPU)
#include <nanovdb/tools/cuda/PointsToGrid.cuh>
#include <nanovdb/tools/cuda/MeshToGrid.cuh>
#include <nanovdb/tools/cuda/DilateGrid.cuh>
#include <nanovdb/tools/cuda/CoarsenGrid.cuh>
#include <nanovdb/tools/cuda/RefineGrid.cuh>
#include <nanovdb/tools/cuda/PruneGrid.cuh>
#include <nanovdb/tools/cuda/MergeGrids.cuh>
#include <nanovdb/tools/cuda/SignedFloodFill.cuh>
// CPU builders & primitives
#include <nanovdb/tools/GridBuilder.h>
#include <nanovdb/tools/CreatePrimitives.h>
// IndexGrid / VBM
#include <nanovdb/tools/VoxelBlockManager.h>
#include <nanovdb/tools/cuda/VoxelBlockManager.cuh>
// Stats, validation, blind data
#include <nanovdb/tools/GridStats.h>
#include <nanovdb/tools/GridChecksum.h>
#include <nanovdb/tools/GridValidator.h>
#include <nanovdb/tools/cuda/AddBlindData.cuh>
// OpenVDB bridge (gated behind NANOVDB_USE_OPENVDB)
#include <nanovdb/tools/CreateNanoGrid.h>
#include <nanovdb/tools/NanoToOpenVDB.h>
```
```cmake
find_package(nanovdb CONFIG REQUIRED)
target_link_libraries(my_app PRIVATE nanovdb::nanovdb)
```
---
## Build types (`NanoGrid<T>`)
| `T` | Stored per voxel |
|-------------------|-------------------------------------------|
| `float`, `double` | Scalar (SDF, density) |
| `int32_t`, `int64_t` | Integer |
| `Vec3f`, `Vec3d` | 3-vector (velocity, color) |
| `bool` | Active state only |
| `ValueIndex` | Sequential index over all voxels |
| `ValueOnIndex` | Sequential index over **active** voxels |
| `ValueMask` | Topology only (no per-voxel values) |
| `Half` | 16-bit float |
| `Fp4`/`Fp8`/`Fp16`/`FpN` | Quantized float (4/8/16 bits) |
| `Point` | Point counts (for point grids) |
---
## File IO
```cpp
namespace io = nanovdb::io;
auto h = io::readGrid("file.nvdb"); // first grid
auto h2 = io::readGrid("file.nvdb", 2); // grid #2 (0-based)
auto hN = io::readGrid("file.nvdb", "density"); // by name
auto hAll = io::readGrid("file.nvdb", -1); // all → one handle
auto v = io::readGrids("file.nvdb"); // vector<handle>, one per segment
io::writeGrid("out.nvdb", h); // single
io::writeGrid("out.nvdb", h, io::Codec::BLOSC); // compressed
io::writeGrids("out.nvdb", handles); // many
```
---
## Inspecting a grid
```cpp
const auto* grid = handle.grid<float>(); // nullptr if T mismatch
if (!grid) throw std::runtime_error("type mismatch");
grid->gridName(); // const char*
grid->gridClass(); // GridClass enum (LevelSet, FogVolume, etc.)
grid->gridType(); // GridType enum (Float, Vec3f, OnIndex, ...)
grid->voxelSize(); // Vec3d
grid->activeVoxelCount(); // uint64_t
grid->indexBBox(); // CoordBBox
grid->worldBBox(); // BBox<Vec3d>
grid->tree().background(); // T
grid->worldToIndex(p); // Vec3<T>, p is world-space
grid->indexToWorld(p); // Vec3<T>, p is index-space (fractional)
```
---
## Random access (ReadAccessor)
```cpp
// One accessor per thread; not thread-safe; never copy between CPU & GPU
auto acc = grid->getAccessor();
float v = acc.getValue(nanovdb::Coord(1, 2, 3));
bool on = acc.isActive(nanovdb::Coord(1, 2, 3));
const auto* leaf = acc.getNode<nanovdb::NanoLeaf<float>>(); // cached leaf ptr, if needed
// probe* = value/state/leaf-existence in ONE descent (disambiguates the
// two "got background" cases: inactive-vs-no-node)
float pv;
bool active = acc.probeValue(nanovdb::Coord(1,2,3), pv); // pv set; ret = state
const nanovdb::NanoLeaf<float>* lf = acc.probeLeaf(nanovdb::Coord(1,2,3)); // nullptr if no leaf
```
---
## Sequential access (NodeManager)
```cpp
#include <nanovdb/NodeManager.h>
auto nmh = nanovdb::createNodeManager(*grid);
const auto* mgr = nmh.mgr<float>();
auto leafN = mgr->leafCount();
auto lowerN = mgr->lowerCount();
auto upperN = mgr->upperCount();
for (uint64_t i = 0; i < leafN; ++i) {
const auto& leaf = mgr->leaf(i);
for (auto it = leaf.beginValueOn(); it; ++it) {
float v = *it;
auto c = it.getCoord();
}
}
```
GPU. Pass the device-side `NodeManager*` to the kernel and index leaves by
block, voxels within a leaf by thread:
```cpp no-compile
#include <nanovdb/cuda/NodeManager.cuh>
auto dnmh = nanovdb::cuda::createNodeManager<float>(dGrid);
const auto* dmgr = dnmh.template deviceMgr<float>(); // device pointer
// One block per leaf; 64 threads cover the 512 voxels in each leaf.
perLeafKernel<<<dmgr->leafCount(), 64>>>(dmgr); // kernel defined just below
```
The kernel:
```cpp
__global__ void perLeafKernel(const nanovdb::NodeManager<float>* mgr)
{
using LeafT = nanovdb::NanoLeaf<float>;
const uint32_t leafIdx = blockIdx.x;
if (leafIdx >= mgr->leafCount()) return;
const LeafT& leaf = mgr->leaf(leafIdx);
// 512 voxels per leaf (LeafT::SIZE == 1<<3*LOG2DIM == 8^3).
for (uint32_t n = threadIdx.x; n < LeafT::SIZE; n += blockDim.x) {
if (!leaf.isActive(n)) continue;
float v = leaf.getValue(n);
nanovdb::Coord c = leaf.offsetToGlobalCoord(n);
// ... do something with (v, c)
}
}
```
Same methods exist on `lower(i)` and `upper(i)` (internal nodes) — you'll
mostly want `leaf(i)` since that's where actual voxel values live.
Indexed leaf-voxel API (callable from `__hostdev__` code):
| Call | Returns |
|-------------------------------|-------------------------------------------|
| `leaf.getValue(n)` | voxel value at offset `n` (0..511) |
| `leaf.isActive(n)` | active state at offset `n` |
| `leaf.offsetToGlobalCoord(n)` | the global `Coord` for offset `n` |
| `leaf.origin()` | the leaf's `(8,8,8)`-aligned origin Coord |
| `LeafT::SIZE` | `512`, voxels per leaf |
---
## Math primitives (all `__hostdev__`)
```cpp
nanovdb::Coord c(1, 2, 3);
auto c2 = nanovdb::Coord::Floor(nanovdb::Vec3f(1.7f, 2.1f, 3.9f)); // (1, 2, 3)
nanovdb::Vec3f v(1, 2, 3);
auto d = v.dot(v); // 14
auto n = v.length(); // sqrt(14)
auto u = v.normalize();
nanovdb::math::BBox<nanovdb::Vec3f> b(nanovdb::Vec3f(0,0,0),
nanovdb::Vec3f(10,10,10));
b.expand(nanovdb::Vec3f(5,5,5));
b.isInside(nanovdb::Vec3f(1,1,1));
```
---
## Sampling
```cpp
#include <nanovdb/math/SampleFromVoxels.h>
auto acc = grid->getAccessor();
auto sampler = nanovdb::math::createSampler<1>(acc); // 0=nearest, 1=trilinear, 2=triquadratic, 3=tricubic
auto idxPt = grid->worldToIndex(worldPt);
float v = sampler(idxPt);
```
---
## HDDA ray-march
```cpp
#include <nanovdb/math/Ray.h>
#include <nanovdb/math/HDDA.h>
auto acc = grid->getAccessor();
nanovdb::math::Ray<float> wRay({0,0,-5}, {0,0,1});
auto iRay = wRay.worldToIndexF(*grid);
// High-level: ZeroCrossing drives an HDDA and stops at the first sign change.
nanovdb::Coord ijk;
float v, t;
if (nanovdb::math::ZeroCrossing(iRay, acc, ijk, v, t)) { /* hit at ijk */ }
// Low-level: HDDA ctor takes the ray AND the start dimension from getDim.
nanovdb::Coord start(0, 0, 0);
nanovdb::math::HDDA<decltype(iRay), nanovdb::Coord> hdda(iRay, acc.getDim(start, iRay));
while (hdda.step()) {
if (acc.isActive(hdda.voxel())) break; // first active voxel
}
```
---
## GPU: transfer + kernel
```cpp
#include <nanovdb/cuda/DeviceBuffer.h>
#include <nanovdb/cuda/GridHandle.cuh>
auto devH = hostH.copy<nanovdb::cuda::DeviceBuffer>();
devH.deviceUpload(); // REQUIRED: push to device, else deviceGrid()==nullptr
const auto* dGrid = devH.deviceGrid<float>();
__global__ void k(const nanovdb::NanoGrid<float>* g,
const nanovdb::Coord* coords, float* out, int N) {
int tid = blockIdx.x * blockDim.x + threadIdx.x;
if (tid >= N) return;
auto acc = g->getAccessor();
out[tid] = acc.getValue(coords[tid]);
}
k<<<(N+127)/128, 128>>>(dGrid, dCoords, dOut, N);
```
---
## GPU builders
```cpp
// Points → grid (builder-pattern class, not a free function)
nanovdb::tools::cuda::PointsToGrid<nanovdb::Point> p2g(/*voxelSize=*/0.05);
auto ptsH = p2g.getHandle(dPoints, nPts);
const auto* ptsGrid = ptsH.deviceGrid<nanovdb::Point>();
// Mesh → OnIndexGrid (narrow-band). Verts are Vec3f, triangles Vec3i.
const nanovdb::Vec3f* dVerts; size_t vertN;
const nanovdb::Vec3i* dTris; size_t triN;
nanovdb::tools::cuda::MeshToGrid<nanovdb::ValueOnIndex> m2g(
dVerts, vertN, dTris, triN, nanovdb::Map(voxelSize));
m2g.setNarrowBandWidth(3.f);
auto meshH = m2g.getHandle();
const auto* meshGrid = meshH.deviceGrid<nanovdb::ValueOnIndex>();
// MeshToGrid + per-active-voxel UDF sidecar
auto [gridH, udfBuf] = m2g.getHandleAndUDF();
```
## CPU builder
```cpp
// Build a tree on the host, then bake into a packed NanoVDB grid.
nanovdb::tools::build::Grid<float> g(/*background=*/0.0f,
"my_grid", nanovdb::GridClass::FogVolume);
g.setTransform(/*scale=*/0.1); // voxel size
auto acc = g.getAccessor(); // not thread-safe
acc.setValue(nanovdb::Coord(0, 0, 0), 1.0f);
auto h = nanovdb::tools::createNanoGrid(g); // GridHandle<HostBuffer>
```
## Primitive helpers
```cpp
auto h = nanovdb::tools::createLevelSetSphere<float>(
/*radius=*/10.0, /*center=*/{0,0,0}, /*voxelSize=*/0.1);
// also: createLevelSetTorus, createLevelSetBox, createFogVolumeSphere, ...
```
---
## Topological operators (GPU)
```cpp
// Builder classes (NOT free functions); ValueOnIndex grids only. Construct
// from a device grid pointer, getHandle() returns a new GridHandle.
using OnIdx = nanovdb::ValueOnIndex;
auto* g = devH.deviceGrid<OnIdx>();
auto dilatedH = nanovdb::tools::cuda::DilateGrid<OnIdx>(g).getHandle();
auto coarseH = nanovdb::tools::cuda::CoarsenGrid<OnIdx>(g).getHandle();
auto fineH = nanovdb::tools::cuda::RefineGrid<OnIdx>(g).getHandle();
const nanovdb::Mask<3>* leafMask = nullptr; // PruneGrid needs a per-leaf mask
auto prunedH = nanovdb::tools::cuda::PruneGrid<OnIdx>(g, leafMask).getHandle();
auto mergedH = nanovdb::tools::cuda::MergeGrids<OnIdx>(g, g).getHandle();
// signedFloodFill is the exception: a free function, in place, on a scalar
// SDF grid (not OnIndex).
nanovdb::tools::cuda::signedFloodFill(devH.deviceGrid<float>());
```
CPU morphology equivalents live under `nanovdb/util/MorphologyHelpers.h`.
## Re-home sidecar data after a topology change (Injection.cuh)
A topology op renumbers the active-voxel indices, so the old sidecar no
longer matches. Move values across, and find the new voxels:
```cpp
#include <nanovdb/util/cuda/Injection.cuh>
#include <nanovdb/util/cuda/DeviceGridTraits.cuh>
#include <nanovdb/util/cuda/Util.h>
using OnIdx = nanovdb::ValueOnIndex;
const auto* srcGrid = devH.deviceGrid<OnIdx>(); // old grid
const auto* dstGrid = devHandle.deviceGrid<OnIdx>(); // new (e.g. dilated) grid
const float* oldSidecar = nullptr;
float* newSidecar = nullptr; // size = new activeVoxelCount(+1)
// Copy overlapping values (non-overlap dst voxels left untouched):
auto srcLeaves = nanovdb::util::cuda::DeviceGridTraits<OnIdx>::getTreeData(srcGrid).mNodeCount[0];
nanovdb::util::cuda::operatorKernel<
nanovdb::util::cuda::InjectGridDataFunctor<OnIdx, float>>
<<<srcLeaves, 256>>>(srcGrid, dstGrid, oldSidecar, newSidecar);
// Per-leaf overlap mask (new voxels = dstMask & ~overlap):
auto dstLeaves = nanovdb::util::cuda::DeviceGridTraits<OnIdx>::getTreeData(dstGrid).mNodeCount[0];
auto maskBuf = nanovdb::cuda::DeviceBuffer::create(dstLeaves*sizeof(nanovdb::Mask<3>), nullptr, false);
auto* overlap = static_cast<nanovdb::Mask<3>*>(maskBuf.deviceData());
unsigned nt = 128, nb = (dstLeaves + nt - 1) / nt;
nanovdb::util::cuda::lambdaKernel<<<nb, nt>>>(dstLeaves,
nanovdb::util::cuda::InjectGridMaskFunctor<OnIdx>(), srcGrid, dstGrid, overlap);
```
Simpler: pre-fill `newSidecar` with a sentinel; slots still holding it after
the data inject are the new voxels.
---
## Stats, validation & integrity
Nodes cache bbox/min/max/avg per node. Mutating values in place leaves them
**stale** — which silently breaks HDDA space-skipping and ray-tracing.
Recompute after any manual edit.
```cpp
#include <nanovdb/tools/GridStats.h> // CPU
// #include <nanovdb/tools/cuda/GridStats.cuh> // GPU (device grid + stream)
nanovdb::NanoGrid<float>* mutableGrid = nullptr; // non-const: it gets written
nanovdb::tools::updateGridStats(mutableGrid, nanovdb::tools::StatsMode::All);
// StatsMode: Disable | BBox | MinMax | All(=Default)
// Extrema over an index-space box, computed directly (no cache):
auto ex = nanovdb::tools::getExtrema(*grid, nanovdb::CoordBBox(nanovdb::Coord(-10), nanovdb::Coord(10)));
float lo = ex.min(), hi = ex.max();
```
Certify a grid after load / build / device transfer:
```cpp
#include <nanovdb/tools/GridChecksum.h>
#include <nanovdb/tools/GridValidator.h>
nanovdb::NanoGrid<float>* mutableGrid = nullptr; // non-const: edited grid
bool good = nanovdb::tools::validateChecksum(grid, nanovdb::CheckMode::Full);
nanovdb::tools::updateChecksum(mutableGrid, nanovdb::CheckMode::Full); // after a legit edit
bool valid = nanovdb::tools::isValid(grid, nanovdb::CheckMode::Full, /*verbose=*/false);
char err[(uint32_t)nanovdb::CheckMode::StrLen] = {}; // CheckMode is scoped → cast
nanovdb::tools::checkGrid(grid, err, nanovdb::CheckMode::Full); // err="" if valid
// CheckMode: Disable/Empty | Half/Partial/Default | Full
// GPU variants: tools/cuda/GridChecksum.cuh, tools/cuda/GridValidator.cuh
```
---
## IndexGrid & VBM
```cpp no-compile
// Build a ValueOnIndex grid from an OpenVDB grid
auto h = nanovdb::tools::createNanoGrid<openvdb::FloatGrid,
nanovdb::ValueOnIndex>(*ovGrid);
// Or from a mesh on the GPU (already returns OnIndexGrid)
nanovdb::tools::cuda::MeshToGrid<nanovdb::ValueOnIndex> mesher(dV, vN, dT, tN, nanovdb::Map(vs));
auto devH = mesher.getHandle();
const auto* g = devH.deviceGrid<nanovdb::ValueOnIndex>();
// External attribute array — same length as active voxel count
size_t N = g->activeVoxelCount();
thrust::device_vector<float> attr(N);
// Read attribute at a coordinate
auto acc = g->getAccessor();
uint64_t idx = acc.getValue(c); // index into attr
float a = attr[idx];
// VBM: SIMT-friendly block iteration (Log2BlockWidth >= 6, i.e. >=64/block)
auto vbmH = nanovdb::tools::buildVoxelBlockManager<6>(g);
// see tools/cuda/VoxelBlockManager.cuh for the GPU decode pattern
```
ChannelAccessor — read the sidecar by attribute, not by index (one typed
call instead of getValue(ijk) → array[idx]). Note: IndexT defaults to
`ValueIndex`, so spell out `ValueOnIndex`:
```cpp
__global__ void readChan(const nanovdb::NanoGrid<nanovdb::ValueOnIndex>* g,
float* dValues, nanovdb::Coord c, float* out) {
nanovdb::ChannelAccessor<float, nanovdb::ValueOnIndex> ca(*g, dValues);
*out = ca.getValue(c); // attribute value (not the index)
// ca(c) same; ca.getIndex(c) for the raw index; ca.probeValue(c, v) for state+value
// ctor (*g, channelID) instead reads an internal blind-data channel
}
```
Bake a sidecar back into a value grid (round-trip partner of the split;
needed by value-typed tools like `signedFloodFill`):
```cpp
#include <nanovdb/tools/cuda/IndexToGrid.cuh>
const auto* idxGrid = devH.deviceGrid<nanovdb::ValueOnIndex>();
const float* dValues = nullptr; // per-active-voxel sidecar
auto floatH = nanovdb::tools::cuda::indexToGrid<float>(idxGrid, dValues);
const auto* floatGrid = floatH.deviceGrid<float>(); // values now in-grid
```
---
## OpenVDB ↔ NanoVDB
```cpp openvdb
#include <nanovdb/tools/CreateNanoGrid.h>
#include <nanovdb/tools/NanoToOpenVDB.h>
// OpenVDB → NanoVDB (optionally with type change)
auto h = nanovdb::tools::createNanoGrid<openvdb::FloatGrid, float>(*ovGrid);
auto h_half = nanovdb::tools::createNanoGrid<openvdb::FloatGrid,
nanovdb::Half>(*ovGrid);
// NanoVDB → OpenVDB
auto ovGrid = nanovdb::tools::nanoToOpenVDB<float>(handle);
```
---
## Blind data (attached arrays)
Read (host or device — `__hostdev__`):
```cpp
uint32_t n = grid->blindDataCount();
const nanovdb::GridBlindMetaData& meta = grid->blindMetaData(0);
// meta.mValueCount, mValueSize, mDataType, mSemantic, mDataClass, mName
int id = grid->findBlindDataForSemantic(nanovdb::GridBlindDataSemantic::PointPosition);
// or: int id = grid->findBlindData("name");
const nanovdb::Vec3f* pos = grid->getBlindData<nanovdb::Vec3f>(id); // nullptr if T mismatch
```
Attach on the GPU (returns a NEW handle with the array appended; fixes the
checksum):
```cpp
#include <nanovdb/tools/cuda/AddBlindData.cuh>
const float* dChannel = nullptr; uint64_t valueCount = 0;
auto h = nanovdb::tools::cuda::addBlindData(
dGrid, dChannel, valueCount,
nanovdb::GridBlindDataClass::ChannelArray,
nanovdb::GridBlindDataSemantic::Unknown, "my_channel");
const auto* g = h.deviceGrid<float>();
```
---
## Common idioms
```cpp
// Iterate active voxels in a grid (CPU, leaf-by-leaf)
auto nmh = nanovdb::createNodeManager(*grid);
auto* mgr = nmh.mgr<float>();
for (uint64_t L = 0; L < mgr->leafCount(); ++L) {
const auto& leaf = mgr->leaf(L);
for (auto it = leaf.beginValueOn(); it; ++it) {
// *it is the value, it.getCoord() is the ijk
}
}
// Sample a grid at a list of world-space points (pts is a std::vector<Vec3f>)
auto sampler = nanovdb::math::createSampler<1>(acc);
std::vector<float> sampled;
for (const auto& w : pts)
sampled.push_back(sampler(grid->worldToIndex(w)));
// Save a level-set sphere to disk
auto h = nanovdb::tools::createLevelSetSphere<float>(10.0, {0,0,0}, 0.1);
nanovdb::io::writeGrid("sphere.nvdb", h);
```
---
## Pitfalls
- **Type mismatch returns nullptr.** Always check `handle.grid<T>()`.
- **Accessors are not thread-safe.** One per thread/warp.
- **Accessors don't cross CPU/GPU.** Build inside the kernel.
- **Topology is frozen.** To "add a voxel" you must rebuild the grid.
- **32-byte alignment.** Don't allocate the buffer yourself with
`new char[N]` — use `HostBuffer` or `DeviceBuffer`, which guarantee
alignment.
- **No runtime `std::` in `__hostdev__` code.** Containers, iostreams,
allocators, `std::sort` etc. won't work on device. For type traits and
compile-time helpers, NanoVDB uses its own `nanovdb::util::is_same` /
`util::enable_if` / `util::conditional` / `util::remove_const` (in
`util/Util.h`) — these compile under C++11 across compilers and don't
need `--expt-relaxed-constexpr`. Where NanoVDB needs more, it pulls from
CCCL's `cuda::std::` (e.g. `cuda::std::numeric_limits` in `math/Math.h`).
Plain `std::` compile-time facilities work with relaxed constexpr but
NanoVDB's own headers avoid them for portability.
- **HDDA is essential for sparse ray-march.** Without it, you're stepping
one voxel at a time through empty space.
---
## Tools / binaries
| Tool | Purpose |
|---------------------|-------------------------------------------|
| `nanovdb_print` | Print metadata for grids in a `.nvdb` |
| `nanovdb_convert` | OpenVDB ↔ NanoVDB on the CLI |
| `nanovdb_validate` | ABI / format validation |
Run with `--help` for options.
File diff suppressed because it is too large Load Diff
+3
View File
@@ -0,0 +1,3 @@
# Transient harness outputs
gen/
build/
+87
View File
@@ -0,0 +1,87 @@
cmake_minimum_required(VERSION 3.20)
project(teachme_tests LANGUAGES CXX CUDA)
# Compile-only gate: object libraries compile each generated translation unit
# to an object file without linking. That is exactly what we want — catch API
# errors at compile time; the fixture's extern stand-ins never need to resolve.
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CUDA_STANDARD 17)
if(NOT DEFINED CMAKE_CUDA_ARCHITECTURES)
set(CMAKE_CUDA_ARCHITECTURES 89) # RTX 6000 Ada; any arch suffices to compile
endif()
# NanoVDB headers pull in CUDA lambdas / relaxed-constexpr code paths.
# diag 177 = "function declared but never referenced": every wrapped block
# becomes an unreferenced static function on purpose, so silence it.
string(APPEND CMAKE_CUDA_FLAGS
" --extended-lambda --expt-relaxed-constexpr --diag-suppress 177")
# NanoVDB include root: this file is doc/nanovdb/TEACHME/test/CMakeLists.txt, so
# the repo root is four dirs up. Headers are at nanovdb/nanovdb/NanoVDB.h, so the
# include root is the repo's `nanovdb/` subdir (a sibling of `doc/`).
get_filename_component(_teachme_dir "${CMAKE_CURRENT_SOURCE_DIR}" DIRECTORY) # TEACHME
get_filename_component(_docnv_dir "${_teachme_dir}" DIRECTORY) # doc/nanovdb
get_filename_component(_doc_dir "${_docnv_dir}" DIRECTORY) # doc
get_filename_component(REPO_ROOT "${_doc_dir}" DIRECTORY) # repo root
set(NANOVDB_INCLUDE_ROOT "${REPO_ROOT}/nanovdb")
set(OPENVDB_INCLUDE_ROOT "${REPO_ROOT}/openvdb")
message(STATUS "NanoVDB include root: ${NANOVDB_INCLUDE_ROOT}")
# Generated TUs + manifest live in ${GEN_DIR} (passed in, defaults to ./gen).
if(NOT DEFINED GEN_DIR)
set(GEN_DIR "${CMAKE_CURRENT_SOURCE_DIR}/gen")
endif()
set(_manifest "${GEN_DIR}/manifest.txt")
if(NOT EXISTS "${_manifest}")
message(FATAL_ERROR "No manifest at ${_manifest}. Run the extractor first "
"(see run.sh).")
endif()
file(STRINGS "${_manifest}" _lines)
set(_core_srcs "")
set(_ovdb_srcs "")
foreach(_line IN LISTS _lines)
if(_line STREQUAL "")
continue()
endif()
separate_arguments(_parts UNIX_COMMAND "${_line}")
list(GET _parts 0 _stem)
list(GET _parts 1 _needs_ovdb)
set(_src "${GEN_DIR}/${_stem}.cu")
if(_needs_ovdb STREQUAL "1")
list(APPEND _ovdb_srcs "${_src}")
else()
list(APPEND _core_srcs "${_src}")
endif()
endforeach()
# Core (NanoVDB-only) translation units.
if(_core_srcs)
add_library(teachme_blocks OBJECT ${_core_srcs})
target_include_directories(teachme_blocks PRIVATE
"${NANOVDB_INCLUDE_ROOT}" "${GEN_DIR}" "${CMAKE_CURRENT_SOURCE_DIR}")
list(LENGTH _core_srcs _n_core)
message(STATUS "teachme_blocks: ${_n_core} core TU(s)")
endif()
# OpenVDB-dependent translation units — only if OpenVDB is available.
if(_ovdb_srcs)
find_package(OpenVDB QUIET)
if(OpenVDB_FOUND)
add_library(teachme_blocks_openvdb OBJECT ${_ovdb_srcs})
target_compile_definitions(teachme_blocks_openvdb PRIVATE NANOVDB_USE_OPENVDB)
target_include_directories(teachme_blocks_openvdb PRIVATE
"${NANOVDB_INCLUDE_ROOT}" "${OPENVDB_INCLUDE_ROOT}"
"${GEN_DIR}" "${CMAKE_CURRENT_SOURCE_DIR}")
target_link_libraries(teachme_blocks_openvdb PRIVATE OpenVDB::openvdb)
list(LENGTH _ovdb_srcs _n_ovdb)
message(STATUS "teachme_blocks_openvdb: ${_n_ovdb} OpenVDB TU(s)")
else()
list(LENGTH _ovdb_srcs _n_ovdb)
message(WARNING "OpenVDB not found — skipping ${_n_ovdb} OpenVDB TU(s). "
"These blocks are reference-checked but not compiled.")
endif()
endif()
+89
View File
@@ -0,0 +1,89 @@
# TEACHME test harness
Keeps the code in the lesson honest. Two gates:
1. **Reference validation** (`teachme_tests.py check-refs`) — every
`#include <...>` in a code block must resolve to a real header in the
NanoVDB tree (or be explicitly exempt). Catches stale-path rot — e.g. a
header that was renamed out from under the docs.
2. **Compilation** (`teachme_tests.py extract` + CMake) — code blocks tagged
as compilable are extracted into `.cu` translation units and compiled
(compile-only, `-c`) against the real NanoVDB headers. Catches API-shape
drift — wrong namespace, a free function that's actually a class, a
renamed method, a changed signature.
Run both with `./run.sh` from this directory.
## Why compile-only
The harness compiles (`nvcc -c`) but does not link or run. NanoVDB is
header-only and the goal is to catch *API* errors (does this call name a
real symbol with a compatible signature?), which all surface at compile
time. Skipping the link step means a fixture header can declare the
"given" symbols a fragment assumes (`grid`, `acc`, device pointers) as
`extern` globals — unresolved at link, but fully type-checked at compile.
Everything compiles as CUDA (`.cu`) because even host-only examples pull
in CUDA code paths (under `__CUDACC__`) when they include headers like
`NodeManager.h` or `io/IO.h`.
## Block tagging
Code blocks are fenced with an info string: a language followed by
whitespace-separated tags, e.g.
```cpp compile
```cpp compile-global continuation
```cpp skip-refs
**Opt-out model:** every `cpp` / `cu` block is compiled by default. A block
is wrapped in a function body unless it opens a top-level definition
(`__global__`, `template`, `struct`, `class`, `namespace`, `enum`, `using`,
`typedef`), in which case it is auto-emitted at file scope. Tags only adjust
the defaults:
| Tag | Meaning |
|----------------|------------------------------------------------------------------------------------------|
| *(none)* | Compiled. Auto: file scope if it opens a top-level definition, else function-wrapped. |
| `no-compile` | Reference-check includes only; do **not** compile. For illustrative/partial fragments. |
| `global` | Force file-scope emission (override the auto-detection). |
| `main` | Block is already a complete program (its own `main`); compile verbatim. |
| `continuation` | Append this block to the previous block's translation unit (shares its symbols). |
| `openvdb` | Block needs OpenVDB; compiled only when OpenVDB is found. |
| `skip-refs` | Do **not** validate includes in this block — for intentionally-stale counter-examples. |
A translation unit is a sequence of segments. A non-`continuation` block
starts a new TU; each following `continuation` block adds a segment. At emit
time, file-scope segments go to file scope (in order) and function segments
each get their own wrapper function — so a kernel definition followed by a
`continuation` launch line compose into one valid TU.
Every TU includes `fixture.cuh`, which pulls in the common NanoVDB headers
(a deliberately broad "kitchen-sink" set, so terse fragments compile without
restating includes) and declares the stand-in "given" symbols (`grid`,
`acc`, device pointers, etc.) as `extern`. Including the heavy `tools/cuda/*`
headers in every TU makes per-TU compilation slower; that's an accepted
trade for letting fragments stay terse. If the suite grows large, switch to
per-TU includes driven by each block's own `#include` lines.
## Coverage
`check-refs` reports how many code blocks are compile-tagged vs
reference-only, so the fraction of the lesson that is actually
compile-gated is visible (and ideally climbing).
## Exemptions
Includes that are intentionally not real are skipped by the validator:
- ellipsis placeholders such as `#include <openvdb/...>`
- any include inside a `skip-refs` block (e.g. the deprecated
`nanovdb/util/cuda/CudaDeviceBuffer.h` shown as a "this path is stale"
counter-example)
## Files
- `teachme_tests.py` — the extractor + reference validator
- `fixture.cuh` — common headers + `extern` stand-in symbols
- `CMakeLists.txt` — compiles the generated TUs
- `run.sh` — driver: check-refs → extract → cmake configure+build → report
+92
View File
@@ -0,0 +1,92 @@
// fixture.cuh — common headers + stand-in "given" symbols for TEACHME
// code-block compilation.
//
// Lesson fragments assume context: "you have a grid", "you have device
// points", etc. The harness compiles each block (compile-only, never linked)
// with this header in scope, so those assumed symbols need declarations with
// the right *types* — enough for the compiler to type-check member calls and
// catch API drift. They are declared `extern` (no definitions): a block that
// declares its own local `grid`/`acc`/etc. simply shadows the extern, and
// since we never link, the unresolved externs cost nothing.
//
// Grow this file as the tagging pass surfaces more assumed names. Keep the
// types matching how the lesson uses each symbol.
#pragma once
// ---- common headers (header guards make re-includes in blocks harmless) ----
// Core read path + IO + math + CUDA.
#include <nanovdb/NanoVDB.h>
#include <nanovdb/GridHandle.h>
#include <nanovdb/HostBuffer.h>
#include <nanovdb/NodeManager.h>
#include <nanovdb/io/IO.h>
#include <nanovdb/math/Ray.h>
#include <nanovdb/math/HDDA.h>
#include <nanovdb/math/SampleFromVoxels.h>
#include <nanovdb/math/Stencils.h>
#include <nanovdb/cuda/DeviceBuffer.h>
#include <nanovdb/cuda/GridHandle.cuh>
#include <nanovdb/cuda/NodeManager.cuh>
// Commonly-assumed "tools" headers — ambient so terse fragments compile
// without restating the include. (OpenVDB-conversion headers are NOT here;
// blocks that need them are tagged `openvdb` and include them explicitly.)
#include <nanovdb/tools/CreatePrimitives.h>
#include <nanovdb/tools/CreateNanoGrid.h>
#include <nanovdb/tools/GridBuilder.h>
#include <nanovdb/tools/VoxelBlockManager.h>
#include <nanovdb/tools/cuda/PointsToGrid.cuh>
#include <nanovdb/tools/cuda/MeshToGrid.cuh>
#include <nanovdb/tools/cuda/DilateGrid.cuh>
#include <nanovdb/tools/cuda/CoarsenGrid.cuh>
#include <nanovdb/tools/cuda/RefineGrid.cuh>
#include <nanovdb/tools/cuda/PruneGrid.cuh>
#include <nanovdb/tools/cuda/MergeGrids.cuh>
#include <nanovdb/tools/cuda/SignedFloodFill.cuh>
#include <cstdint>
#include <cstddef>
#include <cmath>
#include <string>
#include <vector>
#include <iostream>
#include <stdexcept>
// ---- stand-in "given" symbols (extern: declared, never defined) ----
// Handles
extern nanovdb::GridHandle<nanovdb::HostBuffer> handle;
extern nanovdb::GridHandle<nanovdb::HostBuffer> hostH;
extern nanovdb::GridHandle<nanovdb::cuda::DeviceBuffer> devH;
extern nanovdb::GridHandle<nanovdb::cuda::DeviceBuffer> devHandle;
extern std::vector<nanovdb::GridHandle<nanovdb::HostBuffer>> handles;
// Grids (host + device pointers into a float grid)
extern const nanovdb::NanoGrid<float>* grid;
extern const nanovdb::NanoGrid<float>* dGrid;
// A read accessor + sampler over the float grid (fragments often assume these
// already exist from a prior step).
extern nanovdb::DefaultReadAccessor<float> acc;
// Device point / coord / output buffers used by kernel launches.
// dPoints is non-const: PointsToGrid<Point> writes input points into the
// grid's blind data via getPoint<element_type>(...) = ..., which requires a
// non-const element type.
extern nanovdb::Vec3f* dPoints;
extern std::size_t nPts;
extern const nanovdb::Vec3f* dPts;
extern const nanovdb::Vec3f* worldPts;
extern const nanovdb::Coord* dCoords;
extern const nanovdb::Coord* coords;
extern float* dOut;
extern float* out;
extern int N;
// Host-side query containers + single query points
extern std::vector<nanovdb::Vec3f> pts;
extern std::vector<nanovdb::Coord> queries;
extern nanovdb::Vec3f worldPt; // a single world-space query point
extern nanovdb::Vec3f p; // generic world-space point
// Misc scalars
extern double voxelSize;
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env bash
# Drive the TEACHME test harness: reference-check -> extract -> compile.
#
# Usage: ./run.sh
# Exits nonzero if any reference is dangling or any compile-tagged block
# fails to compile.
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
TEACHME="$(dirname "$HERE")"
GEN="$HERE/gen"
BUILD="$HERE/build"
MD=(
"$TEACHME/nanovdb_user_lesson.md"
"$TEACHME/nanovdb_user_cheatsheet.md"
"$TEACHME/README.md"
)
echo "==> [1/3] reference check"
python3 "$HERE/teachme_tests.py" check-refs "${MD[@]}"
echo "==> [2/3] extract compile-tagged blocks"
rm -rf "$GEN"
mkdir -p "$GEN"
python3 "$HERE/teachme_tests.py" extract "${MD[@]}" --out "$GEN"
n_tu=$(find "$GEN" -name '*.cu' | wc -l | tr -d ' ')
if [ "$n_tu" -eq 0 ]; then
echo "==> [3/3] compile: no compile-tagged blocks yet — nothing to build"
echo "PASS (reference check only)"
exit 0
fi
echo "==> [3/3] compile $n_tu translation unit(s)"
mkdir -p "$BUILD"
cmake -S "$HERE" -B "$BUILD" -DGEN_DIR="$GEN" -DCMAKE_BUILD_TYPE=Release \
> "$BUILD/configure.log" 2>&1 || { cat "$BUILD/configure.log"; exit 1; }
# Cap parallelism: each TU pulls in the kitchen-sink fixture (most of NanoVDB's
# heavy headers), so nvcc is memory-hungry. Default to all cores locally; CI
# sets TEACHME_JOBS lower so the parallel nvcc processes stay within the
# runner's RAM (8 unbounded jobs OOM a 32 GB runner).
jobs="${TEACHME_JOBS:-$(nproc)}"
# Keep going (--keep-going / -k) so every failing TU is reported, not just the
# first. Capture cmake's status directly (don't pipe through tee, which would
# mask it).
set +e
cmake --build "$BUILD" --parallel "$jobs" -- -k > "$BUILD/build.log" 2>&1
status=$?
set -e
cat "$BUILD/build.log"
if [ "$status" -eq 0 ]; then
echo "PASS: all $n_tu translation unit(s) compiled"
else
echo "FAIL: one or more translation units did not compile (see above)"
exit 1
fi
+319
View File
@@ -0,0 +1,319 @@
#!/usr/bin/env python3
"""TEACHME lesson test harness: reference validation + block extraction.
Two subcommands:
check-refs <md...> Validate every #include in code blocks resolves
against the NanoVDB include root. Reports
compile-coverage. Exits nonzero on dangling refs.
extract <md...> --out <dir> Extract compile-tagged code blocks into .cu
translation units under <dir>, ready for CMake.
See README.md for the block tagging convention.
"""
import argparse
import os
import re
import sys
# Repo layout: this file is at doc/nanovdb/TEACHME/test/teachme_tests.py, so the
# repo root is four levels up. The NanoVDB include root (the dir that contains
# the `nanovdb/` header subdir) is the repo's `nanovdb/`; OpenVDB's is `openvdb/`.
HERE = os.path.dirname(os.path.abspath(__file__))
REPO_ROOT = os.path.abspath(os.path.join(HERE, "..", "..", "..", ".."))
NANOVDB_INCLUDE_ROOT = os.path.join(REPO_ROOT, "nanovdb")
OPENVDB_INCLUDE_ROOT = os.path.join(REPO_ROOT, "openvdb")
INCLUDE_ROOTS = [NANOVDB_INCLUDE_ROOT, OPENVDB_INCLUDE_ROOT]
FENCE_RE = re.compile(r"^```(\w+)?(.*)$")
INCLUDE_RE = re.compile(r'^\s*#\s*include\s*<([^>]+)>')
# Inline-code spans in prose that look like header paths, e.g.
# `nanovdb/tools/GridBuilder.h`. Used to catch stale "Read: ..." references.
PROSE_PATH_RE = re.compile(r'`((?:nanovdb|openvdb)/[\w./]+\.(?:h|cuh|cc|cpp))`')
# A line carrying this marker is exempt from prose-path checking (for
# intentionally-stale counter-examples).
STALE_OK = "teachme:stale-ok"
def resolves(inc):
"""True if an include path exists under any include root."""
return any(os.path.isfile(os.path.join(r, inc)) for r in INCLUDE_ROOTS)
# Languages whose blocks we care about.
CODE_LANGS = {"cpp", "cu", "c++"}
# Opt-OUT model: every cpp/cu block is compiled unless tagged `no-compile`.
# A line that begins a file-scope definition forces file-scope emission even
# without an explicit `global` tag.
FILE_SCOPE_RE = re.compile(r'^\s*(template\b|struct\b|class\b|namespace\b|'
r'__global__\b|enum\b|using\b|typedef\b)')
class Block:
__slots__ = ("path", "start_line", "lang", "tags", "lines")
def __init__(self, path, start_line, lang, tags):
self.path = path
self.start_line = start_line
self.lang = lang
self.tags = tags
self.lines = []
@property
def text(self):
return "".join(self.lines)
def parse_blocks(path):
"""Yield Block objects for every fenced code block in a markdown file."""
blocks = []
with open(path, encoding="utf-8") as f:
lines = f.readlines()
i = 0
in_block = None
for lineno, line in enumerate(lines, 1):
m = FENCE_RE.match(line)
if m:
if in_block is None:
lang = (m.group(1) or "").lower()
tags = set(m.group(2).split())
in_block = Block(path, lineno, lang, tags)
else:
blocks.append(in_block)
in_block = None
continue
if in_block is not None:
in_block.lines.append(line)
if in_block is not None:
sys.stderr.write(f"{path}: unterminated code fence opened at "
f"line {in_block.start_line}\n")
return blocks
def is_exempt_include(inc):
"""Includes that are intentionally not real headers."""
if "..." in inc: # ellipsis placeholder, e.g. openvdb/...
return True
return False
def scan_prose_paths(path):
"""Yield (lineno, token) for header-path references in prose inline code,
skipping lines inside fenced code blocks and lines marked stale-ok."""
with open(path, encoding="utf-8") as f:
lines = f.readlines()
in_fence = False
for lineno, line in enumerate(lines, 1):
if FENCE_RE.match(line) and line.lstrip().startswith("```"):
in_fence = not in_fence
continue
if in_fence or STALE_OK in line:
continue
for tok in PROSE_PATH_RE.findall(line):
yield lineno, tok
def cmd_check_refs(args):
code_blocks = 0
compiled_blocks = 0
dangling = [] # (path, lineno, inc, kind)
for path in args.md:
# 1) #include lines inside code blocks
for b in parse_blocks(path):
if b.lang not in CODE_LANGS:
continue
code_blocks += 1
if "no-compile" not in b.tags:
compiled_blocks += 1
if "skip-refs" in b.tags:
continue
for off, line in enumerate(b.lines):
im = INCLUDE_RE.match(line)
if not im:
continue
inc = im.group(1).strip()
if is_exempt_include(inc):
continue
if not resolves(inc):
dangling.append((path, b.start_line + 1 + off, inc, "include"))
# 2) header-path references in prose ("Read: `nanovdb/.../X.h`")
for lineno, tok in scan_prose_paths(path):
if not resolves(tok):
dangling.append((path, lineno, tok, "prose"))
if dangling:
sys.stderr.write("DANGLING REFERENCES:\n")
for path, lineno, ref, kind in dangling:
rel = os.path.relpath(path)
sys.stderr.write(f" {rel}:{lineno}: [{kind}] {ref!r} does not "
f"resolve under any include root\n")
pct = (100.0 * compiled_blocks / code_blocks) if code_blocks else 0.0
print(f"code blocks: {code_blocks} compiled: {compiled_blocks} "
f"({pct:.0f}%) no-compile: {code_blocks - compiled_blocks}")
if dangling:
print(f"FAIL: {len(dangling)} dangling reference(s)")
return 1
print("OK: all references resolve")
return 0
def split_block(block):
"""Split one block's lines into ordered (kind, lines) sub-segments.
`kind` is 'global' (file scope) or 'fn' (function body). Explicit `global`
/ `main` tags force the whole block. Otherwise we brace-walk: a top-level
definition (matched by FILE_SCOPE_RE at brace depth 0, e.g. a __global__
kernel) becomes a file-scope chunk spanning until its braces balance;
everything else at depth 0 (launches, statements) becomes a function chunk.
This lets a single block mix a kernel definition and its launch.
"""
if "global" in block.tags:
return [("global", block.lines)]
if "main" in block.tags:
return [("main", block.lines)]
segs = []
fn_buf = []
lines = block.lines
i, n = 0, len(lines)
def flush_fn():
if fn_buf:
segs.append(("fn", fn_buf[:]))
fn_buf.clear()
while i < n:
line = lines[i]
stripped = line.lstrip()
if stripped.startswith("#"): # preprocessor: neutral (hoisted later)
fn_buf.append(line)
i += 1
continue
if FILE_SCOPE_RE.match(line): # start of a top-level definition
flush_fn()
chunk = []
depth = 0
saw_brace = False
while i < n:
l = lines[i]
chunk.append(l)
i += 1
depth += l.count("{") - l.count("}")
if "{" in l:
saw_brace = True
if saw_brace and depth <= 0:
break
if not saw_brace and l.rstrip().endswith(";"):
break # brace-less decl, e.g. `using X = ...;`
segs.append(("global", chunk))
continue
fn_buf.append(line) # ordinary statement → function body
i += 1
flush_fn()
return segs if segs else [("fn", lines)]
def cmd_extract(args):
os.makedirs(args.out, exist_ok=True)
# A TU has a list of (kind, lines, line_no) sub-segments + an openvdb flag.
tus = []
current = None
for path in args.md:
for b in parse_blocks(path):
if b.lang not in CODE_LANGS:
continue
# Opt-out: compile everything except blocks tagged no-compile.
if "no-compile" in b.tags:
continue
subsegs = [(k, lines, b.start_line) for (k, lines) in split_block(b)]
cont = "continuation" in b.tags
if cont and current is not None:
current["segs"].extend(subsegs)
if "openvdb" in b.tags:
current["openvdb"] = True
else:
current = {
"segs": list(subsegs),
"openvdb": "openvdb" in b.tags,
"src": os.path.basename(path),
"line": b.start_line,
}
tus.append(current)
manifest = []
for n, tu in enumerate(tus):
stem = f"tu_{n:03d}_{os.path.splitext(tu['src'])[0]}_{tu['line']}"
fname = os.path.join(args.out, stem + ".cu")
# Hoist all preprocessor directives (#include/#define/#pragma) to file
# scope, deduped in first-seen order, so function-wrapped segments
# never carry an #include into block scope (illegal for headers that
# define static __device__ functions).
hoisted = []
seen_hoist = set()
seg_render = [] # (kind, body_lines, line)
for kind, lines, line in tu["segs"]:
body = []
for ln in lines:
if ln.lstrip().startswith("#"):
if ln not in seen_hoist:
seen_hoist.add(ln)
hoisted.append(ln)
else:
body.append(ln)
if body:
seg_render.append((kind, body, line))
out = ['#include "fixture.cuh"\n']
out += hoisted
out.append("\n")
fn_count = 0
for kind, body, line in seg_render:
out.append(f"// --- from {tu['src']}:{line} ---\n")
if kind in ("global", "main"):
out += body
else: # fn
out.append(f"static void _seg_{fn_count}() {{\n")
out += body
out.append("\n}\n")
fn_count += 1
out.append("\n")
with open(fname, "w", encoding="utf-8") as f:
f.writelines(out)
manifest.append((stem, tu["openvdb"]))
# Emit a manifest CMake can read: <stem> <needs_openvdb 0|1>
with open(os.path.join(args.out, "manifest.txt"), "w") as f:
for stem, ovdb in manifest:
f.write(f"{stem} {1 if ovdb else 0}\n")
n_ovdb = sum(1 for _, o in manifest if o)
print(f"extracted {len(manifest)} translation unit(s) "
f"({n_ovdb} need OpenVDB) into {args.out}")
return 0
def main():
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
sub = ap.add_subparsers(dest="cmd", required=True)
p1 = sub.add_parser("check-refs", help="validate #include references")
p1.add_argument("md", nargs="+", help="markdown files")
p1.set_defaults(func=cmd_check_refs)
p2 = sub.add_parser("extract", help="extract compile-tagged blocks to .cu")
p2.add_argument("md", nargs="+", help="markdown files")
p2.add_argument("--out", required=True, help="output directory")
p2.set_defaults(func=cmd_extract)
args = ap.parse_args()
sys.exit(args.func(args))
if __name__ == "__main__":
main()
+4
View File
@@ -6,3 +6,7 @@ Welcome to the NanoVDB documentation page.
* @subpage NanoVDB_FAQ
* @subpage NanoVDB_SourceTree
* @subpage NanoVDB_HelloWorld
For a hands-on introduction, point an LLM coding agent (Claude Code, Cursor, or
similar) at the [TEACHME lessons](https://github.com/AcademySoftwareFoundation/openvdb/tree/master/doc/nanovdb/TEACHME)
and ask it to teach you the NanoVDB API interactively.
+16
View File
@@ -31,6 +31,7 @@ option(NANOVDB_BUILD_PYTHON_MODULE "Build the nanovdb Python module" OFF)
option(NANOVDB_USE_INTRINSICS "Build with hardware intrinsics support" OFF)
option(NANOVDB_USE_CUDA "Build with CUDA support" OFF)
option(NANOVDB_CUDA_KEEP_PTX "Keep CUDA PTX" OFF)
option(NANOVDB_CUDA_WERROR "Treat all NVCC diagnostics as errors" ON)
option(NANOVDB_USE_OLD_ACCESSOR "Use the old (buggy) ReadAccessor instead of the new fixed one" OFF)
option(NANOVDB_USE_OPENVDB "Build with OpenVDB support" OFF)
@@ -96,6 +97,21 @@ if(NANOVDB_USE_CUDA)
set(CMAKE_CUDA_FLAGS "${NANOVDB_CUDA_EXTENDED_LAMBDA} -use_fast_math -lineinfo ${CMAKE_CUDA_FLAGS}")
if(NANOVDB_CUDA_WERROR OR OPENVDB_CXX_STRICT)
# Device-side analog of the host -Werror configured in OpenVDBCXX.cmake:
# every nvcc front-end diagnostic becomes an error. NANOVDB_CUDA_WERROR
# enables it independently of the host-side strict flags.
set(CMAKE_CUDA_FLAGS "--Werror=all-warnings ${CMAKE_CUDA_FLAGS}")
endif()
if(WIN32)
# NVCC analog of the /wd4251 and /wd4275 suppressions in OpenVDBCXX.cmake:
# it's not possible to use STL types in DLL interfaces in a portable and
# reliable way, so the dll-interface diagnostics (#1394 and #1388) fire on
# every include of the OpenVDB core headers from a CUDA source.
set(CMAKE_CUDA_FLAGS "--diag-suppress=1388,1394 ${CMAKE_CUDA_FLAGS}")
endif()
# workaround for win32 bug when nvcc "--keep" is used.
if(WIN32)
if(NANOVDB_CUDA_KEEP_PTX)
+208 -51
View File
@@ -8,8 +8,9 @@
\date January 8, 2020
\brief Defines GridHandle, which manages a host, and possibly a device,
memory buffer containing one or more NanoVDB grids.
\brief Defines GridHandle, which manages a memory buffer containing one
or more NanoVDB grids: host-resident, dual host/device, or a
single-space device-only buffer.
*/
#ifndef NANOVDB_GRID_HANDLE_H_HAS_BEEN_INCLUDED
@@ -30,18 +31,63 @@ namespace nanovdb {
struct GridHandleMetaData {uint64_t offset, size; GridType gridType;};
namespace detail {
/// @brief Allocates @c bytes of host-readable storage for a GridHandle:
/// through @c BufferT::create for buffers that provide it, and through
/// @c pool's resource for a host-accessible single-space buffer (e.g. a
/// pinned-resource cuda::Buffer). The braced third argument selects the
/// uninitialized-storage constructor without naming its tag type, so
/// this header stays CUDA-free.
template<typename BufferT>
inline BufferT createHostStorage(uint64_t bytes, const BufferT& pool)
{
if constexpr (BufferHasHostSingle<BufferT>::value && BufferHasStream<BufferT>::value) {
return BufferT(pool.stream(), pool.resource(), bytes, {});// stream-ordered resource: allocate on the pool's retained stream
} else if constexpr (BufferHasHostSingle<BufferT>::value) {
return BufferT(pool.resource(), bytes, {});
} else {
return BufferT::create(bytes, &pool);
}
}
}// namespace detail
/// @brief This class serves to manage a buffer containing one or more NanoVDB Grids.
///
/// @note It is important to note that this class does NOT depend on OpenVDB.
template<typename BufferT = HostBuffer>
class GridHandle
{
static_assert(!(BufferTraits<BufferT>::hasDeviceDual && BufferHasDeviceSingle<BufferT>::value),
"a buffer cannot be both dual-space and single-space");
static_assert(!(BufferHasDeviceSingle<BufferT>::value || BufferHasHostSingle<BufferT>::value)
|| BufferHasByteElements<BufferT>::value,
"GridHandle requires byte-addressed single-space storage, e.g. cuda::Buffer<std::byte, R>");
std::vector<GridHandleMetaData> mMetaData;
BufferT mBuffer;
template <typename T>
static T* no_const(const T* ptr) { return const_cast<T*>(ptr); }
/// @brief Shared lookup behind grid() and deviceGrid(): the @a n'th grid
/// within @a base, or nullptr when @a base is null, @a n is out of
/// range, or the value type does not match the grid.
template<typename ValueT>
const NanoGrid<ValueT>* gridAt(const void* base, uint32_t n) const
{
if (base == nullptr || n >= mMetaData.size() || mMetaData[n].gridType != toGridType<ValueT>()) return nullptr;
return util::PtrAdd<NanoGrid<ValueT>>(base, mMetaData[n].offset);
}
/// @brief Adopts a buffer whose metadata is already known, so a deep copy
/// does not re-parse and a non-default-constructible buffer (e.g.
/// over a ResourceRef) never needs default construction.
GridHandle(BufferT&& buffer, std::vector<GridHandleMetaData> meta)
: mMetaData(std::move(meta))
, mBuffer(std::move(buffer)) {}
public:
using BufferType = BufferT;
@@ -55,7 +101,15 @@ public:
/// @brief Move constructor from a host buffer
/// @param buffer buffer containing one or more NanoGrids that will be moved into this GridHandle
/// @throw Will throw and error with the buffer does not contain a valid NanoGrid!
template<typename T = BufferT, typename util::disable_if<BufferTraits<T>::hasDeviceDual, int>::type = 0>
template<typename T = BufferT, typename util::disable_if<BufferTraits<T>::hasDeviceDual || BufferHasDeviceSingle<T>::value, int>::type = 0>
GridHandle(T&& buffer);
/// @brief Move constructor from a single-space device buffer: the grid
/// metadata is read through the device on the buffer's stream.
/// @param buffer buffer containing one or more NanoGrids that will be moved into this GridHandle
/// @throw Will throw an error if the buffer does not contain a valid NanoGrid
/// @note The implementation of this template specialization is in nanovdb/cuda/GridHandle.cuh since it requires CUDA
template<typename T = BufferT, typename util::enable_if<BufferHasDeviceSingle<T>::value, int>::type = 0, typename = void>
GridHandle(T&& buffer);
/// @brief Constructs an empty GridHandle
@@ -65,14 +119,14 @@ public:
GridHandle(const GridHandle&) = delete;
/// @brief Move copy-constructor
GridHandle(GridHandle&& other) noexcept {
mBuffer = std::move(other.mBuffer);
mMetaData = std::move(other.mMetaData);
}
GridHandle(GridHandle&& other) noexcept
: mMetaData(std::move(other.mMetaData))
, mBuffer(std::move(other.mBuffer)) {}
/// @brief clear this GridHandle to an empty handle
void reset() {
mBuffer.clear();
if constexpr (BufferHasDestroy<BufferT>::value) mBuffer.destroy();
else mBuffer.clear();
mMetaData.clear();
}
@@ -87,11 +141,25 @@ public:
}
/// @brief Performs a deep copy of the GridHandle, possibly templated on a different buffer type
/// @tparam OtherBufferT Buffer type of the deep copy
/// @param buffer optional buffer used for allocation
/// @tparam OtherBufferT Buffer type of the deep copy. A single-space device
/// handle only copies to its own buffer type (device-to-device,
/// ordered on the source's retained stream), so call it as
/// copy<BufferT>() -- the default OtherBufferT does not compile.
/// @param buffer optional buffer used for allocation; a compile error for
/// single-space handles, which allocate through the source buffer's
/// resource (use the no-argument overload)
/// @return A new handle of the specified buffer type that contains a deep copy of the current handle
template <typename OtherBufferT = HostBuffer>
GridHandle<OtherBufferT> copy(const OtherBufferT& buffer = OtherBufferT()) const;
GridHandle<OtherBufferT> copy(const OtherBufferT& buffer) const;
/// @brief Deep copy without a pool argument. The single-space path
/// allocates through the source buffer's resource and never
/// constructs a pool buffer, so it works for buffers that are not
/// default-constructible, e.g. over a ResourceRef. The host path
/// default-constructs the pool argument, so it requires a
/// default-constructible OtherBufferT.
template <typename OtherBufferT = HostBuffer>
GridHandle<OtherBufferT> copy() const;
/// @brief Return a reference to the buffer
BufferT& buffer() { return mBuffer; }
@@ -99,13 +167,15 @@ public:
/// @brief Return a const reference to the buffer
const BufferT& buffer() const { return mBuffer; }
/// @brief Returns a non-const pointer to the data.
//@{
/// @brief Returns a pointer to the host data; not available for a
/// single-space device buffer, which has no host-readable bytes.
/// @warning Note that the return pointer can be NULL if the GridHandle was not initialized
template<typename U = BufferT, typename util::disable_if<BufferHasDeviceSingle<U>::value, int>::type = 0>
void* data() { return mBuffer.data(); }
/// @brief Returns a const pointer to the data.
/// @warning Note that the return pointer can be NULL if the GridHandle was not initialized
template<typename U = BufferT, typename util::disable_if<BufferHasDeviceSingle<U>::value, int>::type = 0>
const void* data() const { return mBuffer.data(); }
//@}
template<typename U = BufferT>
typename util::enable_if<BufferTraits<U>::hasDeviceDual, const void*>::type
@@ -120,6 +190,16 @@ public:
typename util::enable_if<BufferTraits<U>::hasDeviceDual, void*>::type
deviceData(int device) { return mBuffer.deviceData(device); }
//@{
/// @brief For a single-space buffer the device data is the buffer itself.
template<typename U = BufferT>
typename util::enable_if<BufferHasDeviceSingle<U>::value, const void*>::type
deviceData() const { return mBuffer.data(); }
template<typename U = BufferT>
typename util::enable_if<BufferHasDeviceSingle<U>::value, void*>::type
deviceData() { return mBuffer.data(); }
//@}
//@{
/// @brief Returns the size in bytes of the raw memory buffer managed by this GridHandle.
[[deprecated("Use GridHandle::bufferSize instead.")]] uint64_t size() const { return mBuffer.size(); }
@@ -140,7 +220,7 @@ public:
/// @param n Index of the (host) grid pointer to be returned
/// @warning Note that the return pointer can be NULL if the GridHandle no host grid, @a n is invalid
/// or if the template parameter does not match the specified grid!
template<typename ValueT>
template<typename ValueT, typename U = BufferT, typename util::disable_if<BufferHasDeviceSingle<U>::value, int>::type = 0>
const NanoGrid<ValueT>* grid(uint32_t n = 0) const;
/// @brief Returns a host pointer to the @a n'th NanoVDB grid encoded in this GridHandle.
@@ -148,7 +228,7 @@ public:
/// @param n Index of the (host) grid pointer to be returned
/// @warning Note that the return pointer can be NULL if the GridHandle no host grid, @a n is invalid
/// or if the template parameter does not match the specified grid!
template<typename ValueT>
template<typename ValueT, typename U = BufferT, typename util::disable_if<BufferHasDeviceSingle<U>::value, int>::type = 0>
NanoGrid<ValueT>* grid(uint32_t n = 0) {return const_cast<NanoGrid<ValueT>*>(static_cast<const GridHandle*>(this)->template grid<ValueT>(n));}
/// @brief Return a const pointer to the @a n'th grid encoded in this GridHandle on the device, e.g. GPU
@@ -169,6 +249,18 @@ public:
typename util::enable_if<BufferTraits<U>::hasDeviceDual, NanoGrid<ValueT>*>::type
deviceGrid(uint32_t n=0){return const_cast<NanoGrid<ValueT>*>(static_cast<const GridHandle*>(this)->template deviceGrid<ValueT>(n));}
//@{
/// @brief Return a pointer to the @a n'th grid of a single-space device buffer
/// @warning Note that the return pointer can be NULL if the GridHandle is empty, @a n is invalid,
/// or if the template parameter does not match the specified grid.
template<typename ValueT, typename U = BufferT>
typename util::enable_if<BufferHasDeviceSingle<U>::value, const NanoGrid<ValueT>*>::type
deviceGrid(uint32_t n=0) const { return this->template gridAt<ValueT>(mBuffer.data(), n); }
template<typename ValueT, typename U = BufferT>
typename util::enable_if<BufferHasDeviceSingle<U>::value, NanoGrid<ValueT>*>::type
deviceGrid(uint32_t n=0){return const_cast<NanoGrid<ValueT>*>(static_cast<const GridHandle*>(this)->template deviceGrid<ValueT>(n));}
//@}
/// @brief Upload the grid to the device, e.g. from CPU to GPU
/// @note This method is only available if the buffer supports devices
template<typename U = BufferT>
@@ -230,17 +322,21 @@ public:
/// @brief Access to the GridData of the n'th grid in the current handle
/// @param n zero-based ID of the grid
/// @return Const pointer to the n'th GridData in the current handle
template<typename U = BufferT, typename util::disable_if<BufferHasDeviceSingle<U>::value, int>::type = 0>
const GridData* gridData(uint32_t n = 0) const;
/// @brief Returns a const point to the @a n'th grid meta data
/// @param n zero-based ID of the grid
/// @warning Note that the return pointer can be NULL if the GridHandle was not initialized
template<typename U = BufferT, typename util::disable_if<BufferHasDeviceSingle<U>::value, int>::type = 0>
const GridMetaData* gridMetaData(uint32_t n = 0) const;
/// @brief Write a specific grid in this buffer to an output stream
/// @param os output stream that the buffer will be written to
/// @param n zero-based index of the grid to be written to stream
void write(std::ostream& os, uint32_t n) const {
static_assert(!BufferHasDeviceSingle<BufferT>::value,
"GridHandle::write requires host-accessible grids: cuda::copyTo a host-readable handle first");
if (const GridData* data = this->gridData(n)) {
os.write((const char*)data, data->mGridSize);
} else {
@@ -251,6 +347,9 @@ public:
/// @brief Write the entire grid buffer to an output stream
/// @param os output stream that the buffer will be written to
void write(std::ostream& os) const {
static_assert(!BufferHasDeviceSingle<BufferT>::value,
"GridHandle::write requires host-accessible grids: cuda::copyTo a host-readable handle first");
for (uint32_t n=0; n<this->gridCount(); ++n) this->write(os, n);
}
@@ -328,6 +427,7 @@ public:
// --------------------------> Implementation of private methods in GridHandle <------------------------------------
template<typename BufferT>
template<typename U, typename util::disable_if<BufferHasDeviceSingle<U>::value, int>::type>
inline const GridData* GridHandle<BufferT>::gridData(uint32_t n) const
{
const void *data = this->data();
@@ -336,6 +436,7 @@ inline const GridData* GridHandle<BufferT>::gridData(uint32_t n) const
}// const GridData* GridHandle<BufferT>::gridData(uint32_t n) const
template<typename BufferT>
template<typename U, typename util::disable_if<BufferHasDeviceSingle<U>::value, int>::type>
inline const GridMetaData* GridHandle<BufferT>::gridMetaData(uint32_t n) const
{
const auto *data = this->data();
@@ -355,11 +456,11 @@ inline __hostdev__ void cpyGridHandleMeta(const GridData *data, GridHandleMetaDa
// template specialization of move constructor from a host buffer
template<typename BufferT>
template<typename T, typename util::disable_if<BufferTraits<T>::hasDeviceDual, int>::type>
template<typename T, typename util::disable_if<BufferTraits<T>::hasDeviceDual || BufferHasDeviceSingle<T>::value, int>::type>
GridHandle<BufferT>::GridHandle(T&& buffer)
: mBuffer(std::move(buffer))
{
static_assert(util::is_same<T,BufferT>::value, "Expected U==BufferT");
mBuffer = std::move(buffer);
if (auto *data = reinterpret_cast<const GridData*>(mBuffer.data())) {
if (!data->isValid()) throw std::runtime_error("GridHandle was constructed with an invalid host buffer");
mMetaData.resize(data->mGridCount);
@@ -371,19 +472,38 @@ template<typename BufferT>
template <typename OtherBufferT>
inline GridHandle<OtherBufferT> GridHandle<BufferT>::copy(const OtherBufferT& other) const
{
if (mBuffer.isEmpty()) return GridHandle<OtherBufferT>();// return an empty handle
auto buffer = OtherBufferT::create(mBuffer.size(), &other);
static_assert(!(BufferHasDeviceSingle<BufferT>::value || BufferHasDeviceSingle<OtherBufferT>::value),
"GridHandle::copy(pool) cannot honor a pool argument for a single-space device buffer, "
"whose copy allocates through the source buffer's resource: use the no-argument copy()");
if (mBuffer.size() == 0) return GridHandle<OtherBufferT>();// return an empty handle
auto buffer = detail::createHostStorage<OtherBufferT>(mBuffer.size(), other);
std::memcpy(buffer.data(), mBuffer.data(), mBuffer.size());// deep copy of buffer
return GridHandle<OtherBufferT>(std::move(buffer));
}// GridHandle<OtherBufferT> GridHandle<BufferT>::copy(const OtherBufferT& other) const
template<typename BufferT>
template<typename ValueT>
template <typename OtherBufferT>
inline GridHandle<OtherBufferT> GridHandle<BufferT>::copy() const
{
if constexpr (BufferHasDeviceSingle<BufferT>::value || BufferHasDeviceSingle<OtherBufferT>::value) {
static_assert(util::is_same<OtherBufferT, BufferT>::value && BufferHasDeviceSingle<BufferT>::value,
"GridHandle::copy is same-space only: a single-space device handle copies to its own "
"buffer type; use cuda::copyTo (cuda/GridHandle.cuh) to move grids across address spaces");
// Device-to-device deep copy; for a stream-ordered resource it is
// ordered on the source's retained stream, so synchronize that stream
// before reading the result. Metadata is host-resident, so the copy
// adopts it directly with no device re-parse.
return GridHandle(mBuffer.copy(), mMetaData);
} else {
return this->copy(OtherBufferT());
}
}// GridHandle<OtherBufferT> GridHandle<BufferT>::copy() const
template<typename BufferT>
template<typename ValueT, typename U, typename util::disable_if<BufferHasDeviceSingle<U>::value, int>::type>
inline const NanoGrid<ValueT>* GridHandle<BufferT>::grid(uint32_t n) const
{
const void *data = mBuffer.data();
if (data == nullptr || n >= mMetaData.size() || mMetaData[n].gridType != toGridType<ValueT>()) return nullptr;
return util::PtrAdd<NanoGrid<ValueT>>(data, mMetaData[n].offset);
return this->template gridAt<ValueT>(mBuffer.data(), n);
}// const NanoGrid<ValueT>* GridHandle<BufferT>::grid(uint32_t n) const
template<typename BufferT>
@@ -391,14 +511,14 @@ template<typename ValueT, typename U>
inline typename util::enable_if<BufferTraits<U>::hasDeviceDual, const NanoGrid<ValueT>*>::type
GridHandle<BufferT>::deviceGrid(uint32_t n) const
{
const void *data = mBuffer.deviceData();
if (data == nullptr || n >= mMetaData.size() || mMetaData[n].gridType != toGridType<ValueT>()) return nullptr;
return util::PtrAdd<NanoGrid<ValueT>>(data, mMetaData[n].offset);
return this->template gridAt<ValueT>(mBuffer.deviceData(), n);
}// GridHandle<BufferT>::deviceGrid(uint32_t n) cons
template<typename BufferT>
void GridHandle<BufferT>::read(std::istream& is, const BufferT& pool)
{
static_assert(!BufferHasDeviceSingle<BufferT>::value,
"GridHandle::read requires a host-accessible buffer: read into a host-readable handle, then cuda::copyTo");
const std::streampos start = is.tellg();// remember where the raw buffer begins
GridData data;
is.read((char*)&data, sizeof(GridData));
@@ -409,7 +529,7 @@ void GridHandle<BufferT>::read(std::istream& is, const BufferT& pool)
is.read((char*)&data, sizeof(GridData));
sum += data.mGridSize;
}
auto buffer = BufferT::create(size + sum, &pool);
auto buffer = detail::createHostStorage(size + sum, pool);
is.seekg(start);// rewind to the start of the raw buffer
is.read((char*)(buffer.data()), buffer.size());
*this = GridHandle(std::move(buffer));
@@ -422,6 +542,8 @@ void GridHandle<BufferT>::read(std::istream& is, const BufferT& pool)
template<typename BufferT>
void GridHandle<BufferT>::read(std::istream& is, uint32_t n, const BufferT& pool)
{
static_assert(!BufferHasDeviceSingle<BufferT>::value,
"GridHandle::read requires a host-accessible buffer: read into a host-readable handle, then cuda::copyTo");
GridData data;
is.read((char*)&data, sizeof(GridData));
if (data.isValid()) {
@@ -430,7 +552,7 @@ void GridHandle<BufferT>::read(std::istream& is, uint32_t n, const BufferT& pool
is.seekg(data.mGridSize - sizeof(GridData), std::ios::cur);// skip grid
is.read((char*)&data, sizeof(GridData));
}
auto buffer = BufferT::create(data.mGridSize, &pool);
auto buffer = detail::createHostStorage(data.mGridSize, pool);
is.seekg(-sizeof(GridData), std::ios::cur);// rewind
is.read((char*)(buffer.data()), data.mGridSize);
tools::updateGridCount((GridData*)buffer.data(), 0u, 1u);
@@ -444,6 +566,8 @@ void GridHandle<BufferT>::read(std::istream& is, uint32_t n, const BufferT& pool
template<typename BufferT>
void GridHandle<BufferT>::read(std::istream& is, const std::string &gridName, const BufferT& pool)
{
static_assert(!BufferHasDeviceSingle<BufferT>::value,
"GridHandle::read requires a host-accessible buffer: read into a host-readable handle, then cuda::copyTo");
static const std::streamsize byteSize = sizeof(GridData);
GridData data;
is.read((char*)&data, byteSize);
@@ -456,7 +580,7 @@ void GridHandle<BufferT>::read(std::istream& is, const std::string &gridName, co
is.seekg(-byteSize, std::ios::cur);// rewind
}
if (n>data.mGridCount) throw std::runtime_error("No raw grid named \""+gridName+"\"");
auto buffer = BufferT::create(data.mGridSize, &pool);
auto buffer = detail::createHostStorage(data.mGridSize, pool);
is.read((char*)(buffer.data()), data.mGridSize);
tools::updateGridCount((GridData*)buffer.data(), 0u, 1u);
*this = GridHandle(std::move(buffer));
@@ -476,6 +600,10 @@ template<typename BufferT, template <class, class...> class VectorT = std::vecto
inline VectorT<GridHandle<BufferT>>
splitGrids(const GridHandle<BufferT> &handle, const BufferT* other = nullptr)
{
static_assert(!BufferHasDeviceSingle<BufferT>::value,
"splitGrids requires a buffer type providing create(): cuda::copyTo a HostBuffer handle first");
static_assert(!BufferHasHostSingle<BufferT>::value,
"splitGrids requires a buffer type providing create(): copy the handle to a HostBuffer first");
using HandleT = GridHandle<BufferT>;
const void *ptr = handle.data();
if (ptr == nullptr) return VectorT<HandleT>();
@@ -493,6 +621,51 @@ splitGrids(const GridHandle<BufferT> &handle, const BufferT* other = nullptr)
return handles;
}// splitGrids
/// @brief Combines (or merges) multiple GridHandles into a single GridHandle containing all grids
/// @tparam BufferT Type of the input and output grid buffers
/// @param handles array of non-owning pointers to the GridHandles to be combined
/// @param count number of pointers in @a handles
/// @param pool optional pool used for allocation of output GridHandle
/// @return single GridHandle containing all input grids
/// @note This overload borrows the input handles, so it can be used when the
/// handles are owned elsewhere (e.g. by a language binding) and can be
/// neither moved from nor cheaply copied.
template<typename BufferT>
inline GridHandle<BufferT>
mergeGrids(const GridHandle<BufferT>* const* handles, size_t count, const BufferT* pool = nullptr)
{
static_assert(!BufferHasDeviceSingle<BufferT>::value,
"mergeGrids requires a buffer type providing create(): cuda::copyTo HostBuffer handles first");
static_assert(!BufferHasHostSingle<BufferT>::value,
"mergeGrids requires a buffer type providing create(): copy the handles to HostBuffer first");
uint64_t size = 0u;
uint32_t counter = 0u, gridCount = 0u;
for (size_t i = 0; i < count; ++i) {
gridCount += handles[i]->gridCount();
for (uint32_t n=0; n<handles[i]->gridCount(); ++n) size += handles[i]->gridSize(n);
}
// Return an empty handle when there is nothing to merge: BufferT::create(0)
// can produce a non-null zero-byte region, from which the GridHandle
// constructor would attempt to read a GridData header.
if (gridCount == 0u) return GridHandle<BufferT>();
auto buffer = BufferT::create(size, pool);
void *dst = buffer.data();
for (size_t i = 0; i < count; ++i) {
const GridHandle<BufferT> &h = *handles[i];
for (uint32_t n=0; n<h.gridCount(); ++n) {
// gridData(n) applies the per-grid offset, so padded (non-contiguous)
// source buffers are copied correctly.
const GridData *src = h.gridData(n);
NANOVDB_ASSERT(src && src->isValid());
const uint64_t gridSize = h.gridSize(n);
std::memcpy(dst, src, gridSize);
tools::updateGridCount(reinterpret_cast<GridData*>(dst), counter++, gridCount);
dst = util::PtrAdd(dst, gridSize);
}
}
return GridHandle<BufferT>(std::move(buffer));
}// mergeGrids
/// @brief Combines (or merges) multiple GridHandles into a single GridHandle containing all grids
/// @tparam BufferT Type of the input and output grid buffers
/// @param handles Vector of GridHandles to be combined
@@ -502,26 +675,10 @@ template<typename BufferT, template <class, class...> class VectorT>
inline GridHandle<BufferT>
mergeGrids(const VectorT<GridHandle<BufferT>> &handles, const BufferT* pool = nullptr)
{
uint64_t size = 0u;
uint32_t counter = 0u, gridCount = 0u;
for (auto &h : handles) {
gridCount += h.gridCount();
for (uint32_t n=0; n<h.gridCount(); ++n) size += h.gridSize(n);
}
auto buffer = BufferT::create(size, pool);
void *dst = buffer.data();
for (auto &h : handles) {
const void *src = h.data();
for (uint32_t n=0; n<h.gridCount(); ++n) {
std::memcpy(dst, src, h.gridSize(n));
GridData *data = reinterpret_cast<GridData*>(dst);
NANOVDB_ASSERT(data->isValid());
tools::updateGridCount(data, counter++, gridCount);
dst = util::PtrAdd(dst, data->mGridSize);
src = util::PtrAdd(src, data->mGridSize);
}
}
return GridHandle<BufferT>(std::move(buffer));
std::vector<const GridHandle<BufferT>*> ptrs;
ptrs.reserve(handles.size());
for (auto &h : handles) ptrs.push_back(&h);
return mergeGrids<BufferT>(ptrs.data(), ptrs.size(), pool);
}// mergeGrids
} // namespace nanovdb
+50 -1
View File
@@ -87,6 +87,7 @@
#include <cassert>// for assert
#include <sstream>// for std::stringstream
#include <cstring>// for memcpy
#include <type_traits>// for std::void_t
#define checkPtr(ptr, msg) \
{ \
@@ -98,9 +99,57 @@ namespace nanovdb {
template<typename BufferT>
struct BufferTraits
{
static constexpr bool hasDeviceDual = false;
static constexpr bool hasDeviceDual = false;
static constexpr bool hasDeviceSingle = false;
};
/// @brief Detects whether @c BufferTraits<BufferT> defines @c hasDeviceSingle,
/// i.e. whether the buffer manages a single device-resident allocation.
/// @details Defaults to false when the trait member is absent, so pre-existing
/// BufferTraits specializations (in or out of tree) that only define
/// hasDeviceDual keep compiling unchanged.
template<typename BufferT, typename = void>
struct BufferHasDeviceSingle { static constexpr bool value = false; };
template<typename BufferT>
struct BufferHasDeviceSingle<BufferT, std::void_t<decltype(BufferTraits<BufferT>::hasDeviceSingle)>>
{ static constexpr bool value = BufferTraits<BufferT>::hasDeviceSingle; };
/// @brief Companion detection for BufferTraits<...>::hasHostSingle: a
/// single-space buffer whose storage is host-accessible (e.g. a
/// pinned-resource cuda::Buffer).
template<typename BufferT, typename = void>
struct BufferHasHostSingle { static constexpr bool value = false; };
template<typename BufferT>
struct BufferHasHostSingle<BufferT, std::void_t<decltype(BufferTraits<BufferT>::hasHostSingle)>>
{ static constexpr bool value = BufferTraits<BufferT>::hasHostSingle; };
/// @brief Detects whether a buffer exposes a retained stream (a stream()
/// member), i.e. whether its resource is stream-ordered. Used to pick
/// the buffer's stream-taking constructor without naming CUDA types.
template<typename BufferT, typename = void>
struct BufferHasStream { static constexpr bool value = false; };
template<typename BufferT>
struct BufferHasStream<BufferT, std::void_t<decltype(std::declval<const BufferT&>().stream())>>
{ static constexpr bool value = true; };
/// @brief Detects whether a buffer's elements are byte-sized. Buffers that
/// expose no ElementType (e.g. HostBuffer) address raw bytes by
/// definition, so the primary defaults to true.
template<typename BufferT, typename = void>
struct BufferHasByteElements { static constexpr bool value = true; };
template<typename BufferT>
struct BufferHasByteElements<BufferT, std::void_t<typename BufferT::ElementType>>
{ static constexpr bool value = sizeof(typename BufferT::ElementType) == 1; };
/// @brief Detects whether a buffer provides destroy(), the cuda::Buffer
/// spelling for releasing its storage. Handle reset() dispatches to it
/// when present and falls back to the legacy clear() otherwise.
template<typename BufferT, typename = void>
struct BufferHasDestroy { static constexpr bool value = false; };
template<typename BufferT>
struct BufferHasDestroy<BufferT, std::void_t<decltype(std::declval<BufferT&>().destroy())>>
{ static constexpr bool value = true; };
// ----------------------------> HostBuffer <--------------------------------------
/// @brief This is a buffer that contains a shared or private pool
+163
View File
@@ -1808,6 +1808,42 @@ struct ProbeValue;
template<typename BuildT>
struct GetNodeInfo;
/// @brief Result of a fused ReadAccessor::getDimAndActive traversal: the
/// getDim(ijk, ray) step size and the policy-dependent active state of @a ijk.
///
/// The low 31 bits store @c dim and bit 31 stores @c active, saving one live
/// return register versus {uint32_t, bool}. The default FloatTree's upper-node
/// maximum is 4096. The constructor supports `return {dim, active}`; dim() and
/// active() decode the packed value.
struct DimAndActive
{
uint32_t packed{0u};
__hostdev__ DimAndActive() : packed(0u) {}
__hostdev__ DimAndActive(uint32_t d, bool a)
: packed((d & 0x7FFFFFFFu) | (uint32_t(a) << 31))
{
NANOVDB_ASSERT(d < (uint32_t(1) << 31)); // dim must fit in 31 bits (bit 31 = active)
}
__hostdev__ uint32_t dim() const { return packed & 0x7FFFFFFFu; }
__hostdev__ bool active() const { return (packed >> 31) != 0u; }
};
/// @brief Policy that exactly matches separate getDim and isActive calls. A
/// skip flag still controls @c dim but cannot stop the active-state descent:
/// GridStats flags value range versus render delta, not activeness. This costs
/// a full descent through skipped InternalNodes; use it when @c active is read
/// unconditionally, such as by HDDA iterators observing upper-level activeness.
struct ActiveExact {};
/// @brief Policy that preserves getDim's skip-flag fast path. If an
/// InternalNode short-circuits, @c active is unspecified and must be gated on
/// @c dim. Otherwise the leaf-level active read is fused into the descent. Use
/// for logic such as `if (hdda.dim() > 1 || !active) continue;`, where the
/// short-circuit ensures @c active is read only after a leaf-level descent.
struct ActiveOnLeafOnly {};
// ----------------------------> CheckMode <----------------------------------
/// @brief List of different modes for computing for a checksum
@@ -3136,6 +3172,23 @@ private:
return ChildNodeType::dim(); // background
}
/// @brief Returns dim and active state in one traversal, sharing cache and
/// terminal tile/mask reads. Root has no skip flag, so @c Policy affects
/// only the child descent; see ActiveExact and ActiveOnLeafOnly.
template<typename Policy = ActiveExact, typename RayT, typename AccT>
__hostdev__ DimAndActive getDimAndActiveAndCache(const CoordType& ijk, const RayT& ray, const AccT& acc) const
{
if (const Tile* tile = this->probeTile(ijk)) {
if (tile->isChild()) {
const auto* child = this->getChild(tile);
acc.insert(ijk, child);
return child->template getDimAndActiveAndCache<Policy>(ijk, ray, acc);
}
return {uint32_t(1) << ChildT::TOTAL, tile->state > 0u}; // tile value
}
return {ChildNodeType::dim(), false}; // background is never active
}
template<typename OpT, typename AccT, typename... ArgsT>
__hostdev__ typename OpT::Type getAndCache(const CoordType& ijk, const AccT& acc, ArgsT&&... args) const
{
@@ -3632,6 +3685,39 @@ private:
return ChildNodeType::dim(); // tile value
}
/// @brief Returns dim and active state in one traversal. On a skip flag,
/// ActiveExact resolves the true active state but returns this node's dim;
/// ActiveOnLeafOnly returns {dim, false} immediately. Without a skip flag
/// both policies are identical. See GridStats::setFlag.
template<typename Policy = ActiveExact, typename RayT, typename AccT>
__hostdev__ DimAndActive getDimAndActiveAndCache(const CoordType& ijk, const RayT& ray, const AccT& acc) const
{
//if (!ray.intersects( this->bbox() )) return {1<<TOTAL, ...};
// Separate paths keep `skip` out of ActiveOnLeafOnly's recursion,
// saving ~2 registers/thread and avoiding an occupancy cliff.
if constexpr (util::is_same<Policy, ActiveOnLeafOnly>::value) {
if (DataType::mFlags & uint32_t(1u))
return {this->dim(), false}; // active unspecified per policy
const uint32_t n = CoordToOffset(ijk);
if (DataType::mChildMask.isOn(n)) {
const ChildT* child = this->getChild(n);
acc.insert(ijk, child);
return child->template getDimAndActiveAndCache<Policy>(ijk, ray, acc);
}
return {ChildNodeType::dim(), DataType::mValueMask.isOn(n)};
} else {
const bool skip = (DataType::mFlags & uint32_t(1u));
const uint32_t n = CoordToOffset(ijk);
if (DataType::mChildMask.isOn(n)) {
const ChildT* child = this->getChild(n);
acc.insert(ijk, child);
const auto r = child->template getDimAndActiveAndCache<Policy>(ijk, ray, acc);
return {skip ? this->dim() : r.dim(), r.active()};
}
return {skip ? this->dim() : ChildNodeType::dim(), DataType::mValueMask.isOn(n)};
}
}
template<typename OpT, typename AccT, typename... ArgsT>
__hostdev__ typename OpT::Type getAndCache(const CoordType& ijk, const AccT& acc, ArgsT&&... args) const
{
@@ -4572,6 +4658,21 @@ private:
return ChildNodeType::dim();
}
/// @brief Returns dim and active state in one traversal. @c Policy is
/// accepted for recursion but does not change leaf behavior: the skip flag
/// controls dim while active always comes from mValueMask. Branching here
/// raised parallelForKernel from 64 to 66 registers/thread and reduced
/// occupancy from four to three blocks/SM, negating the descent savings;
/// leaf skip flags are rare because upper-level flags usually short-circuit.
template<typename Policy = ActiveExact, typename RayT, typename AccT>
__hostdev__ DimAndActive getDimAndActiveAndCache(const CoordT& ijk, const RayT& /*ray*/, const AccT&) const
{
//if (!ray.intersects( this->bbox() )) return {1 << LOG2DIM, ...};
const uint32_t n = CoordToOffset(ijk);
const uint32_t dimOut = (DataType::mFlags & uint8_t(1u)) ? this->dim() : ChildNodeType::dim();
return {dimOut, DataType::mValueMask.isOn(n)};
}
template<typename OpT, typename AccT, typename... ArgsT>
__hostdev__ auto
//__hostdev__ decltype(OpT::get(util::declval<const LeafNode&>(), util::declval<uint32_t>(), util::declval<ArgsT>()...))
@@ -4878,6 +4979,15 @@ public:
{
return mRoot->getDimAndCache(ijk, ray, *this);
}
/// @brief Fused getDim + isActive: one tree walk returns both the HDDA
/// step size and the active state of @a ijk, shaving the second descent
/// / cache-check / leaf ValueMask lookup. See @c ActiveExact /
/// @c ActiveOnLeafOnly for the @c Policy contract.
template<typename Policy = ActiveExact, typename RayT>
__hostdev__ DimAndActive getDimAndActive(const CoordType& ijk, const RayT& ray) const
{
return mRoot->template getDimAndActiveAndCache<Policy>(ijk, ray, *this);
}
template<typename OpT, typename... ArgsT>
__hostdev__ auto get(const CoordType& ijk, ArgsT&&... args) const
{
@@ -4993,6 +5103,17 @@ public:
return mRoot->getDimAndCache(ijk, ray, *this);
}
/// @brief Fused getDim + isActive: one tree walk returns both the HDDA
/// step size and the active state of @a ijk, shaving the second descent
/// / cache-check / leaf ValueMask lookup. See @c ActiveExact /
/// @c ActiveOnLeafOnly for the @c Policy contract.
template<typename Policy = ActiveExact, typename RayT>
__hostdev__ DimAndActive getDimAndActive(const CoordType& ijk, const RayT& ray) const
{
if (this->isCached(ijk)) return mNode->template getDimAndActiveAndCache<Policy>(ijk, ray, *this);
return mRoot->template getDimAndActiveAndCache<Policy>(ijk, ray, *this);
}
template<typename OpT, typename... ArgsT>
__hostdev__ typename OpT::Type get(const CoordType& ijk, ArgsT&&... args) const
{
@@ -5175,6 +5296,26 @@ public:
return mRoot->getDimAndCache(ijk, ray, *this);
}
/// @brief Fused getDim + isActive: one tree walk returns both the HDDA
/// step size and the active state of @a ijk, shaving the second descent
/// / cache-check / leaf ValueMask lookup. See @c ActiveExact /
/// @c ActiveOnLeafOnly for the @c Policy contract.
template<typename Policy = ActiveExact, typename RayT>
__hostdev__ DimAndActive getDimAndActive(const CoordType& ijk, const RayT& ray) const
{
#ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
const CoordValueType dirty = this->computeDirty(ijk);
#else
auto&& dirty = ijk;
#endif
if (this->isCached1(dirty)) {
return mNode1->template getDimAndActiveAndCache<Policy>(ijk, ray, *this);
} else if (this->isCached2(dirty)) {
return mNode2->template getDimAndActiveAndCache<Policy>(ijk, ray, *this);
}
return mRoot->template getDimAndActiveAndCache<Policy>(ijk, ray, *this);
}
template<typename OpT, typename... ArgsT>
__hostdev__ typename OpT::Type get(const CoordType& ijk, ArgsT&&... args) const
{
@@ -5446,6 +5587,28 @@ public:
return mRoot->getDimAndCache(ijk, ray, *this);
}
/// @brief Fused getDim + isActive: one tree walk returns both the HDDA
/// step size and the active state of @a ijk, shaving the second descent
/// / cache-check / leaf ValueMask lookup. See @c ActiveExact /
/// @c ActiveOnLeafOnly for the @c Policy contract.
template<typename Policy = ActiveExact, typename RayT>
__hostdev__ DimAndActive getDimAndActive(const CoordType& ijk, const RayT& ray) const
{
#ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
const CoordValueType dirty = this->computeDirty(ijk);
#else
auto&& dirty = ijk;
#endif
if (this->isCached<LeafT>(dirty)) {
return ((LeafT*)mNode[0])->template getDimAndActiveAndCache<Policy>(ijk, ray, *this);
} else if (this->isCached<NodeT1>(dirty)) {
return ((NodeT1*)mNode[1])->template getDimAndActiveAndCache<Policy>(ijk, ray, *this);
} else if (this->isCached<NodeT2>(dirty)) {
return ((NodeT2*)mNode[2])->template getDimAndActiveAndCache<Policy>(ijk, ray, *this);
}
return mRoot->template getDimAndActiveAndCache<Policy>(ijk, ray, *this);
}
private:
/// @brief Allow nodes to insert themselves into the cache.
template<typename>
+36 -13
View File
@@ -53,10 +53,14 @@ struct NodeManagerData
template<typename BufferT>
class NodeManagerHandle
{
static_assert(!(BufferHasDeviceSingle<BufferT>::value || BufferHasHostSingle<BufferT>::value)
|| BufferHasByteElements<BufferT>::value,
"NodeManagerHandle requires byte-addressed single-space storage, e.g. cuda::Buffer<std::byte, R>");
GridType mGridType{GridType::Unknown};
BufferT mBuffer;
template<typename BuildT>
template<typename BuildT, typename U = BufferT, typename util::disable_if<BufferHasDeviceSingle<U>::value, int>::type = 0>
const NodeManager<BuildT>* getMgr() const {
return mGridType == toGridType<BuildT>() ? (const NodeManager<BuildT>*)mBuffer.data() : nullptr;
}
@@ -72,7 +76,9 @@ class NodeManagerHandle
public:
/// @brief Move constructor from a buffer
NodeManagerHandle(GridType gridType, BufferT&& buffer) : mGridType(gridType) { mBuffer = std::move(buffer); }
NodeManagerHandle(GridType gridType, BufferT&& buffer)
: mGridType(gridType)
, mBuffer(std::move(buffer)) {}// member-initialized, so a non-default-constructible buffer works
/// @brief Empty ctor
NodeManagerHandle() = default;
/// @brief Disallow copy-construction
@@ -87,15 +93,19 @@ public:
return *this;
}
/// @brief Move copy-constructor
NodeManagerHandle(NodeManagerHandle&& other) noexcept {
mGridType = other.mGridType;
mBuffer = std::move(other.mBuffer);
NodeManagerHandle(NodeManagerHandle&& other) noexcept
: mGridType(other.mGridType)
, mBuffer(std::move(other.mBuffer))
{
other.mGridType = GridType::Unknown;
}
/// @brief Default destructor
~NodeManagerHandle() { this->reset(); }
/// @brief clear the buffer
void reset() { mBuffer.clear(); }
/// @brief Release the buffer's storage
void reset() {
if constexpr (BufferHasDestroy<BufferT>::value) mBuffer.destroy();
else mBuffer.clear();
}
/// @brief Return a reference to the buffer
BufferT& buffer() { return mBuffer; }
@@ -103,15 +113,15 @@ public:
/// @brief Return a const reference to the buffer
const BufferT& buffer() const { return mBuffer; }
/// @brief Returns a non-const pointer to the data.
///
//@{
/// @brief Returns a pointer to the host data; not available for a
/// single-space device buffer, which has no host-readable bytes.
/// @warning Note that the return pointer can be NULL if the NodeManagerHandle was not initialized
template<typename U = BufferT, typename util::disable_if<BufferHasDeviceSingle<U>::value, int>::type = 0>
void* data() { return mBuffer.data(); }
/// @brief Returns a const pointer to the data.
///
/// @warning Note that the return pointer can be NULL if the NodeManagerHandle was not initialized
template<typename U = BufferT, typename util::disable_if<BufferHasDeviceSingle<U>::value, int>::type = 0>
const void* data() const { return mBuffer.data(); }
//@}
/// @brief Returns the size in bytes of the raw memory buffer managed by this NodeManagerHandle's allocator.
uint64_t size() const { return mBuffer.size(); }
@@ -142,6 +152,19 @@ public:
typename util::enable_if<BufferTraits<U>::hasDeviceDual, NodeManager<BuildT>*>::type
deviceMgr() { return no_const(this->template getDeviceMgr<BuildT>()); }
//@{
/// @brief Return a pointer to the NodeManager of a single-space device buffer
/// @warning Note that the return pointer can be NULL if the template parameter does not match the specified grid!
template<typename BuildT, typename U = BufferT>
typename util::enable_if<BufferHasDeviceSingle<U>::value, const NodeManager<BuildT>*>::type
deviceMgr() const {
return mGridType == toGridType<BuildT>() ? (const NodeManager<BuildT>*)mBuffer.data() : nullptr;
}
template<typename BuildT, typename U = BufferT>
typename util::enable_if<BufferHasDeviceSingle<U>::value, NodeManager<BuildT>*>::type
deviceMgr() { return no_const(static_cast<const NodeManagerHandle*>(this)->template deviceMgr<BuildT>()); }
//@}
/// @brief Upload the NodeManager to the device, e.g. from CPU to GPU
///
/// @note This method is only available if the buffer supports devices
+2
View File
@@ -7,6 +7,8 @@ A lightweight GPU friendly version of VDB initially targeting rendering applicat
* [Frequently asked questions](../../doc/nanovdb/FAQ.md)
* [Source tree](../../doc/nanovdb/SourceTree.md)
* [Examples](../../doc/nanovdb/HelloWorld.md)
* [TEACHME — learn NanoVDB by prompting an LLM coding agent](../../doc/nanovdb/TEACHME/README.md)
* [Python examples](python/examples/)
### Copyright Contributors to the OpenVDB Project
### SPDX-License-Identifier: Apache-2.0
+108 -11
View File
@@ -50,7 +50,7 @@ struct StreamHolder<true> { cudaStream_t mStream = 0; };
/// is_resource). When @c R provides both interfaces the stream-ordered
/// one is used.
/// @details With a stream-ordered resource the Buffer retains the stream of
/// the most recent allocation (or the one supplied via setStream)
/// the most recent allocation (or the one supplied via set_stream)
/// and orders its deallocation on that stream. Buffer is move-only.
template<typename T, typename R = DeviceResource>
class Buffer : private detail::StreamHolder<is_async_resource<R>::value>
@@ -62,6 +62,18 @@ class Buffer : private detail::StreamHolder<is_async_resource<R>::value>
static constexpr bool IsAsync = is_async_resource<R>::value;
public:
/// @brief Element and resource types, for generic code that rebinds one
/// or constructs sibling buffers over the same resource.
using ElementType = T;
using ResourceType = R;
/// @brief Alias for a sibling buffer over the same resource with a
/// different element type.
template<typename U>
using rebind = Buffer<U, R>;
private:
R mResource;
T* mData = nullptr;
size_t mSize = 0; // element count
@@ -133,7 +145,7 @@ public:
Buffer& operator=(Buffer&& other) noexcept
{
if (this != &other) {
this->clear();
this->destroy();
static_cast<detail::StreamHolder<IsAsync>&>(*this) = other;
mResource = std::move(other.mResource);
mData = other.mData;
@@ -156,6 +168,11 @@ public:
return out;
}
/// @brief Returns a deep copy of this buffer ordered on the retained
/// stream, i.e. copy(this->stream()).
template<typename S = R, std::enable_if_t<is_async_resource<S>::value, int> = 0>
Buffer copy() const { return this->copy(this->stream()); }
/// @brief Returns a deep copy of this buffer, allocated from a copy of the
/// synchronous resource.
template<typename S = R, std::enable_if_t<!is_async_resource<S>::value && is_resource<S>::value, int> = 0>
@@ -168,7 +185,7 @@ public:
/// @brief D-tor. A stream-ordered resource frees on the retained stream;
/// a synchronous resource frees immediately.
~Buffer() { this->clear(); }
~Buffer() { this->destroy(); }
/// @brief Returns the retained stream, i.e. the stream the buffer's memory
/// will be freed on.
@@ -179,9 +196,11 @@ public:
/// deallocation (and destruction) is ordered on @c stream instead.
/// @param stream cuda stream subsequent deallocation is ordered on
/// @warning The caller is responsible for ordering @c stream after any
/// in-flight work that uses the buffer's memory.
/// in-flight work that uses the buffer's memory. This deliberately
/// does not synchronize, matching cuda::buffer's set_stream, which
/// avoids implicit synchronization in fundamental primitives.
template<typename S = R, std::enable_if_t<is_async_resource<S>::value, int> = 0>
void setStream(cudaStream_t stream) { this->mStream = stream; }
void set_stream(cudaStream_t stream) { this->mStream = stream; }
/// @brief Resizes the buffer to @c count elements, preserving the leading
/// min(old, new) elements. Every operation — the new allocation, the
@@ -219,7 +238,7 @@ public:
mSize = count;
}
else {
this->mStream = stream; // no reallocation: setStream semantics
this->mStream = stream; // no reallocation: set_stream semantics
}
}
@@ -253,6 +272,13 @@ public:
T* data() { return mData; }
const T* data() const { return mData; }
/// @brief Returns a copy of the resource; for a ResourceRef this refers
/// to the same underlying instance.
/// @note Requires R to be copy-constructible (the cuda::mr convention:
/// resources are cheap handles). A resource that owns its pool by
/// value hands the caller an independent copy of that pool.
R resource() const { return mResource; }
/// @brief Returns the number of elements.
size_t size() const { return mSize; }
@@ -263,13 +289,45 @@ public:
bool empty() const { return mSize == 0; }
/// @brief Frees the buffer memory (if any) and resets to the empty state.
void clear()
/// A stream-ordered resource frees on the retained stream.
/// @note Spelled destroy to match cuda::buffer. This is the name to use.
void destroy()
{
this->deallocate(mData, mSize);
mData = nullptr;
mSize = 0;
}
/// @brief Frees the buffer memory (if any) and resets to the empty state.
/// @deprecated Use destroy(): the handles now dispatch to it directly.
[[deprecated("Use cuda::Buffer::destroy instead")]]
void clear() { this->destroy(); }
/// @brief Frees the buffer memory (if any) on @c stream and resets to the
/// empty state. @c stream becomes the retained stream.
/// @param stream cuda stream the deallocation is ordered on
/// @warning The caller is responsible for ordering @c stream after any
/// in-flight work that uses the buffer's memory.
template<typename S = R, std::enable_if_t<is_async_resource<S>::value, int> = 0>
void destroy(cudaStream_t stream)
{
this->mStream = stream;
this->destroy();
}
/// @brief Exchanges the contents of this buffer with @c other. Neither
/// buffer allocates, frees, or copies element data.
/// @param other buffer to exchange contents with
void swap(Buffer& other) noexcept
{
auto& lhs = static_cast<detail::StreamHolder<IsAsync>&>(*this);
auto& rhs = static_cast<detail::StreamHolder<IsAsync>&>(other);
std::swap(lhs, rhs);
std::swap(mResource, other.mResource);
std::swap(mData, other.mData);
std::swap(mSize, other.mSize);
}
private:
/// @brief Returns @c count * sizeof(T), throwing std::runtime_error if the
/// byte size would overflow size_t instead of silently wrapping into
@@ -351,18 +409,57 @@ public:
bool empty() const { return mSize == 0; }
/// @brief Detaches the view (nulls the pointer and zeroes the size)
/// without touching the underlying storage. This is the one
/// deliberate deviation from std::span, required by the buffer
/// static interface: GridHandle::reset() calls buffer.clear().
void clear()
/// without touching the underlying storage -- the view is
/// non-owning, so "destroying" it releases nothing. This is the
/// one deliberate deviation from std::span, required by the buffer
/// static interface the handles consume through reset().
void destroy()
{
mData = nullptr;
mSize = 0;
}
/// @brief Detaches the view.
/// @deprecated Use destroy(): the handles now dispatch to it directly.
[[deprecated("Use cuda::BufferView::destroy instead")]]
void clear() { this->destroy(); }
}; // BufferView<T> class
} // namespace cuda
// Primary template defined in HostBuffer.h; declared here so this header
// stays self-contained without pulling in the host-buffer machinery.
template<typename BufferT>
struct BufferTraits;
/// @brief GridHandle support for the single-space cuda::Buffer: the buffer
/// owns exactly one allocation, resident on the device, so the handle
/// parses metadata through a device read and exposes only the device
/// accessors. Requires byte-addressed storage.
/// @note This trait doubles as the definition of the single-space
/// device-buffer concept: a buffer whose BufferTraits specialization
/// sets hasDeviceSingle guarantees ElementType and ResourceType
/// typedefs, data(), size() and size_bytes() (byte-addressed elements,
/// enforced by the consumer), resource(), copy(), destroy(), and stream()
/// when the resource is stream-ordered. Any consumer of hasDeviceSingle
/// may rely on exactly this interface and nothing more; in particular,
/// scratch allocates through resource() as a cuda::Buffer, so a
/// conforming buffer never needs to be constructible by a consumer.
template<typename T, typename R>
struct BufferTraits<cuda::Buffer<T, R>>
{
static constexpr bool hasDeviceDual = false;
// Device-resident storage; the byte-addressed requirement is enforced by
// the single-space GridHandle constructor, so trait queries stay
// answerable for any element type.
static constexpr bool hasDeviceSingle = !cuda::is_host_accessible_resource<R>::value;
// A buffer over a host-accessible resource (e.g. PinnedResource) is
// host-readable single-space storage: GridHandle parses its metadata on
// the host, exposes the host accessors, and allocates reads and copies
// through the buffer's resource.
static constexpr bool hasHostSingle = cuda::is_host_accessible_resource<R>::value;
};
} // namespace nanovdb
#endif // end of NANOVDB_CUDA_BUFFER_H_HAS_BEEN_INCLUDED
+105 -4
View File
@@ -38,6 +38,7 @@ class DeviceBuffer
uint64_t mSize; // total number of bytes managed by this buffer (assumed to be identical for host and device)
void *mCpuData, **mGpuData; // raw pointers to the host and device buffers
int mDeviceCount, mManaged;// if mManaged is non-zero this class is responsible for allocating and freeing memory buffers. Otherwise this is assumed to be handled externally
cudaEvent_t *mEvents = nullptr;// per-device event marking the last use of each managed device buffer (parallel to mGpuData, length mDeviceCount). Every use waits on this event before issuing work and re-records it afterwards, so the single event transitively covers EVERY stream the buffer has been used on. Frees then wait on it, which orders them after all outstanding work: freeing on the default stream alone is only safe for blocking streams, and freeing on the last-used stream alone is only safe when just one stream was used.
/// @brief Initialize buffer
/// @param size byte size of buffer to be initialized
@@ -46,6 +47,33 @@ class DeviceBuffer
/// @warning size is expected to be non-zero. Use clear() clear buffer!
void init(uint64_t size, int device, cudaStream_t stream);
/// @brief Free every managed device allocation, each ordered after all tracked uses of the
/// buffer, and destroy the tracking events.
/// @param stream Stream the frees are issued on for allocations owned by the CURRENT device.
/// A stream belongs to a single device, so it cannot carry frees for other devices'
/// memory pools; allocations on other devices are freed on their own device's default
/// stream, after switching to that device.
/// @note Destroying an event with a pending wait is safe: CUDA releases it once the device
/// has completed it.
void freeDeviceBuffers(cudaStream_t stream)
{
int current = 0;
cudaCheck(cudaGetDevice(&current));
for (int i = 0; i < mDeviceCount; ++i) {
if (mGpuData[i]) {
const cudaStream_t freeStream = (i == current) ? stream : cudaStream_t{0};
if (i != current) cudaCheck(cudaSetDevice(i));
this->orderAfterPriorUses(i, freeStream);
cudaCheck(util::cuda::freeAsync(mGpuData[i], freeStream));
if (i != current) cudaCheck(cudaSetDevice(current));
}
if (mEvents && mEvents[i]) {
cudaCheck(cudaEventDestroy(mEvents[i]));
mEvents[i] = nullptr;
}
}
}
public:
using PtrT = std::shared_ptr<DeviceBuffer>;
@@ -122,8 +150,10 @@ public:
, mGpuData(other.mGpuData)
, mDeviceCount(other.mDeviceCount)
, mManaged(other.mManaged)
, mEvents(other.mEvents)
{
other.mCpuData = other.mGpuData = nullptr;
other.mEvents = nullptr;
other.mSize = other.mDeviceCount = other.mManaged = 0;
}
@@ -142,6 +172,8 @@ public:
}
/// @brief Destructor frees memory on both the host and device
/// @note Each managed device free waits on that device's tracking event first, so it is
/// ordered after every stream the buffer was used on, not just the most recent one.
~DeviceBuffer() { this->clear(); };
/// @brief Static factory method that return an instance of this buffer
@@ -226,8 +258,59 @@ public:
///////////////////////////////////////////////////////////////////////
/// @brief Order work subsequently issued on @a stream after every prior use of this
/// device buffer, whichever stream those uses were issued on. The consume-side
/// companion of recordUse: an external consumer (e.g. a zero-copy array-interface
/// export) calls this with its own stream before reading, so it cannot observe a
/// partially-written buffer after asynchronous uploads or recorded kernels.
/// @param device Device whose buffer is about to be read
/// @param stream Stream the consumer's work will be issued on
void orderAfterPriorUses(int device, cudaStream_t stream) const
{
if (mEvents && mEvents[device]) cudaCheck(cudaStreamWaitEvent(stream, mEvents[device], 0));
}
/// @brief Record that this buffer's device data was just used on @a stream, so that the
/// buffer's device frees (destructor, move-assignment, clear) are ordered after that
/// work. Uses issued through deviceUpload/deviceDownload are recorded automatically;
/// callers that enqueue their own kernels or copies against the raw pointer returned
/// by deviceData() should call this afterwards. Without it, such work is only safe if
/// it is on a blocking stream (which the free, issued on the default stream, waits on
/// implicitly) or if the caller synchronizes before the buffer is cleared/destroyed.
/// @param device Device whose buffer was used
/// @param stream Stream the work was issued on
/// @note Recording chains across streams: @a stream is first ordered after the previously
/// recorded use (if any) so the single per-device event transitively covers every
/// recorded use, not just the last one. Without this, concurrent uses on streams A
/// then B would leave only B's event, and the device free could run while A's work
/// is still in flight. The side effect is that work subsequently issued on @a stream
/// also waits on the previously recorded use -- acceptable for a shared buffer, where
/// later-recorded consumers observing earlier writes is the expected ordering. Note
/// this also serializes CONCURRENT READERS that record uses (the single event cannot
/// distinguish read-read from write-read); if that ever matters in a profile, the
/// upgrade path is a read/write-separated or per-record event scheme, not a revert.
void recordUse(int device, cudaStream_t stream)
{
if (!mEvents) return;
if (mEvents[device] == nullptr) {// events are per-device, so create it on the right one
int current = 0;
cudaCheck(cudaGetDevice(&current));
if (current != device) cudaCheck(cudaSetDevice(device));
cudaCheck(cudaEventCreateWithFlags(&mEvents[device], cudaEventDisableTiming));
if (current != device) cudaCheck(cudaSetDevice(current));
} else {
// Re-recording MOVES the event; chain first so the new capture also covers the
// prior recorded use (waiting on a never-recorded or completed event is a no-op).
cudaCheck(cudaStreamWaitEvent(stream, mEvents[device], 0));
}
cudaCheck(cudaEventRecord(mEvents[device], stream));
}
/// @brief Retuns a raw pointer to the specified device/GPU buffer managed by this allocator.
/// @warning Note that the pointer can be NULL!
/// @note Work enqueued against this raw pointer is invisible to the buffer's lifetime
/// tracking: on a non-blocking stream, call recordUse afterwards (or synchronize
/// before the buffer is cleared/destroyed) so the device free is ordered after it.
void* deviceData(int device) const {
NANOVDB_ASSERT(device >= 0 && device < mDeviceCount);
return mGpuData[device];
@@ -301,6 +384,10 @@ public:
/// @}
/// @brief De-allocate all memory managed by this allocator and set all pointers to NULL
/// @param stream Stream the device frees are issued on. The frees are additionally ordered
/// after every stream the buffer was used on (via the per-device tracking event), so
/// @a stream selects where the free is enqueued, not what it is ordered against - any
/// stream is safe to pass here regardless of where the buffer was used.
void clear(cudaStream_t stream = 0);
void clear(void* stream){this->clear(cudaStream_t(stream));}
@@ -310,18 +397,22 @@ public:
inline DeviceBuffer& DeviceBuffer::operator=(DeviceBuffer&& other) noexcept
{
if (mManaged) {// first free all the managed data buffers
if (this == &other) return *this;// self-move would free our buffers and then read them back
if (mManaged) {// first free all the managed data buffers, ordered after every use of each
cudaCheck(cudaFreeHost(mCpuData));
for (int i=0; i<mDeviceCount; ++i) cudaCheck(util::cuda::freeAsync(mGpuData[i], 0));
this->freeDeviceBuffers(cudaStream_t{0});
}
delete [] mGpuData;
delete [] mEvents;
mSize = other.mSize;
mCpuData = other.mCpuData;
mGpuData = other.mGpuData;
mDeviceCount = other.mDeviceCount;
mManaged = other.mManaged;
mEvents = other.mEvents;
other.mCpuData = nullptr;
other.mGpuData = nullptr;
other.mEvents = nullptr;
other.mSize = 0;
other.mDeviceCount = 0;
other.mManaged = 0;
@@ -333,6 +424,7 @@ inline void DeviceBuffer::init(uint64_t size, int device, cudaStream_t stream)
if (size==0) return;
cudaCheck(cudaGetDeviceCount(&mDeviceCount));
mGpuData = new void*[mDeviceCount]();// NULL initialization
mEvents = new cudaEvent_t[mDeviceCount]();// NULL initialization; created lazily on first use
NANOVDB_ASSERT(device >= cudaCpuDeviceId && device < mDeviceCount);
if (device == cudaCpuDeviceId) {
cudaCheck(cudaMallocHost((void**)&mCpuData, size)); // un-managed pinned memory on the host (can be slow to access!). Always 32B aligned
@@ -340,6 +432,7 @@ inline void DeviceBuffer::init(uint64_t size, int device, cudaStream_t stream)
} else {
cudaCheck(util::cuda::mallocAsync(mGpuData+device, size, stream)); // un-managed memory on the device, always 32B aligned!
checkPtr(mGpuData[device], "cuda::DeviceBuffer::init: failed to allocate device buffer");
this->recordUse(device, stream);// the free must be ordered after this allocation
}
mSize = size;
mManaged = 1;// i.e. this instance is responsible for allocating and delete memory
@@ -354,7 +447,11 @@ inline void DeviceBuffer::deviceUpload(int device, cudaStream_t stream, bool syn
cudaCheck(util::cuda::mallocAsync(mGpuData+device, mSize, stream)); // un-managed memory on the device, always 32B aligned!
}
checkPtr(mGpuData[device], "uninitialized gpu destination data");
// Order this transfer after any use of the buffer on another stream, then mark it as the
// latest use, so the tracking event keeps covering every stream the buffer has seen.
this->orderAfterPriorUses(device, stream);
cudaCheck(cudaMemcpyAsync(mGpuData[device], mCpuData, mSize, cudaMemcpyHostToDevice, stream));
this->recordUse(device, stream);
if (sync) cudaCheck(cudaStreamSynchronize(stream));
} // DeviceBuffer::deviceUpload
@@ -374,7 +471,9 @@ inline void DeviceBuffer::deviceDownload(int device, cudaStream_t stream, bool s
cudaCheck(cudaMallocHost((void**)&mCpuData, mSize)); // un-managed pinned memory on the host (can be slow to access!). Always 32B aligned
}
checkPtr(mCpuData, "uninitialized cpu destination data");
this->orderAfterPriorUses(device, stream);
cudaCheck(cudaMemcpyAsync(mCpuData, mGpuData[device], mSize, cudaMemcpyDeviceToHost, stream));
this->recordUse(device, stream);
if (sync) cudaCheck(cudaStreamSynchronize(stream));
} // DeviceBuffer::deviceDownload
@@ -387,13 +486,15 @@ inline void DeviceBuffer::deviceDownload(void* stream, bool sync)
inline void DeviceBuffer::clear(cudaStream_t stream)
{
if (mManaged) {// free all the managed data buffers
if (mManaged) {// free all the managed data buffers, ordered after every use of each
cudaCheck(cudaFreeHost(mCpuData));
for (int i=0; i<mDeviceCount; ++i) cudaCheck(util::cuda::freeAsync(mGpuData[i], stream));
this->freeDeviceBuffers(stream);
}
delete [] mGpuData;
delete [] mEvents;
mCpuData = nullptr;
mGpuData = nullptr;
mEvents = nullptr;
mSize = 0;
mDeviceCount = 0;
mManaged = 0;
+190
View File
@@ -123,6 +123,17 @@ struct is_async_resource<R, std::void_t<
decltype(std::declval<R&>().deallocate(std::declval<void*>(), size_t{0}, size_t{0}))>>
: std::true_type {};
/// @brief Detection trait: @c is_host_accessible_resource<R>::value is true
/// iff @c R declares `static constexpr bool HOST_ACCESSIBLE = true`,
/// i.e. its allocations are mapped into the host address space (e.g.
/// PinnedResource). Defaults to false: allocations are device-resident.
/// @note Unlike the void_t detections above, this checks the member's VALUE:
/// a resource declaring HOST_ACCESSIBLE = false stays device-resident.
template<typename R, typename = void>
struct is_host_accessible_resource : std::false_type {};
template<typename R>
struct is_host_accessible_resource<R, typename std::enable_if<bool(R::HOST_ACCESSIBLE)>::type> : std::true_type {};
/// @brief Detection trait: @c is_resource<R>::value is true iff @c R models
/// the synchronous Resource concept, i.e. exposes
/// allocate(size_t, size_t) and deallocate(void*, size_t, size_t).
@@ -145,6 +156,185 @@ struct is_resource<R, std::void_t<
decltype(std::declval<R&>().deallocate(std::declval<void*>(), size_t{0}, size_t{0}))>>
: std::true_type {};
/// @brief CRTP base supplying the synchronous half of the resource concept in
/// terms of the stream-ordered half, so a custom stream-ordered resource
/// only has to write allocate_async and deallocate_async.
/// @tparam Derived the resource deriving from this base
/// @details A stream-ordered resource must also model the synchronous concept
/// (is_async_resource implies is_resource), which means writing four
/// methods where two would do. The synchronous pair is not a bare
/// delegate: memory from allocate must be usable immediately on any
/// stream, so the null-stream allocation has to be synchronized before
/// it is returned. Omitting that synchronization yields memory that
/// satisfies the concept but is not actually synchronous -- a race
/// rather than a compile error -- so it lives here rather than being
/// rewritten per resource.
/// @code
/// struct MyResource : nanovdb::cuda::SyncFromAsync<MyResource> {
/// static constexpr size_t DEFAULT_ALIGNMENT = 256;
/// void* allocate_async(size_t bytes, size_t alignment, cudaStream_t stream);
/// void deallocate_async(void* p, size_t bytes, size_t alignment, cudaStream_t stream);
/// };
/// @endcode
template <class Derived>
struct SyncFromAsync
{
/// @brief Allocates @c bytes usable on any stream when this returns.
/// @param bytes number of bytes to allocate
/// @param alignment requested alignment
/// @note Every call synchronizes the null stream; on hot paths prefer the
/// stream-ordered pair.
void* allocate(size_t bytes, size_t alignment)
{
void* p = static_cast<Derived&>(*this).allocate_async(bytes, alignment, cudaStream_t{0});
cudaCheck(cudaStreamSynchronize(cudaStream_t{0}));
return p;
}
/// @brief Frees @c p on the null stream.
/// @param p pointer previously returned by allocate
/// @param bytes size passed to the matching allocate
/// @param alignment alignment passed to the matching allocate
/// @note No synchronization here: the synchronous concept's contract is
/// that the memory is already quiescent when deallocate is called.
void deallocate(void* p, size_t bytes, size_t alignment)
{
static_cast<Derived&>(*this).deallocate_async(p, bytes, alignment, cudaStream_t{0});
}
};
/// @brief Synchronous device memory resource backed by cudaMalloc/cudaFree.
/// Models only the Resource concept: it never touches stream-ordered
/// allocation, so it works on devices without memory-pool support
/// (cudaDevAttrMemoryPoolsSupported == 0), where DeviceResource's
/// cudaMallocAsync path fails by design. Pair with AsyncFromSync to
/// drive the stream-ordered builders on such a device.
class MallocResource
{
public:
// cudaMalloc aligns memory to 256 bytes by default
static constexpr size_t DEFAULT_ALIGNMENT = 256;
/// @brief Allocates @c bytes with cudaMalloc; valid on every stream when
/// this returns. A zero request returns nullptr.
void* allocate(size_t bytes, size_t)
{
if (bytes == 0) return nullptr;
void* p = nullptr;
cudaCheck(cudaMalloc(&p, bytes));
return p;
}
/// @brief Frees @c p with cudaFree; the caller guarantees that device work
/// touching the memory has completed.
void deallocate(void* p, size_t, size_t) { cudaCheck(cudaFree(p)); }
};// MallocResource
/// @brief Wrapper presenting a synchronous resource as a stream-ordered one,
/// so it can drive components that require the AsyncResource concept
/// (TempPool and the GPU builders).
/// @tparam R the wrapped synchronous resource, held by value; wrap a
/// ResourceRef<R> to borrow a stateful instance instead.
/// @details The mirror of SyncFromAsync, and the analog of cuda::mr's
/// synchronous_resource_adapter. allocate_async forwards to
/// R::allocate, whose memory is immediately valid on every stream --
/// a stronger guarantee than stream-ordering requires.
/// deallocate_async synchronizes @c stream before R::deallocate,
/// establishing the quiescence the synchronous contract demands.
/// @warning Every deallocation synchronizes its stream, so expect
/// serialization relative to a genuinely stream-ordered resource.
/// That is the unavoidable cost of a synchronous backend under a
/// stream-ordered algorithm; this wrapper exists so the cost is
/// explicit and chosen by the caller -- e.g. on a device without
/// memory-pool support -- rather than silently substituted.
template<class R>
struct AsyncFromSync
{
static_assert(is_resource<R>::value,
"AsyncFromSync requires R to model the synchronous Resource concept");
static constexpr size_t DEFAULT_ALIGNMENT = R::DEFAULT_ALIGNMENT;
/// @brief The adapter is host-accessible iff the adapted resource is.
static constexpr bool HOST_ACCESSIBLE = is_host_accessible_resource<R>::value;
R resource;
/// @brief Allocates through the synchronous resource; the result is valid
/// on every stream, hence trivially valid on @c stream.
void* allocate_async(size_t bytes, size_t alignment, cudaStream_t) { return resource.allocate(bytes, alignment); }
/// @brief Synchronizes @c stream, then frees through the synchronous
/// resource -- the synchronize makes the quiescence contract hold.
/// Null is a no-op and skips the synchronize.
void deallocate_async(void* p, size_t bytes, size_t alignment, cudaStream_t stream)
{
if (p == nullptr) return;
cudaCheck(cudaStreamSynchronize(stream));
resource.deallocate(p, bytes, alignment);
}
/// @brief Synchronous pair, forwarding to the wrapped resource.
void* allocate(size_t bytes, size_t alignment) { return resource.allocate(bytes, alignment); }
void deallocate(void* p, size_t bytes, size_t alignment) { resource.deallocate(p, bytes, alignment); }
};// AsyncFromSync<R>
/// @brief Non-owning reference to a memory resource that is itself a resource:
/// copying the ref shares the underlying resource rather than copying it.
/// @tparam R the referenced resource type
/// @details Types that hold their resource by value -- cuda::Buffer, matching
/// cuda::buffer -- select their ownership semantics by what is placed
/// in that slot: a concrete resource is owned as a copy, while a
/// ResourceRef borrows. This is the same division cuda::mr draws
/// between any_resource (owning) and resource_ref (borrowing), and the
/// same shape as std::pmr::polymorphic_allocator over memory_resource*.
/// Use it when a resource is stateful or long-lived and a container
/// must allocate through *that* instance rather than a copy of it.
/// @warning The referenced resource must outlive every use of this ref and of
/// all copies of it, including any container holding one.
template <class R>
struct ResourceRef
{
static_assert(is_async_resource<R>::value || is_resource<R>::value,
"ResourceRef requires R to model the AsyncResource or the Resource concept");
static constexpr size_t DEFAULT_ALIGNMENT = R::DEFAULT_ALIGNMENT;
/// @brief A reference is host-accessible iff the referenced resource is.
static constexpr bool HOST_ACCESSIBLE = is_host_accessible_resource<R>::value;
/// @brief Constructs a ref borrowing @c resource.
/// @param resource resource to allocate from; must outlive this ref
ResourceRef(R& resource) : mResource(&resource) {}
/// @{
/// @brief Stream-ordered pair, present only when @c R models AsyncResource,
/// so a ref over a synchronous resource does not misreport its tier.
template<class S = R, std::enable_if_t<is_async_resource<S>::value, int> = 0>
void* allocate_async(size_t bytes, size_t alignment, cudaStream_t stream)
{
return mResource->allocate_async(bytes, alignment, stream);
}
template<class S = R, std::enable_if_t<is_async_resource<S>::value, int> = 0>
void deallocate_async(void* p, size_t bytes, size_t alignment, cudaStream_t stream)
{
mResource->deallocate_async(p, bytes, alignment, stream);
}
/// @}
/// @brief Synchronous pair, forwarding to the referenced resource.
void* allocate(size_t bytes, size_t alignment) { return mResource->allocate(bytes, alignment); }
void deallocate(void* p, size_t bytes, size_t alignment) { mResource->deallocate(p, bytes, alignment); }
/// @brief Two refs compare equal iff they reference the same resource, i.e.
/// memory allocated through one may be deallocated through the other.
friend bool operator==(ResourceRef lhs, ResourceRef rhs) { return lhs.mResource == rhs.mResource; }
friend bool operator!=(ResourceRef lhs, ResourceRef rhs) { return lhs.mResource != rhs.mResource; }
private:
R* mResource;
};// ResourceRef<R>
}
} // namespace nanovdb::cuda
+253 -23
View File
@@ -17,21 +17,20 @@
#ifndef NANOVDB_CUDA_GRIDHANDLE_CUH_HAS_BEEN_INCLUDED
#define NANOVDB_CUDA_GRIDHANDLE_CUH_HAS_BEEN_INCLUDED
#include <nanovdb/cuda/Buffer.h>// for the resource-aware scratch buffers below
#include <nanovdb/cuda/DeviceBuffer.h>// required for instantiation of move c-tor of GridHandle
#include <nanovdb/tools/cuda/GridChecksum.cuh>// for cuda::updateChecksum
#include <nanovdb/GridHandle.h>
#include <string>// for the grid index in chain-validation error messages
#include <type_traits>// for std::is_default_constructible in cuda::copyTo
namespace nanovdb {
namespace cuda {
namespace detail {
static __global__ void cpyGridHandleMeta(const GridData *d_data, GridHandleMetaData *d_meta)
{
nanovdb::cpyGridHandleMeta(d_data, d_meta);
}
static __global__ void updateGridCount(GridData *d_data, uint32_t gridIndex, uint32_t gridCount, bool *d_dirty)
{
NANOVDB_ASSERT(gridIndex < gridCount);
@@ -43,6 +42,145 @@ static __global__ void updateGridCount(GridData *d_data, uint32_t gridIndex, uin
}
}
/// @brief Defect classes the device-side chain walk can report, mapped to
/// exceptions on the host by parseGridChain.
enum class ChainError : uint32_t { Ok = 0, Truncated, Invalid, Inconsistent, BadSize };
/// @brief Result of the device-side chain walk: the defect class and the
/// index of the grid the walk failed at.
struct ChainStatus { ChainError error; uint32_t gridIndex; };
/// @brief Validates every header of the grid chain and fills the metadata
/// scratch in a single pass. Bounds are checked before every header
/// read, so a truncated buffer or a forged header can never cause an
/// out-of-bounds access.
static __global__ void parseGridChainKernel(const GridData *d_head, uint64_t bytes, uint32_t count,
GridHandleMetaData *d_meta, ChainStatus *d_status)
{
*d_status = ChainStatus{ChainError::Ok, 0u};
uint64_t offset = 0;
for (uint32_t i = 0; i < count; ++i) {
if (offset + sizeof(GridData) > bytes) { *d_status = {ChainError::Truncated, i}; return; }
const GridData *data = util::PtrAdd<GridData>(d_head, offset);
if (!data->isValid()) { *d_status = {ChainError::Invalid, i}; return; }
if (data->mGridIndex != i || data->mGridCount != count) { *d_status = {ChainError::Inconsistent, i}; return; }
if (data->mGridSize < sizeof(GridData) || data->mGridSize > bytes - offset) { *d_status = {ChainError::BadSize, i}; return; }
d_meta[i] = GridHandleMetaData{offset, data->mGridSize, data->mGridType};
offset += data->mGridSize;
}
}
/// @brief Reads and validates the head of the grid chain, returning the grid
/// count the chain claims. This is the one synchronizing header read
/// that sizing the metadata scratch requires; the rest of the chain is
/// validated on the device by parseGridChain.
inline uint32_t validGridChainHead(const GridData *d_head, uint64_t bytes, cudaStream_t stream)
{
if (bytes < sizeof(GridData))
throw std::runtime_error("GridHandle: grid chain exceeds the device buffer (truncated or corrupt grid data)");
GridData tmp;
cudaCheck(cudaMemcpyAsync(&tmp, d_head, sizeof(GridData), cudaMemcpyDeviceToHost, stream));
cudaCheck(cudaStreamSynchronize(stream));
if (!tmp.isValid()) throw std::runtime_error("GridHandle was constructed with an invalid device buffer");
if (tmp.mGridCount == 0) throw std::runtime_error("GridHandle: device buffer contains no grids");
if (tmp.mGridIndex != 0)
throw std::runtime_error("GridHandle: inconsistent grid index/count in the device buffer's grid chain");
if (uint64_t(tmp.mGridCount) > bytes / sizeof(GridData))// every grid is at least one full header
throw std::runtime_error("GridHandle: grid chain exceeds the device buffer (truncated or corrupt grid data)");
return tmp.mGridCount;
}
/// @brief Bytes of device scratch parseGridChain needs for @c count grids:
/// the metadata array followed by the status word.
inline constexpr uint64_t chainScratchSize(uint32_t count)
{
return count * sizeof(GridHandleMetaData) + sizeof(ChainStatus);
}
/// @brief Launches the combined validate-and-parse walk over the grid chain
/// and fills @c meta from its scratch: one kernel, one readback and
/// one synchronization regardless of the grid count.
/// @param scratch device scratch of at least chainScratchSize(count) bytes
/// @throw std::runtime_error naming the defect when any header fails validation
template<typename ScratchBufferT>
inline void parseGridChain(const GridData *d_head, uint64_t bytes, uint32_t count,
ScratchBufferT &scratch, std::vector<GridHandleMetaData> &meta, cudaStream_t stream)
{
auto *d_meta = reinterpret_cast<GridHandleMetaData*>(scratch.data());
auto *d_status = reinterpret_cast<ChainStatus*>(scratch.data() + count * sizeof(GridHandleMetaData));
parseGridChainKernel<<<1, 1, 0, stream>>>(d_head, bytes, count, d_meta, d_status);
cudaCheckError();
meta.resize(count);
ChainStatus status;
cudaCheck(cudaMemcpyAsync(meta.data(), d_meta, count * sizeof(GridHandleMetaData), cudaMemcpyDeviceToHost, stream));
cudaCheck(cudaMemcpyAsync(&status, d_status, sizeof(ChainStatus), cudaMemcpyDeviceToHost, stream));
cudaCheck(cudaStreamSynchronize(stream));
if (status.error == ChainError::Ok) return;
const std::string where = " (grid " + std::to_string(status.gridIndex) + " of " + std::to_string(count) + ")";
switch (status.error) {
case ChainError::Truncated:
throw std::runtime_error("GridHandle: grid chain exceeds the device buffer (truncated or corrupt grid data)" + where);
case ChainError::Inconsistent:
throw std::runtime_error("GridHandle: inconsistent grid index/count in the device buffer's grid chain" + where);
case ChainError::BadSize:
throw std::runtime_error("GridHandle: grid size field exceeds the device buffer (truncated or corrupt grid data)" + where);
default:
throw std::runtime_error("GridHandle was constructed with an invalid device buffer" + where);
}
}
/// @brief The buffer's retained stream when its resource is stream-ordered,
/// the default stream otherwise.
template<typename BufferT>
inline cudaStream_t retainedStreamOrDefault(const BufferT& buf)
{
if constexpr (is_async_resource<typename BufferT::ResourceType>::value) return buf.stream();
else { (void)buf; return cudaStream_t(0); }
}
/// @brief Constructs the metadata scratch through @c buf's resource, on
/// @c stream for a stream-ordered resource.
template<typename ScratchT, typename BufferT>
inline ScratchT makeMetaScratch(const BufferT& buf, uint64_t count, cudaStream_t stream)
{
if constexpr (is_async_resource<typename BufferT::ResourceType>::value)
return ScratchT(stream, buf.resource(), count, noInit);
else { (void)stream; return ScratchT(buf.resource(), count, noInit); }
}
/// @brief Allocates @c bytes of destination storage for a cross-space
/// transfer: single-space buffers allocate through @c proto's resource
/// (or a default-constructed resource without one), on @c stream when
/// the resource is stream-ordered; buffers providing create() go
/// through it.
template<typename DstBufferT>
inline DstBufferT makeTransferStorage(uint64_t bytes, cudaStream_t stream, const DstBufferT* proto)
{
if constexpr (BufferHasDeviceSingle<DstBufferT>::value || BufferHasHostSingle<DstBufferT>::value) {
using ResourceT = typename DstBufferT::ResourceType;
if (!proto) {
// both branches of a plain conditional would instantiate the
// default-resource constructor, breaking non-default-constructible
// resources (e.g. ResourceRef) even for callers that pass a proto
if constexpr (std::is_default_constructible<ResourceT>::value) {
if constexpr (is_async_resource<ResourceT>::value) return DstBufferT(stream, bytes, noInit);
else return DstBufferT(bytes, noInit);
} else {
throw std::runtime_error("cuda::copyTo: a destination buffer over a non-default-constructible "
"resource requires a prototype buffer");
}
}
if constexpr (is_async_resource<ResourceT>::value) {
return DstBufferT(stream, proto->resource(), bytes, noInit);
} else {
(void)stream;
return DstBufferT(proto->resource(), bytes, noInit);
}
} else {
return DstBufferT::create(bytes, proto);
}
}
}// namespace detail
template<typename BufferT, template <class, class...> class VectorT = std::vector>
@@ -52,10 +190,11 @@ splitGridHandles(const GridHandle<BufferT> &handle, const BufferT* other = nullp
const void *ptr = handle.deviceData();
if (ptr == nullptr) return VectorT<GridHandle<BufferT>>();
VectorT<GridHandle<BufferT>> handles(handle.gridCount());
bool dirty, *d_dirty;// use this to check if the checksum needs to be recomputed
cudaCheck(util::cuda::mallocAsync((void**)&d_dirty, sizeof(bool), stream));
Buffer<bool, DeviceResource> dirtyBuf(stream, 1, noInit);
bool *d_dirty = dirtyBuf.data();
int device = util::cuda::currentDevice();
for (uint32_t n=0; n<handle.gridCount(); ++n) {
bool dirty = false;// set when the checksum needs to be recomputed
auto buffer = BufferT::create(handle.gridSize(n), other, device, stream);
GridData *dst = reinterpret_cast<GridData*>(buffer.deviceData());
const GridData *src = reinterpret_cast<const GridData*>(ptr);
@@ -68,7 +207,6 @@ splitGridHandles(const GridHandle<BufferT> &handle, const BufferT* other = nullp
handles[n] = nanovdb::GridHandle<BufferT>(std::move(buffer));
ptr = util::PtrAdd(ptr, handle.gridSize(n));
}
cudaCheck(util::cuda::freeAsync(d_dirty, stream));
return handles;
}// cuda::splitGridHandles
@@ -85,11 +223,12 @@ mergeGridHandles(const VectorT<GridHandle<BufferT>> &handles, const BufferT* oth
int device = util::cuda::currentDevice();
auto buffer = BufferT::create(size, other, device, stream);
void *dst = buffer.deviceData();
bool dirty, *d_dirty;// use this to check if the checksum needs to be recomputed
cudaCheck(util::cuda::mallocAsync((void**)&d_dirty, sizeof(bool), stream));
Buffer<bool, DeviceResource> dirtyBuf(stream, 1, noInit);
bool *d_dirty = dirtyBuf.data();
for (auto &h : handles) {
const void *src = h.deviceData();
for (uint32_t n=0; n<h.gridCount(); ++n) {
bool dirty = false;// set when the checksum needs to be recomputed
cudaCheck(cudaMemcpyAsync(dst, src, h.gridSize(n), cudaMemcpyDeviceToDevice, stream));
GridData *data = reinterpret_cast<GridData*>(dst);
detail::updateGridCount<<<1, 1, 0, stream>>>(data, counter++, gridCount, d_dirty);
@@ -101,10 +240,82 @@ mergeGridHandles(const VectorT<GridHandle<BufferT>> &handles, const BufferT* oth
src = util::PtrAdd(src, h.gridSize(n));
}
}
cudaCheck(util::cuda::freeAsync(d_dirty, stream));
return GridHandle<BufferT>(std::move(buffer));
}// cuda::mergeGridHandles
/// @brief Deep-copies a grid handle into a different address space: the
/// explicit, stream-carrying transfer between single-space device
/// handles and host-readable handles (HostBuffer or a host-accessible
/// single-space buffer such as a pinned-resource cuda::Buffer).
/// @tparam DstBufferT destination buffer type (specify explicitly)
/// @param src the handle to copy; must not be dual-space (use
/// deviceUpload/deviceDownload on those)
/// @param stream stream the copy is issued on; a device destination buffer
/// with a stream-ordered resource retains it
/// @warning Passing a stream other than the source buffer's retained stream
/// makes the caller responsible for ordering: prior work on the
/// source (and the source's later destruction, which frees on its
/// own stream) must be ordered against @a stream by the caller,
/// e.g. with cudaStreamWaitEvent or a synchronization. The
/// stream-less overload below has no such requirement.
/// @param proto optional buffer whose resource (or pool, for buffers
/// providing create()) allocates the destination storage; without it
/// the destination resource is default-constructed
/// @return a handle of the destination buffer type with equal contents
/// @details The returned handle is immediately usable: a host-readable
/// destination synchronizes @c stream before returning, and a device
/// destination parses (and validates) its metadata on the
/// transferred bytes, which synchronizes internally. A pageable host
/// source or destination (HostBuffer) degrades the copy to
/// synchronous behavior; pinned single-space handles keep it
/// asynchronous.
template<typename DstBufferT, typename SrcBufferT>
inline GridHandle<DstBufferT> copyTo(const GridHandle<SrcBufferT>& src, cudaStream_t stream, const DstBufferT* proto = nullptr)
{
constexpr bool srcDev = BufferHasDeviceSingle<SrcBufferT>::value;
constexpr bool dstDev = BufferHasDeviceSingle<DstBufferT>::value;
static_assert(!BufferTraits<SrcBufferT>::hasDeviceDual && !BufferTraits<DstBufferT>::hasDeviceDual,
"cuda::copyTo does not support dual-space buffers: use deviceUpload/deviceDownload on the handle");
static_assert(srcDev || dstDev,
"cuda::copyTo is for cross-space transfers involving a device buffer: use GridHandle::copy for host-to-host");
const uint64_t bytes = src.bufferSize();
if (bytes == 0u) {
if constexpr (std::is_default_constructible<DstBufferT>::value) {
return GridHandle<DstBufferT>();
} else {
throw std::runtime_error("cuda::copyTo: an empty handle cannot be copied to a buffer type "
"that is not default-constructible");
}
}
DstBufferT dst = detail::makeTransferStorage<DstBufferT>(bytes, stream, proto);
const void* srcPtr;
if constexpr (srcDev) srcPtr = src.deviceData();
else srcPtr = src.data();
constexpr cudaMemcpyKind kind = srcDev ? (dstDev ? cudaMemcpyDeviceToDevice : cudaMemcpyDeviceToHost)
: cudaMemcpyHostToDevice;
cudaCheck(cudaMemcpyAsync(dst.data(), srcPtr, bytes, kind, stream));
if constexpr (dstDev) {
// A synchronous destination resource retains no stream: order the
// metadata parse (which runs on the default stream) after the copy.
if constexpr (!is_async_resource<typename DstBufferT::ResourceType>::value)
cudaCheck(cudaStreamSynchronize(stream));
} else {
cudaCheck(cudaStreamSynchronize(stream));// the host-readable result is the postcondition
}
return GridHandle<DstBufferT>(std::move(dst));// the constructor parses (and validates) the metadata
}// cuda::copyTo
/// @brief Convenience overload issuing the copy on the source buffer's
/// retained stream when it has one (any single-space source over a
/// stream-ordered resource), the default stream otherwise.
template<typename DstBufferT, typename SrcBufferT>
inline GridHandle<DstBufferT> copyTo(const GridHandle<SrcBufferT>& src, const DstBufferT* proto = nullptr)
{
cudaStream_t stream = 0;
if constexpr (BufferHasStream<SrcBufferT>::value) stream = src.buffer().stream();
return copyTo<DstBufferT>(src, stream, proto);
}// cuda::copyTo (retained stream)
}// namespace cuda
template<typename BufferT, template <class, class...> class VectorT = std::vector>
@@ -122,30 +333,49 @@ mergeDeviceGrids(const VectorT<GridHandle<BufferT>> &handles, const BufferT* oth
template<typename BufferT>
template<typename T, typename util::enable_if<BufferTraits<T>::hasDeviceDual, int>::type>
GridHandle<BufferT>::GridHandle(T&& buffer)
: mBuffer(std::move(buffer))
{
static_assert(util::is_same<T,BufferT>::value, "Expected U==BufferT");
mBuffer = std::move(buffer);
if (auto *data = reinterpret_cast<const GridData*>(mBuffer.data())) {
if (!data->isValid()) throw std::runtime_error("GridHandle was constructed with an invalid host buffer");
mMetaData.resize(data->mGridCount);
cpyGridHandleMeta(data, mMetaData.data());
} else {
if (auto *d_data = reinterpret_cast<const GridData*>(mBuffer.deviceData())) {
GridData tmp;
cudaCheck(cudaMemcpy(&tmp, d_data, sizeof(GridData), cudaMemcpyDeviceToHost));
if (!tmp.isValid()) throw std::runtime_error("GridHandle was constructed with an invalid device buffer");
GridHandleMetaData *d_metaData;
cudaMalloc((void**)&d_metaData, tmp.mGridCount*sizeof(GridHandleMetaData));
cuda::detail::cpyGridHandleMeta<<<1,1>>>(d_data, d_metaData);
mMetaData.resize(tmp.mGridCount);
cudaCheck(cudaMemcpy(mMetaData.data(), d_metaData,tmp.mGridCount*sizeof(GridHandleMetaData), cudaMemcpyDeviceToHost));
cudaCheck(cudaFree(d_metaData));
const uint32_t count = cuda::detail::validGridChainHead(d_data, mBuffer.size(), cudaStream_t(0));
// MallocResource: plain cudaMalloc, so this long-standing parse path
// keeps working on devices without memory-pool support.
cuda::Buffer<std::byte, cuda::MallocResource> scratch(cuda::detail::chainScratchSize(count), cuda::noInit);
cuda::detail::parseGridChain(d_data, mBuffer.size(), count, scratch, mMetaData, cudaStream_t(0));
}
}
}// GridHandle(T&& buffer)
// Dummy function that ensures instantiation of the move-constructor above when BufferT=cuda::DeviceBuffer
namespace {auto __dummy(){return GridHandle<cuda::DeviceBuffer>(std::move(cuda::DeviceBuffer()));}}
// move constructor from a single-space device buffer: all device work runs on
// the buffer's retained stream (or the default stream for a synchronous
// resource), and the metadata scratch allocates through the buffer's resource.
template<typename BufferT>
template<typename T, typename util::enable_if<BufferHasDeviceSingle<T>::value, int>::type, typename>
GridHandle<BufferT>::GridHandle(T&& buffer)
: mBuffer(std::move(buffer))
{
static_assert(util::is_same<T,BufferT>::value, "Expected U==BufferT");
using ResourceT = typename BufferT::ResourceType;
if (const GridData *d_data = reinterpret_cast<const GridData*>(mBuffer.data())) {
const cudaStream_t stream = cuda::detail::retainedStreamOrDefault(mBuffer);
const uint32_t count = cuda::detail::validGridChainHead(d_data, mBuffer.size_bytes(), stream);
using ScratchT = cuda::Buffer<std::byte, ResourceT>;
ScratchT scratch = cuda::detail::makeMetaScratch<ScratchT>(mBuffer, cuda::detail::chainScratchSize(count), stream);
cuda::detail::parseGridChain(d_data, mBuffer.size_bytes(), count, scratch, mMetaData, stream);
}
}// GridHandle(T&& buffer) for single-space device buffers
// Emit the dual-buffer move constructor from every CUDA translation unit, so
// host-only translation units that construct GridHandle<cuda::DeviceBuffer>
// (they see only the declaration) always find the symbol at link time. An
// unused private function is not enough: the optimizer may drop the
// complete-object constructor it instantiates.
template GridHandle<cuda::DeviceBuffer>::GridHandle(cuda::DeviceBuffer&&);
} // namespace nanovdb
+91 -3
View File
@@ -18,6 +18,7 @@
#define NANOVDB_CUDA_NODE_MANAGER_CUH_HAS_BEEN_INCLUDED
#include <nanovdb/util/cuda/Util.h>// for cuda::lambdaKernel
#include <nanovdb/cuda/Buffer.h>// for the resource-aware buffers below
#include <nanovdb/cuda/DeviceBuffer.h>
#include <nanovdb/NodeManager.h>
@@ -25,6 +26,23 @@ namespace nanovdb {
namespace cuda {
namespace detail {
/// @brief Constructs a single-space buffer of @c count elements through
/// @c ref, on @c stream when the resource is stream-ordered.
template<typename BufferT, typename RefT>
inline BufferT makeNodeManagerStorage(cudaStream_t stream, const RefT& ref, uint64_t count)
{
if constexpr (is_async_resource<typename BufferT::ResourceType>::value) {
return BufferT(stream, ref, count, noInit);
} else {
(void)stream;
return BufferT(ref, count, noInit);
}
}
}// namespace detail
/// @brief Construct a NodeManager from a device grid pointer
///
/// @param d_grid device grid pointer whose nodes will be accessed sequentially
@@ -41,8 +59,8 @@ createNodeManager(const NanoGrid<BuildT> *d_grid,
cudaCheck(cudaGetDevice(&device));
auto buffer = BufferT::create(sizeof(NodeManagerData), &pool, device, stream);
auto *d_data = (NodeManagerData*)buffer.deviceData();
size_t size = 0u, *d_size;
cudaCheck(util::cuda::mallocAsync((void**)&d_size, sizeof(size_t), stream));
Buffer<size_t, DeviceResource> sizeBuf(stream, 1, noInit);
size_t size = 0u, *d_size = sizeBuf.data();
util::cuda::lambdaKernel<<<1, 1, 0, stream>>>(1, [=] __device__(size_t) {
*d_data = NodeManagerData((void*)d_grid);
*d_size = sizeof(NodeManagerData);
@@ -58,7 +76,7 @@ createNodeManager(const NanoGrid<BuildT> *d_grid,
});
cudaCheckError();
cudaCheck(cudaMemcpyAsync(&size, d_size, sizeof(size_t), cudaMemcpyDeviceToHost, stream));
cudaCheck(util::cuda::freeAsync(d_size, stream));
cudaCheck(cudaStreamSynchronize(stream));// the branch below reads the copied size on the host
if (size > sizeof(NodeManagerData)) {
auto tmp = BufferT::create(size, &pool, device, stream);// only allocate buffer on the device
cudaCheck(cudaMemcpyAsync(tmp.deviceData(), buffer.deviceData(), sizeof(NodeManagerData), cudaMemcpyDeviceToDevice, stream));
@@ -85,6 +103,76 @@ createNodeManager(const NanoGrid<BuildT> *d_grid,
return NodeManagerHandle<BufferT>(toGridType<BuildT>(), std::move(buffer));
}// cuda::createNodeManager
/// @brief Construct a NodeManager over a single-space device buffer whose
/// storage allocates through @a resource, which the returned handle
/// borrows (it must outlive the handle) -- the same ownership contract
/// as the builders.
///
/// @param d_grid device grid pointer whose nodes will be accessed sequentially
/// @param resource device-resident memory resource the handle's storage
/// allocates through; a host-accessible resource (e.g. PinnedResource)
/// is excluded, since the result would carry no device NodeManager
/// @param stream cuda stream
/// @return Handle over cuda::Buffer<std::byte, ResourceRef<ResourceT>> that
/// contains a device NodeManager
template <typename BuildT, typename ResourceT>
inline typename util::enable_if<(is_resource<ResourceT>::value || is_async_resource<ResourceT>::value)
&& !is_host_accessible_resource<ResourceT>::value,
NodeManagerHandle<Buffer<std::byte, ResourceRef<ResourceT>>>>::type
createNodeManager(const NanoGrid<BuildT> *d_grid,
ResourceT& resource,
cudaStream_t stream = 0)
{
using RefT = ResourceRef<ResourceT>;
using BufferT = Buffer<std::byte, RefT>;
RefT ref(resource);
auto buffer = detail::makeNodeManagerStorage<BufferT>(stream, ref, sizeof(NodeManagerData));
auto *d_data = reinterpret_cast<NodeManagerData*>(buffer.data());
auto sizeBuf = detail::makeNodeManagerStorage<Buffer<size_t, RefT>>(stream, ref, 1);
size_t size = 0u, *d_size = sizeBuf.data();
util::cuda::lambdaKernel<<<1, 1, 0, stream>>>(1, [=] __device__(size_t) {
*d_data = NodeManagerData((void*)d_grid);
*d_size = sizeof(NodeManagerData);
auto &tree = d_grid->tree();
if (NodeManager<BuildT>::FIXED_SIZE && d_grid->isBreadthFirst()) {
d_data->mLinear = uint8_t(1u);
d_data->mOff[0] = util::PtrDiff(tree.template getFirstNode<0>(), d_grid);
d_data->mOff[1] = util::PtrDiff(tree.template getFirstNode<1>(), d_grid);
d_data->mOff[2] = util::PtrDiff(tree.template getFirstNode<2>(), d_grid);
} else {
*d_size += sizeof(uint64_t)*tree.totalNodeCount();
}
});
cudaCheckError();
cudaCheck(cudaMemcpyAsync(&size, d_size, sizeof(size_t), cudaMemcpyDeviceToHost, stream));
cudaCheck(cudaStreamSynchronize(stream));// the branch below reads the copied size on the host
if (size > sizeof(NodeManagerData)) {
auto tmp = detail::makeNodeManagerStorage<BufferT>(stream, ref, size);
cudaCheck(cudaMemcpyAsync(tmp.data(), buffer.data(), sizeof(NodeManagerData), cudaMemcpyDeviceToDevice, stream));
buffer = std::move(tmp);
d_data = reinterpret_cast<NodeManagerData*>(buffer.data());
util::cuda::lambdaKernel<<<1, 1, 0, stream>>>(1, [=] __device__ (size_t) {
auto &tree = d_grid->tree();
int64_t *ptr0 = d_data->mPtr[0] = reinterpret_cast<int64_t*>(d_data + 1);
int64_t *ptr1 = d_data->mPtr[1] = d_data->mPtr[0] + tree.nodeCount(0);
int64_t *ptr2 = d_data->mPtr[2] = d_data->mPtr[1] + tree.nodeCount(1);
// Performs depth first traversal but breadth first insertion
for (auto it2 = tree.root().cbeginChild(); it2; ++it2) {
*ptr2++ = util::PtrDiff(&*it2, d_grid);
for (auto it1 = it2->cbeginChild(); it1; ++it1) {
*ptr1++ = util::PtrDiff(&*it1, d_grid);
for (auto it0 = it1->cbeginChild(); it0; ++it0) {
*ptr0++ = util::PtrDiff(&*it0, d_grid);
}// loop over child nodes of the lower internal node
}// loop over child nodes of the upper internal node
}// loop over child nodes of the root node
});
cudaCheckError();
}
return NodeManagerHandle<BufferT>(toGridType<BuildT>(), std::move(buffer));
}// cuda::createNodeManager (single-space)
}// namespace cuda
template <typename BuildT, typename BufferT = cuda::DeviceBuffer>
+5
View File
@@ -30,6 +30,11 @@ public:
// alignment is always satisfied; 256 is the nominal default advertised here.
static constexpr size_t DEFAULT_ALIGNMENT = 256;
/// @brief Pinned allocations are mapped into the host address space, so
/// containers over this resource remain host-readable (detected by
/// nanovdb::cuda::is_host_accessible_resource).
static constexpr bool HOST_ACCESSIBLE = true;
/// @brief Synchronous allocation of page-locked host memory.
/// @param bytes number of bytes to allocate
/// @param alignment requested alignment (ignored; cudaMallocHost is page-aligned)
+28 -23
View File
@@ -10,6 +10,7 @@
#ifndef NANOVDB_CUDA_TEMPPOOL_H_HAS_BEEN_INCLUDED
#define NANOVDB_CUDA_TEMPPOOL_H_HAS_BEEN_INCLUDED
#include <nanovdb/cuda/Buffer.h>
#include <nanovdb/cuda/DeviceResource.h>
#include <cstddef>
@@ -21,31 +22,33 @@ namespace cuda {
template <class Resource>
class TempPool {
static_assert(is_async_resource<Resource>::value,
"TempPool allocates stream-ordered scratch and requires an AsyncResource");
// The buffer borrows the pool's resource through a ResourceRef rather than
// copying it, preserving the pool's contract that all traffic reaches the
// caller's resource instance (which may be stateful).
using BufferT = Buffer<std::byte, ResourceRef<Resource>>;
public:
/// @brief Default c-tor of an empty memory pool that uses the default
/// instance of @c Resource for all allocations.
TempPool() : mResource(&default_resource<Resource>()), mData(nullptr), mSize(0), mRequestedSize(0), mStream(nullptr) {}
TempPool() : TempPool(default_resource<Resource>()) {}
/// @brief C-tor of an empty memory pool that routes all allocations through
/// the supplied @c Resource instance.
/// @param resource resource instance to allocate from; must outlive this pool.
explicit TempPool(Resource& resource) : mResource(&resource), mData(nullptr), mSize(0), mRequestedSize(0), mStream(nullptr) {}
/// @brief Destructor. Frees the managed memory on the stream of the most
/// recent reallocate(), so the stream-ordered free is ordered after
/// the work that used the memory (rather than on the null stream).
~TempPool() {
mRequestedSize = 0;
mResource->deallocate_async(mData, mSize, Resource::DEFAULT_ALIGNMENT, mStream);
mData = nullptr;
mSize = 0;
explicit TempPool(Resource& resource)
: mResource(&resource)
, mBuffer(cudaStream_t{0}, ResourceRef<Resource>(resource), 0, noInit)
{
}
/// @brief Returns a non-const void pointer to the data managed by this instance.
void* data() {return mData;}
void* data() {return mBuffer.data();}
/// @brief Returns a non-const reference to the actual size of the data managed by this instance.
/// @note Returned by reference because cub's two-pass API takes the storage
/// size as a size_t&, so this cannot forward Buffer::size() by value.
size_t& size() {return mSize;}
/// @brief Returns a non-const reference to the requested size of the data managed by this instance.
@@ -54,25 +57,27 @@ public:
/// @brief Returns the stream that the managed memory was last (re)allocated on,
/// i.e. the stream this pool will free on at destruction.
cudaStream_t stream() const {return mStream;}
cudaStream_t stream() const {return mBuffer.stream();}
/// @brief Re-allocation of the data managed by this instance. Only has affect if the pool in empty or
/// the requested memory is larger than the existing size.
/// @param stream cuda stream used for asynchronous de-allocation and allocation.
/// @note Scratch is discarded, never resized: preserving a prefix of
/// temporary storage would be a wasted copy.
void reallocate(cudaStream_t stream) {
if (!mData || mRequestedSize > mSize) {
mResource->deallocate_async(mData, mSize, Resource::DEFAULT_ALIGNMENT, stream);
mData = mResource->allocate_async(mRequestedSize, Resource::DEFAULT_ALIGNMENT, stream);
mSize = mRequestedSize;
if (mBuffer.empty() || mRequestedSize > mSize) {
mBuffer.destroy(stream);// free the outgrown block on this stream
mBuffer = BufferT(stream, ResourceRef<Resource>(*mResource), mRequestedSize, noInit);
mSize = mBuffer.size();
} else {
mBuffer.set_stream(stream);// retained so the d-tor frees on the most-recently-used stream
}
mStream = stream;// retained so the destructor frees on the most-recently-used stream
}
private:
Resource *mResource;
void *mData;
size_t mSize;
size_t mRequestedSize;
cudaStream_t mStream;
Resource *mResource;// non-owning; must outlive this pool and its buffer
BufferT mBuffer;
size_t mSize{0};
size_t mRequestedSize{0};
};// TempPool<Resource> class
using TempDevicePool = TempPool<DeviceResource>;
+22 -2
View File
@@ -117,8 +117,28 @@ nanovdb_example(NAME "ex_coarsen_nanovdb_cuda" OPENVDB)
nanovdb_example(NAME "ex_mesh_to_grid_cuda" OPENVDB)
if(CUDAToolkit_FOUND)
nanovdb_example(NAME "ex_make_mgpu_nanovdb") # requires cuRAND
target_link_libraries(ex_make_mgpu_nanovdb PRIVATE CUDA::curand)
# ex_make_mgpu_nanovdb uses TF32 WMMA which requires SM_80+ (Ampere or newer).
# Gate on CMAKE_CUDA_ARCHITECTURES (what the build actually targets) rather than
# probing the configure-time machine's GPU: the two can differ (cross-compiling,
# building on one machine to run on another, CI runners without a GPU), and
# probing via "bash -c nvidia-smi ..." isn't portable to Windows either way.
set(_nanovdb_has_sm80 OFF)
foreach(_cuda_arch IN LISTS CMAKE_CUDA_ARCHITECTURES)
string(REGEX MATCH "^[0-9]+" _cuda_arch_number "${_cuda_arch}")
if(_cuda_arch_number GREATER_EQUAL 80)
set(_nanovdb_has_sm80 ON)
endif()
endforeach()
if(_nanovdb_has_sm80)
nanovdb_example(NAME "ex_make_mgpu_nanovdb") # requires cuRAND and SM_80+ (TF32 WMMA)
target_link_libraries(ex_make_mgpu_nanovdb PRIVATE CUDA::curand)
else()
message(STATUS "Skipping ex_make_mgpu_nanovdb: requires CUDA architecture 80 or newer (CMAKE_CUDA_ARCHITECTURES=${CMAKE_CUDA_ARCHITECTURES})")
endif()
unset(_nanovdb_has_sm80)
unset(_cuda_arch_number)
unset(_cuda_arch)
endif()
if(NANOVDB_USE_MAGICAVOXEL)
@@ -3,7 +3,9 @@
#pragma once
#ifndef _USE_MATH_DEFINES
#define _USE_MATH_DEFINES
#endif
#include <cmath>
#include <chrono>
#include <fstream>
@@ -1,7 +1,9 @@
// Copyright Contributors to the OpenVDB Project
// SPDX-License-Identifier: Apache-2.0
#ifndef _USE_MATH_DEFINES
#define _USE_MATH_DEFINES
#endif
#include <cmath>
#include <chrono>
@@ -3,7 +3,9 @@
#if defined(NANOVDB_USE_OPENVDB)
#ifndef _USE_MATH_DEFINES
#define _USE_MATH_DEFINES
#endif
#include <cmath>
#include <chrono>
@@ -31,7 +31,7 @@ extern "C" void launch_kernels(const nanovdb::NodeManager<float>* deviceMgr,
// Simple wrapper that makes sure nanovdb::cuda::createNodeManager is initiated
extern "C" void cudaCreateNodeManager(const nanovdb::NanoGrid<float> *d_grid,
nanovdb::NodeManagerHandle<nanovdb::CudaDeviceBuffer> *handle)
nanovdb::NodeManagerHandle<nanovdb::cuda::DeviceBuffer> *handle)
{
*handle = std::move(nanovdb::cuda::createNodeManager<float>(d_grid));
}
}
@@ -3,7 +3,9 @@
#pragma once
#ifndef _USE_MATH_DEFINES
#define _USE_MATH_DEFINES
#endif
#include <cmath>
#include <chrono>
#include <fstream>
@@ -1,7 +1,9 @@
// Copyright Contributors to the OpenVDB Project
// SPDX-License-Identifier: Apache-2.0
#ifndef _USE_MATH_DEFINES
#define _USE_MATH_DEFINES
#endif
#include <cmath>
#include <chrono>
@@ -3,7 +3,9 @@
#if defined(NANOVDB_USE_OPENVDB)
#ifndef _USE_MATH_DEFINES
#define _USE_MATH_DEFINES
#endif
#include <cmath>
#include <chrono>
@@ -3,7 +3,9 @@
#pragma once
#ifndef _USE_MATH_DEFINES
#define _USE_MATH_DEFINES
#endif
#include <cmath>
#include <chrono>
#include <fstream>
@@ -1,7 +1,9 @@
// Copyright Contributors to the OpenVDB Project
// SPDX-License-Identifier: Apache-2.0
#ifndef _USE_MATH_DEFINES
#define _USE_MATH_DEFINES
#endif
#include <cmath>
#include <chrono>
@@ -3,7 +3,9 @@
#pragma once
#ifndef _USE_MATH_DEFINES
#define _USE_MATH_DEFINES
#endif
#include <cmath>
#include <chrono>
#include <fstream>
@@ -44,11 +46,41 @@ inline __hostdev__ void mortonEncode(uint32_t& code, uint32_t x, uint32_t y)
template<typename RenderFn, typename GridT>
inline float renderImage(bool useCuda, const RenderFn renderOp, int width, int height, float* image, const GridT* grid)
{
#if defined(__CUDACC__)
// Kernel-only timing on the GPU path. cudaEventElapsedTime measures only
// the work between the two record points on the default stream, so it
// excludes host-side launch latency and the cudaDeviceSynchronize wakeup
// that std::chrono would otherwise pick up. Events are created/destroyed
// per call; the ~10us creation cost is dwarfed by the kernel itself.
if (useCuda) {
cudaEvent_t startEv, stopEv;
NANOVDB_CUDA_CHECK_ERROR(cudaEventCreate(&startEv), __FILE__, __LINE__);
NANOVDB_CUDA_CHECK_ERROR(cudaEventCreate(&stopEv), __FILE__, __LINE__);
NANOVDB_CUDA_CHECK_ERROR(cudaEventRecord(startEv, 0), __FILE__, __LINE__);
computeForEach(
true, width * height, 256, __FILE__, __LINE__, [renderOp, image, grid] __hostdev__(int start, int end) {
renderOp(start, end, image, grid);
});
NANOVDB_CUDA_CHECK_ERROR(cudaEventRecord(stopEv, 0), __FILE__, __LINE__);
NANOVDB_CUDA_CHECK_ERROR(cudaEventSynchronize(stopEv), __FILE__, __LINE__);
float ms = 0.f;
NANOVDB_CUDA_CHECK_ERROR(cudaEventElapsedTime(&ms, startEv, stopEv), __FILE__, __LINE__);
NANOVDB_CUDA_CHECK_ERROR(cudaEventDestroy(startEv), __FILE__, __LINE__);
NANOVDB_CUDA_CHECK_ERROR(cudaEventDestroy(stopEv), __FILE__, __LINE__);
return ms;
}
#endif
using ClockT = std::chrono::high_resolution_clock;
auto t0 = ClockT::now();
computeForEach(
useCuda, width * height, 512, __FILE__, __LINE__, [renderOp, image, grid] __hostdev__(int start, int end) {
useCuda, width * height, 256, __FILE__, __LINE__, [renderOp, image, grid] __hostdev__(int start, int end) {
renderOp(start, end, image, grid);
});
computeSync(useCuda, __FILE__, __LINE__);
@@ -33,7 +33,7 @@ int main(int ac, char** av)
throw std::runtime_error("Grid must be a level set");
}
const int numIterations = 50;
const int numIterations = 200;
const int width = 1024;
const int height = 1024;
@@ -1,9 +1,13 @@
// Copyright Contributors to the OpenVDB Project
// SPDX-License-Identifier: Apache-2.0
#ifndef _USE_MATH_DEFINES
#define _USE_MATH_DEFINES
#endif
#include <algorithm>
#include <cmath>
#include <chrono>
#include <vector>
#if defined(NANOVDB_USE_CUDA)
#include <nanovdb/cuda/DeviceBuffer.h>
@@ -18,6 +22,34 @@ using BufferT = nanovdb::HostBuffer;
#include "common.h"
// Number of untimed warmup iterations to run before the timed loop. These
// pay the kernel cold-start, lazy module load and GPU clock ramp-up costs
// that would otherwise contaminate the first measured iteration.
static constexpr int NUM_WARMUP_ITERATIONS = 2;
// Print min / median / mean / max instead of just mean. A single outlier
// (context switch, OS jitter, thermal blip) pulls the mean noticeably but
// leaves the median alone, so the median is the more trustworthy stat.
static void
reportStats(const char *label, const std::vector<float> &samples)
{
if (samples.empty()) return;
std::vector<float> sorted(samples); // copy so we don't reorder the caller's data
std::sort(sorted.begin(), sorted.end());
const float minMs = sorted.front();
const float maxMs = sorted.back();
const float medianMs = sorted[sorted.size() / 2];
float sum = 0;
for (float s : sorted) sum += s;
const float meanMs = sum / float(sorted.size());
std::cout << label
<< " min=" << minMs << " ms"
<< " median=" << medianMs << " ms"
<< " mean=" << meanMs << " ms"
<< " max=" << maxMs << " ms"
<< " (n=" << sorted.size() << ")" << std::endl;
}
void runNanoVDB(nanovdb::GridHandle<BufferT>& handle, int numIterations, int width, int height, BufferT& imageBuffer)
{
using GridT = nanovdb::FloatGrid;
@@ -69,14 +101,16 @@ void runNanoVDB(nanovdb::GridHandle<BufferT>& handle, int numIterations, int wid
};
{
float durationAvg = 0;
for (int i = 0; i < numIterations; ++i) {
float duration = renderImage(false, renderOp, width, height, h_outImage, h_grid);
//std::cout << "Duration(NanoVDB-Host) = " << duration << " ms" << std::endl;
durationAvg += duration;
for (int i = 0; i < NUM_WARMUP_ITERATIONS; ++i) {
(void)renderImage(false, renderOp, width, height, h_outImage, h_grid);
}
durationAvg /= numIterations;
std::cout << "Average Duration(NanoVDB-Host) = " << durationAvg << " ms" << std::endl;
std::vector<float> samples;
samples.reserve(numIterations);
for (int i = 0; i < numIterations; ++i) {
samples.push_back(renderImage(false, renderOp, width, height, h_outImage, h_grid));
}
reportStats("Duration(NanoVDB-Host):", samples);
saveImage("raytrace_level_set-nanovdb-host.pfm", width, height, (float*)imageBuffer.data());
}
@@ -92,14 +126,16 @@ void runNanoVDB(nanovdb::GridHandle<BufferT>& handle, int numIterations, int wid
float* d_outImage = reinterpret_cast<float*>(imageBuffer.deviceData());
{
float durationAvg = 0;
for (int i = 0; i < numIterations; ++i) {
float duration = renderImage(true, renderOp, width, height, d_outImage, d_grid);
//std::cout << "Duration(NanoVDB-Cuda) = " << duration << " ms" << std::endl;
durationAvg += duration;
for (int i = 0; i < NUM_WARMUP_ITERATIONS; ++i) {
(void)renderImage(true, renderOp, width, height, d_outImage, d_grid);
}
durationAvg /= numIterations;
std::cout << "Average Duration(NanoVDB-Cuda) = " << durationAvg << " ms" << std::endl;
std::vector<float> samples;
samples.reserve(numIterations);
for (int i = 0; i < numIterations; ++i) {
samples.push_back(renderImage(true, renderOp, width, height, d_outImage, d_grid));
}
reportStats("Duration(NanoVDB-Cuda):", samples);
imageBuffer.deviceDownload();
saveImage("raytrace_level_set-nanovdb-cuda.pfm", width, height, (float*)imageBuffer.data());
@@ -3,7 +3,9 @@
#if defined(NANOVDB_USE_OPENVDB)
#ifndef _USE_MATH_DEFINES
#define _USE_MATH_DEFINES
#endif
#include <cmath>
#include <chrono>
@@ -17,7 +17,7 @@ int main(int, char**)
{
try {
// returns a GridHandle using CUDA for memory management.
auto handle = nanovdb::io::readGrid<nanovdb::CudaDeviceBuffer>("data/sphere.nvdb");
auto handle = nanovdb::io::readGrid<nanovdb::cuda::DeviceBuffer>("data/sphere.nvdb");
cudaStream_t stream; // Create a CUDA stream to allow for asynchronous copy of pinned CUDA memory.
cudaStreamCreate(&stream);
+141
View File
@@ -0,0 +1,141 @@
// Copyright Contributors to the OpenVDB Project
// SPDX-License-Identifier: Apache-2.0
//
// X-macro list of NanoVDB BuildT types currently exposed to Python.
//
// NanoVDB coding style says "avoid macro functions; use inline and templates
// instead" (docs/codingstyle.txt). This file is a deliberate exception: it
// must emit top-level template instantiations and binding registrations into
// several translation units from one canonical list, which a C++ template
// cannot do. The pattern is contained to this file plus the matching
// per-consumer macro definitions; no runtime behavior is hidden behind a
// macro function.
//
// Consumers define one or more of the kind-specific macros below before
// including this file. Each macro is invoked once per matching type; any
// macro the consumer does not define is treated as a no-op and reset after
// the file is processed. This is the single point that must be edited when
// adding (or removing) a Python-visible BuildT.
//
// Macros:
// NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum)
// Scalar value types with arithmetic semantics — exposed with full
// NodeInfo + setVoxel accessors. Suffix forms Python class names
// (e.g. "Float" -> "FloatGrid"). GridTypeEnum names the
// nanovdb::GridType:: enumerator a grid of this BuildT carries
// (used by the polymorphic handle.grid(n) dispatch).
//
// NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum)
// Vector value types — exposed with a setVoxel accessor but no
// NodeInfo. AccessorName is passed explicitly because the legacy
// Python class names for the original two (Vec3f, RGBA8) are
// inconsistent — kept as-is for backwards compatibility.
//
// NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum)
// The nanovdb::Point build type — exposed with a bare accessor.
// Point grids carry GridType::PointIndex (NOT a GridType::Point;
// that enumerator doesn't exist), so GridTypeEnum is given
// explicitly. Polymorphic dispatch routes PointIndex grids to
// NanoGrid<Point>.
//
// NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum)
// BuildTs whose nanovdb::SetVoxel<T> is unavailable (bool, the
// quantized Fp4/Fp8/Fp16/FpN types, the index types ValueIndex /
// ValueOnIndex, and ValueMask). nanovdb::BuildTraits<T>::is_special
// is true for all of these. Exposed with a bare accessor only — no
// setVoxel, no NodeInfo. The Python accessor's getValue() returns
// the type given by nanovdb::BuildToValueMap<T>::Type — float for
// Fp4 / Fp8 / Fp16 / FpN, uint64 for ValueIndex / ValueOnIndex,
// bool for ValueMask and for the BuildT=bool (Boolean) grid.
//
// NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix)
// Subset that has C++ sampler specializations (used by PyMath
// samplers and PyCreateNanoGrid create*Grid factories).
#ifndef NANOVDB_PY_FOR_EACH_SCALAR_BUILDT
#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum)
#define NANOVDB_PY_LOCAL_DEFINED_SCALAR
#endif
#ifndef NANOVDB_PY_FOR_EACH_VECTOR_BUILDT
#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum)
#define NANOVDB_PY_LOCAL_DEFINED_VECTOR
#endif
#ifndef NANOVDB_PY_FOR_EACH_POINT_BUILDT
#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum)
#define NANOVDB_PY_LOCAL_DEFINED_POINT
#endif
#ifndef NANOVDB_PY_FOR_EACH_READONLY_BUILDT
#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum)
#define NANOVDB_PY_LOCAL_DEFINED_READONLY
#endif
#ifndef NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT
#define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix)
#define NANOVDB_PY_LOCAL_DEFINED_SAMPLEABLE
#endif
NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(float, Float, Float)
NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(double, Double, Double)
NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(int16_t, Int16, Int16)
NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(int32_t, Int32, Int32)
NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(int64_t, Int64, Int64)
NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(uint8_t, UInt8, UInt8)
NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(uint32_t, UInt32, UInt32)
// nanovdb::Half is intentionally not bound. The source declares it as
// `class Half{};` (an empty placeholder for IEEE 754 half-precision, see
// NanoVDB.h around line 180) and the C++ ProbeValue<Half> chain is
// inconsistent — leaf storage carries Half but the ProbeValue specialization
// expects float, causing a type mismatch during instantiation. When the
// upstream Half implementation lands this can be added in a follow-up.
NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec3f, Vec3f,
"Vec3fReadVectorAccessor", Vec3f)
NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec3d, Vec3d,
"Vec3dReadAccessor", Vec3d)
NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec4f, Vec4f,
"Vec4fReadAccessor", Vec4f)
NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec4d, Vec4d,
"Vec4dReadAccessor", Vec4d)
NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec3u8, Vec3u8,
"Vec3u8ReadAccessor", Vec3u8)
NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::Vec3u16, Vec3u16,
"Vec3u16ReadAccessor", Vec3u16)
NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(::nanovdb::math::Rgba8, RGBA8,
"RGBA8ReadAccessor", RGBA8)
NANOVDB_PY_FOR_EACH_POINT_BUILDT(::nanovdb::Point, Point, PointIndex)
NANOVDB_PY_FOR_EACH_READONLY_BUILDT(bool, Boolean, Boolean)
NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::Fp4, Fp4, Fp4)
NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::Fp8, Fp8, Fp8)
NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::Fp16, Fp16, Fp16)
NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::FpN, FpN, FpN)
NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::ValueIndex, Index, Index)
NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::ValueOnIndex, OnIndex, OnIndex)
NANOVDB_PY_FOR_EACH_READONLY_BUILDT(::nanovdb::ValueMask, Mask, Mask)
NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(float, Float)
NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(double, Double)
NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(int32_t, Int32)
NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(::nanovdb::Vec3f, Vec3f)
#ifdef NANOVDB_PY_LOCAL_DEFINED_SCALAR
#undef NANOVDB_PY_LOCAL_DEFINED_SCALAR
#endif
#ifdef NANOVDB_PY_LOCAL_DEFINED_VECTOR
#undef NANOVDB_PY_LOCAL_DEFINED_VECTOR
#endif
#ifdef NANOVDB_PY_LOCAL_DEFINED_POINT
#undef NANOVDB_PY_LOCAL_DEFINED_POINT
#endif
#ifdef NANOVDB_PY_LOCAL_DEFINED_READONLY
#undef NANOVDB_PY_LOCAL_DEFINED_READONLY
#endif
#ifdef NANOVDB_PY_LOCAL_DEFINED_SAMPLEABLE
#undef NANOVDB_PY_LOCAL_DEFINED_SAMPLEABLE
#endif
#undef NANOVDB_PY_FOR_EACH_SCALAR_BUILDT
#undef NANOVDB_PY_FOR_EACH_VECTOR_BUILDT
#undef NANOVDB_PY_FOR_EACH_POINT_BUILDT
#undef NANOVDB_PY_FOR_EACH_READONLY_BUILDT
#undef NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT
+59 -2
View File
@@ -4,8 +4,19 @@ option(NANOVDB_BUILD_PYTHON_UNITTESTS [=[
"Include the NanoVDB Python unit test. Requires a python interpreter]=]
${NANOVDB_BUILD_UNITTESTS})
# Stubs are only consumed from an installed wheel (they ship next to the .so
# in nanovdb/), so default ON only when SKBUILD is driving the build. In an
# in-source dev / OpenVDB-CI build the .pyi is not used and stub generation
# can be skipped — which also dodges the macOS CI's sanitizer-runtime
# DYLD_INSERT_LIBRARIES wrapper, under which Python dlopen'ing the .so
# triggers TSan "Interceptors are not working" and aborts stubgen.
option(NANOVDB_BUILD_PYTHON_STUBS
"Generate .pyi type stubs for the nanovdb Python module (requires nanobind 2.x or newer)."
${SKBUILD})
nanobind_add_module(nanovdb_python NB_STATIC
NanoVDBModule.cc
PyBuildGrid.cc
PyCreateNanoGrid.cc
PyGridChecksum.cc
PyGridHandle.cc
@@ -18,6 +29,8 @@ nanobind_add_module(nanovdb_python NB_STATIC
PyPrimitives.cc
PySampleFromVoxels.cc
PyTools.cc
PyTree.cc
PyVoxelBlockManager.cc
cuda/PyDeviceBuffer.cc
cuda/PyDeviceGridHandle.cu
cuda/PyPointsToGrid.cu
@@ -33,10 +46,47 @@ if(SKBUILD)
set_target_properties(nanovdb_python PROPERTIES INSTALL_RPATH "$ORIGIN/../../openvdb/lib")
install(TARGETS nanovdb_python DESTINATION ${NANOVDB_INSTALL_LIBDIR})
install(FILES __init__.py DESTINATION nanovdb)
# Ship the nanovdb C/C++ headers inside the Python wheel so downstream
# extension authors can compile against the same NanoVDB the wheel was
# built with. nanovdb.get_include() resolves to this directory at runtime.
install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/../
DESTINATION nanovdb/include/nanovdb
FILES_MATCHING
PATTERN "*.h"
PATTERN "*.cuh"
PATTERN "python" EXCLUDE
PATTERN "examples" EXCLUDE
PATTERN "unittest" EXCLUDE
PATTERN "cmd" EXCLUDE
PATTERN "docs" EXCLUDE)
else()
install(TARGETS nanovdb_python DESTINATION ${VDB_PYTHON_INSTALL_DIRECTORY})
endif()
# .pyi type stubs for IDE / type-checker support. Driven by the CMake helper
# that ships with nanobind 2.x; silently skipped on older nanobind where the
# helper does not exist.
if(NANOVDB_BUILD_PYTHON_STUBS AND COMMAND nanobind_add_stub)
set(NANOVDB_STUB_OUTPUT_DIR "${CMAKE_CURRENT_BINARY_DIR}")
nanobind_add_stub(nanovdb_python_stub
MODULE nanovdb
OUTPUT "${NANOVDB_STUB_OUTPUT_DIR}/nanovdb.pyi"
PYTHON_PATH $<TARGET_FILE_DIR:nanovdb_python>
DEPENDS nanovdb_python
MARKER_FILE "${NANOVDB_STUB_OUTPUT_DIR}/py.typed"
INCLUDE_PRIVATE)
if(SKBUILD)
install(FILES
"${NANOVDB_STUB_OUTPUT_DIR}/nanovdb.pyi"
"${NANOVDB_STUB_OUTPUT_DIR}/py.typed"
DESTINATION nanovdb)
endif()
elseif(NANOVDB_BUILD_PYTHON_STUBS)
message(STATUS
"NANOVDB_BUILD_PYTHON_STUBS is ON but nanobind_add_stub is unavailable "
"(nanobind >= 2.0 required). Skipping stub generation.")
endif()
# pytest
if(NANOVDB_BUILD_PYTHON_UNITTESTS)
@@ -49,12 +99,19 @@ if(NANOVDB_BUILD_PYTHON_UNITTESTS)
COMMAND ${Python_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/test/TestNanoVDB.py -v
WORKING_DIRECTORY "${NANOVDB_PYTHON_WORKING_DIR}")
# Smoke-run every example script (they self-skip when optional
# dependencies such as NumPy or OpenVDB are unavailable).
add_test(NAME pytest_nanovdb_examples
COMMAND ${Python_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/test/TestExamples.py -v
WORKING_DIRECTORY "${NANOVDB_PYTHON_WORKING_DIR}")
set_tests_properties(pytest_nanovdb_examples PROPERTIES TIMEOUT 300)
if(WIN32)
set(PYTHONPATH "$ENV{PYTHONPATH};${NANOVDB_PYTHON_WORKING_DIR}")
string(REPLACE "\\;" ";" PYTHONPATH "${PYTHONPATH}")
string(REPLACE ";" "\\;" PYTHONPATH "${PYTHONPATH}")
set_tests_properties(pytest_nanovdb PROPERTIES ENVIRONMENT "PYTHONPATH=${PYTHONPATH}")
set_tests_properties(pytest_nanovdb pytest_nanovdb_examples PROPERTIES ENVIRONMENT "PYTHONPATH=${PYTHONPATH}")
else()
set_tests_properties(pytest_nanovdb PROPERTIES ENVIRONMENT "PYTHONPATH=$ENV{PYTHONPATH}:${NANOVDB_PYTHON_WORKING_DIR}")
set_tests_properties(pytest_nanovdb pytest_nanovdb_examples PROPERTIES ENVIRONMENT "PYTHONPATH=$ENV{PYTHONPATH}:${NANOVDB_PYTHON_WORKING_DIR}")
endif()
endif()
File diff suppressed because it is too large Load Diff
+239
View File
@@ -0,0 +1,239 @@
// Copyright Contributors to the OpenVDB Project
// SPDX-License-Identifier: Apache-2.0
#include "PyBuildGrid.h"
#include <nanobind/stl/array.h>
#include <nanobind/stl/string.h>
#include <nanovdb/NanoVDB.h>
#include <nanovdb/HostBuffer.h>
#include <nanovdb/tools/CreateNanoGrid.h>
#include <nanovdb/tools/GridBuilder.h>
#include <array>
#include <string>
namespace nb = nanobind;
using namespace nb::literals;
using namespace nanovdb;
namespace pynanovdb {
// Bind nanovdb::tools::build::Grid<BuildT> together with its
// ValueAccessor<BuildT> and Tree<BuildT>::WriteAccessor proxies, plus a
// .toNanoVDB() shortcut that bakes the build grid into a host NanoGrid
// via tools::createNanoGrid.
//
// One instantiation per writable BuildT in BuildTypes.def (scalars +
// vectors). All three classes live under the nanovdb.tools.build submodule;
// the Python class names mirror the existing typed-grid naming so
// nanovdb.tools.build.FloatGrid is the mutable counterpart of the read-only
// nanovdb.FloatGrid.
template<typename BuildT>
static void defineBuildGrid(nb::module_& m,
const char* gridName,
const char* valueAccName,
const char* writeAccName)
{
using GridT = tools::build::Grid<BuildT>;
using TreeT = tools::build::Tree<BuildT>;
using AccT = tools::build::ValueAccessor<BuildT>;
using WriteAccT = typename TreeT::WriteAccessor;
using ValueT = typename GridT::ValueType;
// ----- build::Grid<BuildT> -----
nb::class_<GridT>(m, gridName,
"Mutable host-side grid builder for this BuildT. Write voxels via "
"setValue / getAccessor / getWriteAccessor, then call toNanoVDB() "
"to bake a frozen NanoGrid GridHandle.")
.def(nb::init<const ValueT&, const std::string&, GridClass>(),
"background"_a,
"name"_a = std::string(""),
"gridClass"_a = GridClass::Unknown,
"Construct an empty mutable build grid. Voxels read as "
"background until written.")
.def("getValue",
[](const GridT& self, const Coord& ijk) -> ValueT {
return self.getValue(ijk);
},
"ijk"_a,
"Return the value at ijk (background if no leaf covers it).")
.def("setValue",
[](GridT& self, const Coord& ijk, const ValueT& value) {
self.setValue(ijk, value);
},
"ijk"_a, "value"_a,
"Set the voxel value at ijk and mark the voxel active.")
// build::Grid has no top-level isActive(ijk); the read path is via
// ValueAccessor. Spin up a fresh accessor for the single query so
// Python callers don't have to.
.def("isActive",
[](GridT& self, const Coord& ijk) {
AccT acc = self.getAccessor();
return acc.isActive(ijk);
},
"ijk"_a,
"Return True iff ijk is in an active voxel. Equivalent to "
"self.getAccessor().isActive(ijk), but allocates a fresh "
"accessor for each call — for repeated queries use "
"self.getAccessor() and reuse it.")
.def("setValueOn",
[](GridT& self, const Coord& ijk) {
AccT acc = self.getAccessor();
acc.setValueOn(ijk);
},
"ijk"_a,
"Mark ijk active without changing the stored value. Equivalent "
"to self.getAccessor().setValueOn(ijk).")
.def("nodeCount",
[](const GridT& self) -> std::array<size_t, 3> {
return self.nodeCount();
},
"Return a 3-tuple (leaf_count, lower_count, upper_count).")
.def("gridType", &GridT::gridType,
"Return the GridType enumerator this BuildT carries.")
.def("gridClass", &GridT::gridClass,
"Return the GridClass assigned at construction time.")
.def("getName", &GridT::getName,
"Return the grid name (as passed at construction).")
.def("setName", &GridT::setName, "name"_a,
"Replace the grid name.")
.def("setTransform", &GridT::setTransform,
"scale"_a = 1.0,
"translation"_a = Vec3d(0.0),
"Set an affine index-to-world map from a uniform scale and "
"translation. Replaces any prior transform.")
.def_prop_ro("background",
[](const GridT& self) -> ValueT {
return self.mRoot.background();
},
"The background value supplied at construction.")
.def("getAccessor", &GridT::getAccessor,
nb::keep_alive<0, 1>(),
"Return a ValueAccessor wired to this grid's root. Thread-safe "
"for reads, NOT thread-safe for writes. The accessor borrows "
"from this grid — the grid must outlive it.")
// WriteAccessor's defaulted move constructor would leave its
// internal ValueAccessor's `mRoot&` reference dangling (it points
// into the moved-from WriteAccessor's own mRoot field). Bypass
// the move path entirely by heap-allocating and handing nanobind
// ownership — the C++ object stays at a stable address for its
// whole lifetime.
.def("getWriteAccessor",
[](GridT& self) -> WriteAccT* {
return new WriteAccT(self.mRoot, self.mMutex);
},
nb::rv_policy::take_ownership,
nb::keep_alive<0, 1>(),
"Return a WriteAccessor for thread-safe writes; the accessor "
"buffers changes into a private root and merges them into the "
"parent grid on destruction (or on an explicit merge() call). "
"Held by nanobind on the heap so the accessor's internal "
"references stay valid.")
// Baking a large grid is the most expensive operation on this
// class; release the GIL so other Python threads can run during
// the conversion (the lambda only touches C++ state).
.def("toNanoVDB",
[](const GridT& self,
tools::StatsMode sMode,
CheckMode cMode,
int verbose) {
return tools::createNanoGrid<GridT, BuildT, HostBuffer>(
self, sMode, cMode, verbose);
},
nb::call_guard<nb::gil_scoped_release>(),
"sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default,
"verbose"_a = 0,
"Bake this mutable grid into a host NanoGrid<BuildT> and "
"return its GridHandle. The build grid is unchanged. "
"Releases the GIL during conversion.");
// ----- build::ValueAccessor<BuildT> -----
//
// Move-only (copy is deleted) — returned by getAccessor(). Caches the
// last leaf / lower / upper node it touched, so repeated access to
// neighboring coordinates is fast.
nb::class_<AccT>(m, valueAccName,
"Move-only read/write accessor returned by build.Grid.getAccessor(). "
"Caches the most recently visited tree path for fast neighbor access.")
.def("getValue",
[](const AccT& self, const Coord& ijk) -> ValueT {
return self.getValue(ijk);
},
"ijk"_a, "Return the value at ijk (uses the cache).")
.def("setValue",
[](AccT& self, const Coord& ijk, const ValueT& value) {
self.setValue(ijk, value);
},
"ijk"_a, "value"_a,
"Set the value at ijk and mark it active (uses the cache).")
.def("setValueOn",
[](AccT& self, const Coord& ijk) { self.setValueOn(ijk); },
"ijk"_a,
"Mark ijk active without changing the stored value.")
.def("isActive",
[](const AccT& self, const Coord& ijk) {
return self.isActive(ijk);
},
"ijk"_a,
"Return True iff ijk is in an active voxel.")
.def("isValueOn",
[](const AccT& self, const Coord& ijk) {
return self.isValueOn(ijk);
},
"ijk"_a,
"Alias for isActive(ijk).");
// ----- build::Tree<BuildT>::WriteAccessor -----
//
// Move-only. Holds its own root node + a reference to the parent root's
// mutex; on destruction (or explicit merge()) it locks the mutex and
// splices its buffered nodes into the parent. Designed for multi-thread
// writes — one WriteAccessor per thread, no shared mutable state.
nb::class_<WriteAccT>(m, writeAccName,
"Thread-safe write accessor returned by build.Grid.getWriteAccessor(). "
"Buffers writes into a private root and merges them into the parent "
"grid on destruction or explicit merge().")
.def("setValue",
[](WriteAccT& self, const Coord& ijk, const ValueT& value) {
self.setValue(ijk, value);
},
"ijk"_a, "value"_a,
"Buffer a set into this accessor's private root.")
.def("setValueOn",
[](WriteAccT& self, const Coord& ijk) { self.setValueOn(ijk); },
"ijk"_a,
"Buffer an active-state set into this accessor's private root.")
.def("merge", &WriteAccT::merge,
"Lock the parent mutex and splice this accessor's buffered "
"nodes into the parent grid. Called automatically when the "
"accessor is destroyed; calling it explicitly is only "
"necessary if you want the changes visible to the parent "
"before the accessor goes out of scope.");
}
void defineBuildGridModule(nb::module_& toolsModule)
{
nb::module_ buildModule = toolsModule.def_submodule("build");
buildModule.doc() =
"Mutable, voxel-by-voxel CPU grid builder mirroring "
"nanovdb::tools::build::*. Construct a typed Grid (e.g. "
"FloatGrid(0.0, 'mygrid')), populate it with setValue / "
"ValueAccessor / WriteAccessor, then call .toNanoVDB() to bake "
"a host NanoGrid handle.";
// X-macro instantiation over every writable BuildT: scalars (full
// arithmetic) and vectors. Read-only special BuildTs (Boolean, Fp*,
// Index, Mask) and Point are deliberately excluded — they have no
// SetValue<T> specialization and can't be built voxel-by-voxel.
#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \
defineBuildGrid<T>(buildModule, #Suffix "Grid", \
#Suffix "ValueAccessor", #Suffix "WriteAccessor");
#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \
defineBuildGrid<T>(buildModule, #Suffix "Grid", \
#Suffix "ValueAccessor", #Suffix "WriteAccessor");
#include "BuildTypes.def"
}
} // namespace pynanovdb
+20
View File
@@ -0,0 +1,20 @@
// Copyright Contributors to the OpenVDB Project
// SPDX-License-Identifier: Apache-2.0
#ifndef NANOVDB_PYBUILDGRID_HAS_BEEN_INCLUDED
#define NANOVDB_PYBUILDGRID_HAS_BEEN_INCLUDED
#include <nanobind/nanobind.h>
namespace nb = nanobind;
namespace pynanovdb {
/// @brief Register the nanovdb.tools.build submodule and its per-BuildT
/// Grid / ValueAccessor / WriteAccessor classes (one set per writable
/// scalar and vector BuildT in BuildTypes.def). Constructs the submodule
/// as `toolsModule.def_submodule("build")`.
void defineBuildGridModule(nb::module_& toolsModule);
} // namespace pynanovdb
#endif
+550 -6
View File
@@ -11,6 +11,9 @@
#include <nanovdb/tools/CreateNanoGrid.h>
#include <nanovdb/tools/GridBuilder.h>
#include <cstring>
#include <string>
namespace nb = nanobind;
using namespace nb::literals;
using namespace nanovdb;
@@ -35,20 +38,561 @@ GridHandle<HostBuffer> createNanoGridFromFunc(const BuildT&
template<typename BuildT> void defineCreateNanoGrid(nb::module_& m, const char* name)
{
m.def(name, &createNanoGridFromFunc<BuildT>, nb::call_guard<nb::gil_scoped_release>(), "background"_a, "name"_a, "gridClass"_a, "func"_a, "bbox"_a);
m.def(name, &createNanoGridFromFunc<BuildT>, nb::call_guard<nb::gil_scoped_release>(), "background"_a, "name"_a, "gridClass"_a, "func"_a, "bbox"_a,
"Construct a NanoGrid by evaluating func(Coord) over every voxel in bbox. "
"Returns a GridHandle owning the freshly-built grid.");
}
template<typename BufferT> void defineOpenToNanoVDB(nb::module_& m)
{
#ifdef NANOVDB_USE_OPENVDB
m.def("openToNanoVDB", &tools::openToNanoVDB<BufferT>, "base"_a, "sMode"_a = tools::StatsMode::Default, "cMode"_a = CheckMode::Default, "verbose"_a = 0);
m.def("openToNanoVDB", &tools::openToNanoVDB<BufferT>,
"base"_a,
"sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default,
"verbose"_a = 0,
"Convert an OpenVDB base grid to a NanoVDB GridHandle.");
#endif
}
template void defineCreateNanoGrid<float>(nb::module_&, const char*);
template void defineCreateNanoGrid<double>(nb::module_&, const char*);
template void defineCreateNanoGrid<int32_t>(nb::module_&, const char*);
template void defineCreateNanoGrid<Vec3f>(nb::module_&, const char*);
// ============================================================================
// Conversion bindings: AbsDiff/RelDiff oracle classes, and the polymorphic
// createNanoGrid free functions for quantized + index destination BuildTs.
// Each accepts source = NanoGrid<SrcBuildT> OR build::Grid<SrcBuildT>.
// ============================================================================
namespace {
// ----- Quantized (Fp4/Fp8/Fp16) -----
//
// C++ signature: createNanoGrid<SrcGridT, DstBuildT, BufferT>(srcGrid,
// sMode, cMode, ditherOn, verbose, buffer).
//
// Try SrcBuildT against both NanoGrid<SrcBuildT> and build::Grid<SrcBuildT>
// and return an empty nb::object on no match so the caller can fall through
// to the next SrcBuildT.
//
// GIL is held for the isinstance / cast dispatch (which touches the Python
// object's type and reference graph) but released around the underlying
// tools::createNanoGrid traversal — the source data lives in stable C++
// storage whose lifetime is anchored by the Python wrapper passed in via
// py_src, so it's safe to read without holding the GIL.
template<typename SrcBuildT, typename DstBuildT>
nb::object tryQuantizeFpX(nb::handle py_src,
tools::StatsMode sMode,
CheckMode cMode,
bool ditherOn,
int verbose)
{
using NanoSrcT = NanoGrid<SrcBuildT>;
using BuildSrcT = tools::build::Grid<SrcBuildT>;
if (nb::isinstance<NanoSrcT>(py_src)) {
const auto& src = nb::cast<const NanoSrcT&>(py_src);
GridHandle<HostBuffer> handle;
{
nb::gil_scoped_release release;
handle = tools::createNanoGrid<NanoSrcT, DstBuildT, HostBuffer>(
src, sMode, cMode, ditherOn, verbose);
}
return nb::cast(std::move(handle));
}
if (nb::isinstance<BuildSrcT>(py_src)) {
const auto& src = nb::cast<const BuildSrcT&>(py_src);
GridHandle<HostBuffer> handle;
{
nb::gil_scoped_release release;
handle = tools::createNanoGrid<BuildSrcT, DstBuildT, HostBuffer>(
src, sMode, cMode, ditherOn, verbose);
}
return nb::cast(std::move(handle));
}
// Invalid (not None) object signals "SrcBuildT didn't match"; the caller
// tests is_valid() and falls through to the next SrcBuildT.
return nb::object();
}
template<typename DstBuildT>
nb::object createNanoGridFpX(nb::handle py_src,
tools::StatsMode sMode,
CheckMode cMode,
bool ditherOn,
int verbose,
const char* pyFnName)
{
// The C++ Fp{4,8,16,N} preProcess static_asserts SrcValueT == float;
// double sources hit a compile-time error, so we accept float only.
if (auto r = tryQuantizeFpX<float, DstBuildT>(
py_src, sMode, cMode, ditherOn, verbose); r.is_valid()) return r;
std::string msg(pyFnName);
msg += ": source must be a FloatGrid or nanovdb.tools.build.FloatGrid "
"(Fp4/Fp8/Fp16/FpN require a float source value type).";
throw nb::type_error(msg.c_str());
}
// ----- FpN (variable bit-width) -----
//
// C++ signature: createNanoGrid<SrcGridT, FpN, OracleT, BufferT>(srcGrid,
// sMode, cMode, ditherOn, verbose, oracle, buffer). OracleT is AbsDiff or
// RelDiff; the binding exposes both as separate Python overloads.
template<typename SrcBuildT, typename OracleT>
nb::object tryQuantizeFpN(nb::handle py_src,
tools::StatsMode sMode,
CheckMode cMode,
bool ditherOn,
int verbose,
const OracleT& oracle)
{
using NanoSrcT = NanoGrid<SrcBuildT>;
using BuildSrcT = tools::build::Grid<SrcBuildT>;
// Same GIL pattern as tryQuantizeFpX: hold the GIL through the
// isinstance / cast dispatch, release it for the conversion.
if (nb::isinstance<NanoSrcT>(py_src)) {
const auto& src = nb::cast<const NanoSrcT&>(py_src);
GridHandle<HostBuffer> handle;
{
nb::gil_scoped_release release;
handle = tools::createNanoGrid<NanoSrcT, FpN, OracleT, HostBuffer>(
src, sMode, cMode, ditherOn, verbose, oracle);
}
return nb::cast(std::move(handle));
}
if (nb::isinstance<BuildSrcT>(py_src)) {
const auto& src = nb::cast<const BuildSrcT&>(py_src);
GridHandle<HostBuffer> handle;
{
nb::gil_scoped_release release;
handle = tools::createNanoGrid<BuildSrcT, FpN, OracleT, HostBuffer>(
src, sMode, cMode, ditherOn, verbose, oracle);
}
return nb::cast(std::move(handle));
}
// Invalid (not None) object signals "SrcBuildT didn't match"; the caller
// tests is_valid() and falls through to the next SrcBuildT.
return nb::object();
}
template<typename OracleT>
nb::object createNanoGridFpNImpl(nb::handle py_src,
const OracleT& oracle,
tools::StatsMode sMode,
CheckMode cMode,
bool ditherOn,
int verbose)
{
if (auto r = tryQuantizeFpN<float, OracleT>(
py_src, sMode, cMode, ditherOn, verbose, oracle); r.is_valid()) return r;
throw nb::type_error(
"createNanoGridFpN: source must be a FloatGrid or "
"nanovdb.tools.build.FloatGrid (FpN requires a float source value type).");
}
// ----- Index / OnIndex -----
//
// C++ signature: createNanoGrid<SrcGridT, DstBuildT, BufferT>(srcGrid,
// channels, includeStats, includeTiles, verbose, buffer). DstBuildT is
// ValueIndex or ValueOnIndex; the binding exposes both as separate
// named functions. Source set is wider than the quantized variants —
// any arithmetic or vector source can be re-cast as an index grid.
template<typename SrcBuildT, typename DstBuildT>
nb::object tryIndexify(nb::handle py_src,
uint32_t channels,
bool includeStats,
bool includeTiles,
int verbose)
{
using NanoSrcT = NanoGrid<SrcBuildT>;
using BuildSrcT = tools::build::Grid<SrcBuildT>;
// Same GIL pattern as tryQuantizeFpX.
if (nb::isinstance<NanoSrcT>(py_src)) {
const auto& src = nb::cast<const NanoSrcT&>(py_src);
GridHandle<HostBuffer> handle;
{
nb::gil_scoped_release release;
handle = tools::createNanoGrid<NanoSrcT, DstBuildT, HostBuffer>(
src, channels, includeStats, includeTiles, verbose);
}
return nb::cast(std::move(handle));
}
if (nb::isinstance<BuildSrcT>(py_src)) {
const auto& src = nb::cast<const BuildSrcT&>(py_src);
GridHandle<HostBuffer> handle;
{
nb::gil_scoped_release release;
handle = tools::createNanoGrid<BuildSrcT, DstBuildT, HostBuffer>(
src, channels, includeStats, includeTiles, verbose);
}
return nb::cast(std::move(handle));
}
// Invalid (not None) object signals "SrcBuildT didn't match"; the caller
// tests is_valid() and falls through to the next SrcBuildT.
return nb::object();
}
template<typename DstBuildT>
nb::object createIndexImpl(nb::handle py_src,
uint32_t channels,
bool includeStats,
bool includeTiles,
int verbose,
const char* pyFnName)
{
if (auto r = tryIndexify<float, DstBuildT>(py_src, channels, includeStats, includeTiles, verbose); r.is_valid()) return r;
if (auto r = tryIndexify<double, DstBuildT>(py_src, channels, includeStats, includeTiles, verbose); r.is_valid()) return r;
if (auto r = tryIndexify<int32_t, DstBuildT>(py_src, channels, includeStats, includeTiles, verbose); r.is_valid()) return r;
if (auto r = tryIndexify<Vec3f, DstBuildT>(py_src, channels, includeStats, includeTiles, verbose); r.is_valid()) return r;
std::string msg(pyFnName);
msg += ": source must be a FloatGrid, DoubleGrid, Int32Grid, "
"Vec3fGrid, or the matching nanovdb.tools.build.* mutable grid.";
throw nb::type_error(msg.c_str());
}
// ----- tools.CreateNanoGrid: converter class with blind-data authoring -----
//
// Mirrors nanovdb::tools::CreateNanoGrid<SrcGridT>. The C++ class is
// templated on the source grid type, so this binding stores the source
// Python object plus the recorded settings and addBlindData() calls, then
// dispatches over the supported SrcBuildTs when getHandle() is called —
// constructing the C++ converter, replaying the recorded state, and baking
// the handle. The destination BuildT is the source BuildT (the C++
// default); quantized and index destinations remain on the
// createNanoGridFp* / createNanoGridIndex free functions above. Authored
// channels come back zeroed — fill them through the writable NumPy view
// returned by grid.getBlindData(n).
// Byte size of one element of the given blind-data GridType, or 0 when the
// size cannot be derived (the caller must then pass it explicitly). Matches
// the per-type size table enforced by GridBlindMetaData::isValid().
uint32_t blindDataTypeSize(GridType dataType)
{
switch (dataType) {
case GridType::Float: return 4u;
case GridType::Double: return 8u;
case GridType::Int16: return 2u;
case GridType::Int32: return 4u;
case GridType::Int64: return 8u;
case GridType::UInt8: return 1u;
case GridType::UInt32: return 4u;
case GridType::Half: return 2u;
case GridType::RGBA8: return 4u;
case GridType::Fp8: return 1u;
case GridType::Fp16: return 2u;
case GridType::Vec3f: return 12u;
case GridType::Vec3d: return 24u;
case GridType::Vec4f: return 16u;
case GridType::Vec4d: return 32u;
case GridType::Vec3u8: return 3u;
case GridType::Vec3u16: return 6u;
default: return 0u;
}
}
class PyCreateNanoGrid
{
struct BlindDataSpec
{
std::string name;
GridBlindDataSemantic semantic;
GridBlindDataClass dataClass;
GridType dataType;
uint64_t count;
uint32_t size;
};
public:
explicit PyCreateNanoGrid(nb::object src)
: mSrc(std::move(src))
{
if (!(matches<float>() || matches<double>() ||
matches<int32_t>() || matches<Vec3f>())) {
throw nb::type_error(
"CreateNanoGrid: source must be a FloatGrid, DoubleGrid, "
"Int32Grid, Vec3fGrid, or the matching "
"nanovdb.tools.build.* mutable grid.");
}
}
// Validates eagerly (the C++ ctor only NANOVDB_ASSERTs, which release
// builds skip) so mistakes surface here rather than as an invalid grid.
uint64_t addBlindData(const std::string& name,
uint64_t count,
GridType dataType,
GridBlindDataSemantic semantic,
GridBlindDataClass dataClass,
uint32_t size)
{
if (name.size() >= GridBlindMetaData::MaxNameSize) {
throw nb::value_error(
"addBlindData: name exceeds the 255 character limit.");
}
if (size == 0u) size = blindDataTypeSize(dataType);
if (size == 0u) {
throw nb::value_error(
"addBlindData: the element size cannot be derived from this "
"dataType — pass size explicitly.");
}
const GridBlindMetaData meta(0, count, size, semantic, dataClass, dataType);
if (!meta.isValid()) {
throw nb::value_error(
"addBlindData: invalid combination of dataSemantic, "
"dataClass, dataType, and size.");
}
mBlind.push_back(BlindDataSpec{name, semantic, dataClass, dataType, count, size});
return static_cast<uint64_t>(mBlind.size() - 1);
}
void setStats(tools::StatsMode mode) { mStats = mode; }
void setChecksum(CheckMode mode) { mChecksum = mode; }
void setVerbose(int mode) { mVerbose = mode; }
void enableDithering(bool on) { mDither = on; }
nb::object getHandle() const
{
if (auto r = tryGetHandle<float>(); r.is_valid()) return r;
if (auto r = tryGetHandle<double>(); r.is_valid()) return r;
if (auto r = tryGetHandle<int32_t>(); r.is_valid()) return r;
if (auto r = tryGetHandle<Vec3f>(); r.is_valid()) return r;
throw nb::type_error("CreateNanoGrid: unsupported source grid type.");
}
private:
template<typename SrcBuildT> bool matches() const
{
return nb::isinstance<NanoGrid<SrcBuildT>>(mSrc) ||
nb::isinstance<tools::build::Grid<SrcBuildT>>(mSrc);
}
template<typename SrcBuildT> nb::object tryGetHandle() const
{
using NanoSrcT = NanoGrid<SrcBuildT>;
using BuildSrcT = tools::build::Grid<SrcBuildT>;
if (nb::isinstance<NanoSrcT>(mSrc)) return this->bake(nb::cast<const NanoSrcT&>(mSrc));
if (nb::isinstance<BuildSrcT>(mSrc)) return this->bake(nb::cast<const BuildSrcT&>(mSrc));
// Invalid (not None) object signals "SrcBuildT didn't match"; the caller
// tests is_valid() and falls through to the next SrcBuildT.
return nb::object();
}
// Same GIL pattern as tryQuantizeFpX: the dispatch above runs with the
// GIL held, the traversal runs without it (the source's lifetime is
// anchored by mSrc).
template<typename SrcGridT> nb::object bake(const SrcGridT& src) const
{
GridHandle<HostBuffer> handle;
{
nb::gil_scoped_release release;
tools::CreateNanoGrid<SrcGridT> converter(src);
converter.setStats(mStats);
converter.setChecksum(mChecksum);
converter.setVerbose(mVerbose);
converter.enableDithering(mDither);
for (const auto& b : mBlind) {
converter.addBlindData(b.name, b.semantic, b.dataClass, b.dataType,
static_cast<size_t>(b.count),
static_cast<size_t>(b.size));
}
handle = converter.getHandle();
// The C++ converter allocates authored channels without clearing
// them (C++ callers memcpy their payload in). Zero-fill here so
// the NumPy view starts deterministic. The authored channels are
// the first mBlind.size() blind-data entries — any converter-
// added channel (e.g. a long grid name) is appended after them.
if (!mBlind.empty()) {
if (auto* dst = const_cast<GridData*>(handle.gridData())) {
for (size_t i = 0; i < mBlind.size(); ++i) {
const GridBlindMetaData* meta = dst->blindMetaData(uint32_t(i));
std::memset(const_cast<void*>(meta->blindData()), 0,
meta->blindDataSize());
}
}
}
}
return nb::cast(std::move(handle));
}
nb::object mSrc;
std::vector<BlindDataSpec> mBlind;
tools::StatsMode mStats = tools::StatsMode::Default;
CheckMode mChecksum = CheckMode::Default;
int mVerbose = 0;
bool mDither = false;
};
} // namespace
void defineCreateNanoGridConversions(nb::module_& toolsModule)
{
// ------ Oracle classes ------
nb::class_<tools::AbsDiff>(toolsModule, "AbsDiff",
"Compression oracle for FpN: accept the approximation when "
"|exact - approx| <= tolerance. A tolerance of -1.0 (the "
"default) means uninitialized; any non-negative value (including "
"0.0) is treated as initialized by the operator bool() check, "
"or the C++ create function can fill it in via init().")
.def(nb::init<float>(), "tolerance"_a = -1.0f,
"Construct an AbsDiff oracle with the given absolute tolerance.")
.def("getTolerance", &tools::AbsDiff::getTolerance,
"Return the current absolute tolerance.")
.def("setTolerance", &tools::AbsDiff::setTolerance, "tolerance"_a,
"Replace the current absolute tolerance.")
.def("__bool__",
[](const tools::AbsDiff& self) { return bool(self); },
"True iff the tolerance has been initialized (>= 0).");
nb::class_<tools::RelDiff>(toolsModule, "RelDiff",
"Compression oracle for FpN: accept the approximation when "
"|exact - approx| / max(|exact|, |approx|) <= tolerance.")
.def(nb::init<float>(), "tolerance"_a = -1.0f,
"Construct a RelDiff oracle with the given relative tolerance.")
.def("getTolerance", &tools::RelDiff::getTolerance,
"Return the current relative tolerance.")
.def("setTolerance", &tools::RelDiff::setTolerance, "tolerance"_a,
"Replace the current relative tolerance.")
.def("__bool__",
[](const tools::RelDiff& self) { return bool(self); },
"True iff the tolerance has been initialized (>= 0).");
// ------ Quantized fixed-width: Fp4 / Fp8 / Fp16 ------
toolsModule.def("createNanoGridFp4",
[](nb::handle src, tools::StatsMode sMode, CheckMode cMode,
bool ditherOn, int verbose) {
return createNanoGridFpX<Fp4>(src, sMode, cMode, ditherOn, verbose,
"createNanoGridFp4");
},
"src"_a, "sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default, "ditherOn"_a = false, "verbose"_a = 0,
"Quantize a NanoGrid<float> or nanovdb.tools.build.FloatGrid into a "
"NanoGrid<Fp4> (4 bits per voxel). ditherOn adds sub-quantum "
"noise to break up banding.");
toolsModule.def("createNanoGridFp8",
[](nb::handle src, tools::StatsMode sMode, CheckMode cMode,
bool ditherOn, int verbose) {
return createNanoGridFpX<Fp8>(src, sMode, cMode, ditherOn, verbose,
"createNanoGridFp8");
},
"src"_a, "sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default, "ditherOn"_a = false, "verbose"_a = 0,
"Quantize a NanoGrid<float> or nanovdb.tools.build.FloatGrid into a NanoGrid<Fp8> (8 bits "
"per voxel). ditherOn adds sub-quantum noise.");
toolsModule.def("createNanoGridFp16",
[](nb::handle src, tools::StatsMode sMode, CheckMode cMode,
bool ditherOn, int verbose) {
return createNanoGridFpX<Fp16>(src, sMode, cMode, ditherOn, verbose,
"createNanoGridFp16");
},
"src"_a, "sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default, "ditherOn"_a = false, "verbose"_a = 0,
"Quantize a NanoGrid<float> or nanovdb.tools.build.FloatGrid into a NanoGrid<Fp16> (16 bits "
"per voxel). ditherOn adds sub-quantum noise.");
// ------ Variable bit-width: FpN with AbsDiff or RelDiff oracle ------
//
// Two overloads — one per oracle type. Python dispatch picks the
// right one from the oracle argument's type. The createNanoGrid C++
// template's parameter order is (src, sMode, cMode, ditherOn, verbose,
// oracle, buffer); we reorder for Python so oracle comes second
// (most callers want to specify it explicitly), then mode/dither
// parameters as kwargs with defaults.
toolsModule.def("createNanoGridFpN",
[](nb::handle src, const tools::AbsDiff& oracle,
tools::StatsMode sMode, CheckMode cMode, bool ditherOn, int verbose) {
return createNanoGridFpNImpl(src, oracle, sMode, cMode, ditherOn, verbose);
},
"src"_a, "oracle"_a = tools::AbsDiff(),
"sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default, "ditherOn"_a = false, "verbose"_a = 0,
"Quantize a NanoGrid<float> or nanovdb.tools.build.FloatGrid into a NanoGrid<FpN> (variable "
"bits per voxel; each leaf picks the smallest N that satisfies "
"the oracle's tolerance). Pass an AbsDiff oracle for absolute "
"error bound, or use the RelDiff overload for relative error.");
toolsModule.def("createNanoGridFpN",
[](nb::handle src, const tools::RelDiff& oracle,
tools::StatsMode sMode, CheckMode cMode, bool ditherOn, int verbose) {
return createNanoGridFpNImpl(src, oracle, sMode, cMode, ditherOn, verbose);
},
"src"_a, "oracle"_a,
"sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default, "ditherOn"_a = false, "verbose"_a = 0,
"FpN overload accepting a RelDiff oracle for relative error.");
// ------ Index / OnIndex ------
//
// createNanoGridIndex / createNanoGridOnIndex are the canonical names
// for the broad-source-coverage index conversion bindings. A narrower
// createOnIndexGrid factory still lives in PyVoxelBlockManager.cc as
// the test scaffolding entry point used by the VBM unit tests.
toolsModule.def("createNanoGridIndex",
[](nb::handle src, uint32_t channels, bool includeStats,
bool includeTiles, int verbose) {
return createIndexImpl<ValueIndex>(
src, channels, includeStats, includeTiles, verbose,
"createNanoGridIndex");
},
"src"_a, "channels"_a = 0u, "includeStats"_a = true,
"includeTiles"_a = true, "verbose"_a = 0,
"Convert a source grid into a NanoGrid<ValueIndex>. Every voxel "
"(active or inactive) gets a unique uint64 sequential index, with "
"the original values stored as blind data when channels > 0.");
toolsModule.def("createNanoGridOnIndex",
[](nb::handle src, uint32_t channels, bool includeStats,
bool includeTiles, int verbose) {
return createIndexImpl<ValueOnIndex>(
src, channels, includeStats, includeTiles, verbose,
"createNanoGridOnIndex");
},
"src"_a, "channels"_a = 0u, "includeStats"_a = true,
"includeTiles"_a = true, "verbose"_a = 0,
"Convert a source grid into a NanoGrid<ValueOnIndex>. Only the "
"active voxels get a sequential index — the canonical input to "
"buildVoxelBlockManager.");
// ------ CreateNanoGrid converter class (blind-data authoring) ------
nb::class_<PyCreateNanoGrid>(toolsModule, "CreateNanoGrid",
"Reusable converter mirroring nanovdb::tools::CreateNanoGrid. "
"Construct from a NanoGrid or nanovdb.tools.build.* grid (float, "
"double, int32, or Vec3f), optionally declare blind-data channels "
"with addBlindData(), then bake a fresh NanoGrid of the same BuildT "
"with getHandle(). Authored channels come back zero-filled — write "
"their contents through the writable NumPy view returned by "
"grid.getBlindData(n). For quantized (Fp*) or index destinations "
"use the createNanoGridFp* / createNanoGridIndex functions instead.")
.def(nb::init<nb::object>(), "srcGrid"_a,
"Construct a converter reading from srcGrid (a NanoGrid or "
"nanovdb.tools.build.* grid of BuildT float, double, int32, or "
"Vec3f). The converter keeps srcGrid alive.")
.def("addBlindData", &PyCreateNanoGrid::addBlindData,
"name"_a, "count"_a, "dataType"_a = GridType::Float,
"dataSemantic"_a = GridBlindDataSemantic::Unknown,
"dataClass"_a = GridBlindDataClass::AttributeArray,
"size"_a = 0u,
"Declare a blind-data channel of count elements of dataType to "
"be allocated in the destination grid, and return its channel "
"index. size (bytes per element) is derived from dataType when "
"omitted; pass it explicitly for dataTypes without a fixed "
"element size. The C++ signature orders the parameters (name, "
"dataSemantic, dataClass, dataType, count, size) — reordered "
"here so the common case reads addBlindData(name, count).")
.def("setStats", &PyCreateNanoGrid::setStats, "mode"_a,
"Set the StatsMode used when baking the destination grid.")
.def("setChecksum", &PyCreateNanoGrid::setChecksum, "mode"_a,
"Set the CheckMode used when baking the destination grid.")
.def("setVerbose", &PyCreateNanoGrid::setVerbose, "mode"_a = 1,
"Set the verbosity level used when baking the destination grid.")
.def("enableDithering", &PyCreateNanoGrid::enableDithering, "on"_a = true,
"Toggle dithering of the destination grid (only meaningful for "
"quantized BuildTs; kept for parity with the C++ class).")
.def("getHandle", &PyCreateNanoGrid::getHandle,
"Bake and return a GridHandle owning a NanoGrid of the source's "
"BuildT, including any channels declared via addBlindData(). "
"Each call bakes a fresh grid.");
}
#define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) \
template void defineCreateNanoGrid<T>(nb::module_&, const char*);
#include "BuildTypes.def"
template void defineOpenToNanoVDB<HostBuffer>(nb::module_&);
+15
View File
@@ -15,6 +15,21 @@ template<typename BuildT> void defineCreateNanoGrid(nb::module_& m, const char*
template<typename BufferT> void defineOpenToNanoVDB(nb::module_& m);
#endif
/// @brief Bind the AbsDiff / RelDiff quantization oracle classes and the
/// polymorphic createNanoGridFp4 / Fp8 / Fp16 / FpN / Index / OnIndex
/// free functions on the nanovdb.tools submodule. Sources accepted
/// include both NanoGrid<SrcBuildT> and tools::build::Grid<SrcBuildT>:
/// the quantized createNanoGridFp* / FpN paths accept float only
/// (C++ Fp{4,8,16,N}::preProcess static-asserts SrcValueT == float);
/// the createNanoGridIndex / OnIndex paths accept float, double,
/// int32_t, and Vec3f sources. Additional source BuildTs can be
/// added by extending the explicit try-each-SrcBuildT chains in
/// createNanoGridFpX / FpNImpl / createIndexImpl. Also binds the
/// tools.CreateNanoGrid converter class, which mirrors the C++
/// tools::CreateNanoGrid and adds blind-data authoring via
/// addBlindData() (same source BuildT set as the index paths).
void defineCreateNanoGridConversions(nb::module_& toolsModule);
} // namespace pynanovdb
#endif
+58 -3
View File
@@ -14,7 +14,10 @@ namespace pynanovdb {
void defineCheckMode(nb::module_& m)
{
nb::enum_<CheckMode>(m, "CheckMode")
nb::enum_<CheckMode>(m, "CheckMode",
"Selector controlling how aggressively a grid checksum is computed: "
"Disable skips checksumming, Partial covers only the header, Full "
"covers the whole grid, and Default picks the recommended mode.")
.value("Disable", CheckMode::Disable)
.value("Partial", CheckMode::Partial)
.value("Full", CheckMode::Full)
@@ -24,13 +27,65 @@ void defineCheckMode(nb::module_& m)
void defineChecksum(nb::module_& m)
{
nb::class_<Checksum>(m, "Checksum").def(nb::self == nb::self, "rhs"_a).def(nb::self != nb::self, "rhs"_a);
nb::class_<Checksum>(m, "Checksum",
"64-bit checksum value stored in a grid header. Produced by "
"tools.evalChecksum and compared against the stored one by "
"tools.validateChecksum.")
.def(nb::self == nb::self, "rhs"_a,
"Equality of two Checksum values.")
.def(nb::self != nb::self, "rhs"_a,
"Inequality of two Checksum values.")
.def("isEmpty", &Checksum::isEmpty,
"True iff no checksum is stored (checksumming was disabled).")
.def("isHalf", &Checksum::isHalf,
"True iff only the header portion (grid + tree + root) is checksummed.")
.def("isFull", &Checksum::isFull,
"True iff both the header portion and all nodes are checksummed.")
.def("mode", &Checksum::mode,
"CheckMode this checksum was computed with (Disable, Partial, or Full).");
}
void defineUpdateChecksum(nb::module_& m)
{
m.def(
"updateChecksum", [](GridData* gridData, CheckMode mode) { tools::updateChecksum(gridData, mode); }, "gridData"_a, "mode"_a);
"updateChecksum", [](GridData* gridData, CheckMode mode) { tools::updateChecksum(gridData, mode); }, "gridData"_a, "mode"_a,
"Recompute and store the checksum of gridData using the given CheckMode.");
}
void defineEvalChecksumModule(nb::module_& toolsModule)
{
// tools.evalChecksum(grid, mode) — compute a fresh checksum for the
// given grid without writing it back. Mirrors the GridData* overload
// in tools/GridChecksum.h; the polymorphism over BuildT is implicit
// because every NanoGrid<T> is-a GridData in the C++ hierarchy (and
// the Python class binding declares NanoGrid<T> as derived from
// GridData).
toolsModule.def("evalChecksum",
[](const GridData* gridData, CheckMode mode) -> Checksum {
if (gridData == nullptr) {
throw nb::value_error("evalChecksum: grid is None.");
}
return tools::evalChecksum(gridData, mode);
},
"grid"_a, "mode"_a = CheckMode::Default,
nb::call_guard<nb::gil_scoped_release>(),
"Compute and return the Checksum for the given grid using the "
"specified CheckMode. Does not modify the grid.");
// tools.validateChecksum(grid, mode) — compare the stored checksum
// against a freshly computed one and return a bool.
toolsModule.def("validateChecksum",
[](const GridData* gridData, CheckMode mode) -> bool {
if (gridData == nullptr) {
throw nb::value_error("validateChecksum: grid is None.");
}
return tools::validateChecksum(gridData, mode);
},
"grid"_a, "mode"_a = CheckMode::Default,
nb::call_guard<nb::gil_scoped_release>(),
"Return True iff the grid's stored checksum matches a freshly "
"computed one for the given CheckMode. A grid with no stored "
"checksum (Checksum.isEmpty()) is considered valid.");
}
} // namespace pynanovdb
+5
View File
@@ -13,6 +13,11 @@ void defineCheckMode(nb::module_& m);
void defineChecksum(nb::module_& m);
void defineUpdateChecksum(nb::module_& m);
/// @brief Bind tools.evalChecksum and tools.validateChecksum on the
/// nanovdb.tools submodule. Both accept any bound NanoGrid via
/// GridData* upcast (handled by nanobind's class hierarchy).
void defineEvalChecksumModule(nb::module_& toolsModule);
} // namespace pynanovdb
#endif
+1
View File
@@ -22,6 +22,7 @@ void defineHostGridHandle(nb::module_& m)
new (&handle) GridHandle<BufferT>(std::move(buffer));
},
"t"_a.noconvert());
defineGridHandleUtilities<BufferT>(m);
}
} // namespace pynanovdb
+163 -36
View File
@@ -7,62 +7,189 @@
#include <nanobind/stl/string.h>
#include <nanovdb/GridHandle.h>
#include <nanovdb/NanoVDB.h>
#include <vector>
namespace nb = nanobind;
namespace pynanovdb {
// Polymorphic host-side `handle.grid(n)`: dispatch on gridType(n) to the
// matching NanoGrid<BuildT> subclass currently bound in Python. Returns
// None when the underlying BuildT is not bound in this build — see the
// list of bound types in BuildTypes.def. The returned object is parented
// to the handle so the handle is kept alive at least as long as the grid.
template<typename BufferT>
inline nb::object pyHostGrid(nb::handle py_handle, uint32_t n)
{
auto& handle = nb::cast<nanovdb::GridHandle<BufferT>&>(py_handle);
if (n >= handle.gridCount()) return nb::none();
switch (handle.gridType(n)) {
#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \
case nanovdb::GridType::GridTypeEnum: { \
auto* grid = handle.template grid<T>(n); \
return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \
: nb::none(); \
}
#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \
case nanovdb::GridType::GridTypeEnum: { \
auto* grid = handle.template grid<T>(n); \
return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \
: nb::none(); \
}
#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \
case nanovdb::GridType::GridTypeEnum: { \
auto* grid = handle.template grid<T>(n); \
return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \
: nb::none(); \
}
#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) \
case nanovdb::GridType::GridTypeEnum: { \
auto* grid = handle.template grid<T>(n); \
return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \
: nb::none(); \
}
#include "BuildTypes.def"
default:
return nb::none();
}
}
// Free functions splitGrids / mergeGrids exposed at module scope. Templated
// on BufferT so the same definitions work for both the host GridHandle and
// the device GridHandle bindings.
template<typename BufferT> void defineGridHandleUtilities(nb::module_& m)
{
using HandleT = nanovdb::GridHandle<BufferT>;
m.def("splitGrids", [](const HandleT& handle) {
auto handles = nanovdb::splitGrids(handle);
nb::list out;
for (auto& h : handles) {
out.append(std::move(h));
}
return out;
}, nb::arg("handle"),
"Split a multi-grid handle into a list of single-grid handles, "
"each owning a freshly-allocated buffer.");
// mergeGrids: collect non-owning pointers to each handle and forward to
// the borrowing nanovdb::mergeGrids overload. The vector-of-handles
// overload is unusable from a Python list because GridHandle is move-only
// — building the vector would move from (and silently empty) the caller's
// Python objects.
m.def("mergeGrids", [](nb::sequence handles) {
std::vector<const HandleT*> sources;
sources.reserve(nb::len(handles));
for (nb::handle item : handles) {
sources.push_back(&nb::cast<const HandleT&>(item));
}
return nanovdb::mergeGrids<BufferT>(sources.data(), sources.size());
}, nb::arg("handles"),
"Combine a list of GridHandles into a single multi-grid GridHandle. "
"Input handles are read by const reference; the new handle owns a "
"freshly-allocated buffer and the inputs are left untouched.");
}
template<typename BufferT> nb::class_<nanovdb::GridHandle<BufferT>> defineGridHandle(nb::module_& m, const char* name)
{
return nb::class_<nanovdb::GridHandle<BufferT>>(m, name)
.def(nb::init<>())
.def("reset", &nanovdb::GridHandle<BufferT>::reset)
.def("size", &nanovdb::GridHandle<BufferT>::bufferSize)
.def("isEmpty", &nanovdb::GridHandle<BufferT>::isEmpty)
.def("empty", &nanovdb::GridHandle<BufferT>::empty)
return nb::class_<nanovdb::GridHandle<BufferT>>(m, name,
"Owns a buffer holding one or more serialized NanoVDB grids. "
"Construct via nanovdb.tools.create* factories or nanovdb.io.readGrid(s); "
"access individual grids via handle.grid(n).")
.def(nb::init<>(),
"Construct an empty handle. Use the nanovdb.tools.create* "
"factories or nanovdb.io.readGrid(s) instead in normal use.")
.def("reset", &nanovdb::GridHandle<BufferT>::reset,
"Drop the underlying buffer; the handle becomes empty.")
.def("size", &nanovdb::GridHandle<BufferT>::bufferSize,
"Total byte size of the buffer backing this handle (sum of "
"every grid plus any internal padding).")
.def("isEmpty", &nanovdb::GridHandle<BufferT>::isEmpty,
"True iff the handle owns no buffer.")
.def("empty", &nanovdb::GridHandle<BufferT>::empty,
"Same as isEmpty(). Retained for parity with the C++ "
"GridHandle::empty() member.")
.def(
"__bool__", [](const nanovdb::GridHandle<BufferT>& handle) { handle.empty(); }, nb::is_operator())
.def("floatGrid", nb::overload_cast<uint32_t>(&nanovdb::GridHandle<BufferT>::template grid<float>), nb::arg("n") = 0, nb::rv_policy::reference_internal)
.def("doubleGrid",
nb::overload_cast<uint32_t>(&nanovdb::GridHandle<BufferT>::template grid<double>),
nb::arg("n") = 0,
nb::rv_policy::reference_internal)
.def("int32Grid",
nb::overload_cast<uint32_t>(&nanovdb::GridHandle<BufferT>::template grid<int32_t>),
nb::arg("n") = 0,
nb::rv_policy::reference_internal)
.def("vec3fGrid",
nb::overload_cast<uint32_t>(&nanovdb::GridHandle<BufferT>::template grid<nanovdb::Vec3f>),
nb::arg("n") = 0,
nb::rv_policy::reference_internal)
.def("rgba8Grid",
nb::overload_cast<uint32_t>(&nanovdb::GridHandle<BufferT>::template grid<nanovdb::math::Rgba8>),
nb::arg("n") = 0,
nb::rv_policy::reference_internal)
.def("isPadded", &nanovdb::GridHandle<BufferT>::isPadded)
.def("gridCount", &nanovdb::GridHandle<BufferT>::gridCount)
.def("gridSize", &nanovdb::GridHandle<BufferT>::gridSize, nb::arg("n") = 0)
.def("gridType", &nanovdb::GridHandle<BufferT>::gridType, nb::arg("n") = 0)
"__bool__",
[](const nanovdb::GridHandle<BufferT>& handle) { return !handle.empty(); },
nb::is_operator(),
"True iff the handle owns a non-empty buffer (`not isEmpty()`).")
.def("copy",
[](const nanovdb::GridHandle<BufferT>& handle) {
return handle.template copy<BufferT>();
},
"Return a deep copy of this GridHandle backed by a freshly-allocated buffer.")
.def("grid", &pyHostGrid<BufferT>, nb::arg("n") = 0,
nb::keep_alive<0, 1>(),
"Return the n-th grid as a typed Grid subclass selected by "
"gridType(n), or None if the BuildT is not bound in Python. "
"The returned grid keeps this handle alive.")
.def("isPadded", &nanovdb::GridHandle<BufferT>::isPadded,
"True iff this handle's buffer is aligned past the natural "
"GridData alignment (used by the I/O code path).")
.def("gridCount", &nanovdb::GridHandle<BufferT>::gridCount,
"Number of grids stored in this handle.")
.def(
"gridSize",
[](const nanovdb::GridHandle<BufferT>& handle, uint32_t n) {
// GridHandle::gridSize(n) indexes mMetaData[n] unchecked, so an
// out-of-range n (including any n on an empty handle) is UB.
if (n >= handle.gridCount())
throw nb::index_error("gridSize: grid index out of range [0, gridCount()).");
return handle.gridSize(n);
},
nb::arg("n") = 0,
"Byte size of the n-th grid (without padding).")
.def(
"gridType",
[](const nanovdb::GridHandle<BufferT>& handle, uint32_t n) {
// GridHandle::gridType(n) indexes mMetaData[n] unchecked, so an
// out-of-range n (including any n on an empty handle) is UB.
if (n >= handle.gridCount())
throw nb::index_error("gridType: grid index out of range [0, gridCount()).");
return handle.gridType(n);
},
nb::arg("n") = 0,
"GridType enumerator of the n-th grid (e.g. GridType.Float). "
"Cheap to query — does not require materializing the grid.")
.def(
"gridData",
[](nanovdb::GridHandle<BufferT>& handle, uint32_t n) { return nb::bytes(handle.gridData(n), handle.gridSize(n)); },
[](nanovdb::GridHandle<BufferT>& handle, uint32_t n) {
// gridData(n) returns nullptr for out-of-range n, but gridSize(n)
// below indexes mMetaData[n] unchecked — guard both here.
if (n >= handle.gridCount())
throw nb::index_error("gridData: grid index out of range [0, gridCount()).");
return nb::bytes(handle.gridData(n), handle.gridSize(n));
},
nb::arg("n") = 0,
nb::rv_policy::reference_internal)
.def("write", nb::overload_cast<const std::string&>(&nanovdb::GridHandle<BufferT>::write, nb::const_), nb::arg("fileName"))
.def("write", nb::overload_cast<const std::string&, uint32_t>(&nanovdb::GridHandle<BufferT>::write, nb::const_), nb::arg("fileName"), nb::arg("n"))
nb::rv_policy::reference_internal,
"Raw byte contents of the n-th grid as a Python bytes object. "
"Useful for hashing or for handing off to non-NanoVDB tooling.")
.def("write", nb::overload_cast<const std::string&>(&nanovdb::GridHandle<BufferT>::write, nb::const_),
nb::arg("fileName"),
"Write every grid in this handle to the given .nvdb file.")
.def("write", nb::overload_cast<const std::string&, uint32_t>(&nanovdb::GridHandle<BufferT>::write, nb::const_),
nb::arg("fileName"), nb::arg("n"),
"Write just the n-th grid in this handle to the given .nvdb file.")
.def(
"read", [](nanovdb::GridHandle<BufferT>& handle, const std::string& fileName) { handle.read(fileName); }, nb::arg("fileName"))
"read", [](nanovdb::GridHandle<BufferT>& handle, const std::string& fileName) { handle.read(fileName); },
nb::arg("fileName"),
"Replace this handle's contents with every grid read from the "
"given .nvdb file.")
.def(
"read",
[](nanovdb::GridHandle<BufferT>& handle, const std::string& fileName, uint32_t n) { handle.read(fileName, n); },
nb::arg("fileName"),
nb::arg("n"))
nb::arg("n"),
"Replace this handle's contents with the n-th grid read from "
"the given .nvdb file.")
.def(
"read",
[](nanovdb::GridHandle<BufferT>& handle, const std::string& fileName, const std::string& gridName) { handle.read(fileName, gridName); },
nb::arg("fileName"),
nb::arg("gridName"));
nb::arg("gridName"),
"Replace this handle's contents with the grid of the given "
"name read from the .nvdb file.");
}
void defineHostGridHandle(nb::module_& m);
+208 -1
View File
@@ -2,16 +2,26 @@
// SPDX-License-Identifier: Apache-2.0
#include "PyGridStats.h"
#include <nanobind/stl/string.h>
#include <nanovdb/NanoVDB.h>
#include <nanovdb/tools/GridStats.h>
#include <string>
namespace nb = nanobind;
using namespace nb::literals;
using namespace nanovdb;
namespace pynanovdb {
void defineStatsMode(nb::module_& m)
{
nb::enum_<tools::StatsMode>(m, "StatsMode")
nb::enum_<tools::StatsMode>(m, "StatsMode",
"Selector controlling which per-node statistics are computed by "
"tools.updateGridStats: Disable skips stats, BBox refreshes only "
"bounding boxes, MinMax adds min/max, and All adds average and "
"standard deviation as well.")
.value("Disable", tools::StatsMode::Disable)
.value("BBox", tools::StatsMode::BBox)
.value("MinMax", tools::StatsMode::MinMax)
@@ -20,4 +30,201 @@ void defineStatsMode(nb::module_& m)
.value("End", tools::StatsMode::End);
}
namespace {
// ----- Extrema<ValueT> binding (rank 0 and rank 1 share the same surface) -----
template<typename BuildT>
static void defineExtrema(nb::module_& m, const char* name)
{
using ValueT = typename NanoGrid<BuildT>::ValueType;
using ExtremaT = tools::Extrema<ValueT>;
nb::class_<ExtremaT>(m, name,
"Running minimum / maximum accumulator over a stream of values. "
"Build via repeated add(v) calls or via tools.getExtrema(grid, bbox).")
.def(nb::init<>(),
"Default-construct an Extrema with min = numeric_limits::max and "
"max = numeric_limits::lowest, so any subsequent .add(v) gives "
"exact min/max.")
.def("min",
[](const ExtremaT& self) -> ValueT { return self.min(); },
"Return the minimum value observed so far.")
.def("max",
[](const ExtremaT& self) -> ValueT { return self.max(); },
"Return the maximum value observed so far.")
.def("add",
[](ExtremaT& self, const ValueT& v) { self.add(v); },
"value"_a,
"Update min/max with a single sample.")
.def("__bool__",
[](const ExtremaT& self) { return bool(self); },
"True iff the Extrema has accumulated at least one sample "
"(i.e. min <= max).")
.def_static("hasMinMax", &ExtremaT::hasMinMax,
"True for value types where min/max is meaningful "
"(everything except bool).")
.def_static("hasAverage", &ExtremaT::hasAverage,
"Always False — Extrema does not compute averages; "
"use Stats for that.")
.def_static("hasStdDeviation", &ExtremaT::hasStdDeviation,
"Always False — Extrema does not compute standard "
"deviation; use Stats for that.")
.def_static("hasStats", &ExtremaT::hasStats,
"True iff the value type supports the min/max bookkeeping "
"(everything except bool).");
}
// ----- Stats<ValueT> binding (inherits Extrema) -----
template<typename BuildT>
static void defineStats(nb::module_& m, const char* name)
{
using ValueT = typename NanoGrid<BuildT>::ValueType;
using BaseT = tools::Extrema<ValueT>;
using StatsT = tools::Stats<ValueT>;
nb::class_<StatsT, BaseT>(m, name,
"Running min/max/mean/variance/std accumulator over a stream of "
"values. Extends Extrema with sample-count-weighted moments.")
.def(nb::init<>(),
"Default-construct a Stats accumulator with zero samples.")
.def("add",
[](StatsT& self, const ValueT& v) { self.add(v); },
"value"_a,
"Add a single sample.")
.def("size",
[](const StatsT& self) -> size_t { return self.size(); },
"Number of samples accumulated so far.")
.def("avg",
[](const StatsT& self) -> double { return self.avg(); },
"Arithmetic mean of all samples.")
.def("mean",
[](const StatsT& self) -> double { return self.mean(); },
"Alias for avg().")
.def("var",
[](const StatsT& self) -> double { return self.var(); },
"Population variance (Sum(x-mean)^2 / N). Returns 0 if "
"fewer than two samples have been added.")
.def("variance",
[](const StatsT& self) -> double { return self.variance(); },
"Alias for var().")
.def("std",
[](const StatsT& self) -> double { return self.std(); },
"Standard deviation = sqrt(var()).")
.def("stdDev",
[](const StatsT& self) -> double { return self.stdDev(); },
"Alias for std().")
.def_static("hasMinMax", &StatsT::hasMinMax,
"True for value types where min/max is meaningful.")
.def_static("hasAverage", &StatsT::hasAverage,
"True for value types that support mean/variance.")
.def_static("hasStdDeviation", &StatsT::hasStdDeviation,
"True for value types that support standard deviation.")
.def_static("hasStats", &StatsT::hasStats,
"True for value types where full statistics is meaningful.");
}
// ----- updateGridStats polymorphic dispatch ----------------------------------
//
// Match the IsNanoGridValid/callNanoGrid pattern: an Op struct with `known<T>`
// (called for every BuildT that's in scope) and `unknown` (fallback). Only
// scalar + vector BuildTs have a meaningful Stats specialization, so the
// other arms raise.
struct UpdateGridStatsOp
{
template<typename BuildT>
static void known(GridData* gridData, tools::StatsMode mode)
{
using GridT = NanoGrid<BuildT>;
using ValueT = typename GridT::ValueType;
if constexpr (BuildTraits<BuildT>::is_special &&
!util::is_same<bool, BuildT>::value) {
// Special / quantized / index / mask BuildTs don't have an
// arithmetic ValueT, so tools::updateGridStats's MinMax / All
// branches would instantiate Stats<ValueT> / Extrema<ValueT>
// with no meaningful semantics (and may not even compile).
// The Disable and BBox branches use NoopStats, which works
// for any ValueT — drive that directly here so the BBox path
// remains available on special grids (it just recomputes
// node bounding boxes without touching min/max/avg).
if (mode == tools::StatsMode::Disable) {
return;
} else if (mode == tools::StatsMode::BBox) {
tools::GridStats<GridT, tools::NoopStats<ValueT>> stats;
stats.update(*static_cast<GridT*>(gridData));
} else {
throw nb::value_error(
"updateGridStats: this grid's BuildT (special / "
"quantized / index / mask) has no arithmetic value "
"type — only StatsMode.Disable and StatsMode.BBox "
"are supported.");
}
} else {
tools::updateGridStats(static_cast<GridT*>(gridData), mode);
}
}
static void unknown(GridData*, tools::StatsMode) {
throw nb::value_error(
"updateGridStats: unsupported GridType / BuildT combination.");
}
};
// ----- getExtrema (per-BuildT factory) ---------------------------------------
template<typename BuildT>
static tools::Extrema<typename NanoGrid<BuildT>::ValueType>
pyGetExtrema(const NanoGrid<BuildT>& grid, const CoordBBox& bbox)
{
return tools::getExtrema<BuildT>(grid, bbox);
}
} // namespace
void defineGridStatsModule(nb::module_& toolsModule)
{
// Per-BuildT Extrema + Stats. One pair per scalar/vector BuildT — the
// value types are all distinct so we get N pairs of new Python classes.
#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \
defineExtrema<T>(toolsModule, #Suffix "Extrema"); \
defineStats<T>(toolsModule, #Suffix "Stats");
#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \
defineExtrema<T>(toolsModule, #Suffix "Extrema"); \
defineStats<T>(toolsModule, #Suffix "Stats");
#include "BuildTypes.def"
// Polymorphic updateGridStats. Accepts any bound NanoGrid<BuildT> (via
// upcast to GridData*) and dispatches on its mGridType.
toolsModule.def("updateGridStats",
[](GridData* gridData, tools::StatsMode mode) {
if (gridData == nullptr) {
throw nb::value_error("updateGridStats: grid is None.");
}
callNanoGrid<UpdateGridStatsOp>(gridData, mode);
},
"grid"_a, "mode"_a = tools::StatsMode::Default,
nb::call_guard<nb::gil_scoped_release>(),
"Recompute and write per-node statistics into the given grid in "
"place. Polymorphic over BuildT. Scalar, vector, and Boolean "
"grids accept every StatsMode (Disable / BBox / MinMax / All); "
"Boolean grids use the C++ NoopStats path internally regardless "
"of mode because there's no arithmetic min/max/avg/dev on bool. "
"Other special (quantized / index / mask) grids accept Disable "
"and BBox (the latter recomputes node bounding boxes only); "
"MinMax and All raise ValueError because their value type has "
"no arithmetic semantics.");
// Per-BuildT getExtrema. We expose one overload per scalar/vector
// BuildT — they each return a Python-side Extrema<ValueT> of the
// matching name.
#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \
toolsModule.def("getExtrema", &pyGetExtrema<T>, \
"grid"_a, "bbox"_a, nb::call_guard<nb::gil_scoped_release>(), \
"Return the Extrema of all values in the grid that intersect " \
"the given bbox.");
#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \
toolsModule.def("getExtrema", &pyGetExtrema<T>, \
"grid"_a, "bbox"_a, nb::call_guard<nb::gil_scoped_release>(), \
"Return the Extrema of all values in the grid that intersect " \
"the given bbox.");
#include "BuildTypes.def"
}
} // namespace pynanovdb
+6
View File
@@ -11,6 +11,12 @@ namespace pynanovdb {
void defineStatsMode(nb::module_& m);
/// @brief Register per-BuildT Extrema and Stats classes (one set per
/// scalar / vector BuildT in BuildTypes.def) and the polymorphic
/// tools.updateGridStats / tools.getExtrema helpers under the
/// nanovdb.tools submodule.
void defineGridStatsModule(nb::module_& toolsModule);
}
#endif
+133 -2
View File
@@ -2,13 +2,21 @@
// SPDX-License-Identifier: Apache-2.0
#include "PyGridValidator.h"
#include <nanobind/operators.h>
#include <nanobind/stl/pair.h>
#include <nanobind/stl/string.h>
#include <nanovdb/GridHandle.h>
#include <nanovdb/HostBuffer.h>
#include <nanovdb/tools/GridValidator.h>
#ifdef NANOVDB_USE_CUDA
#include <nanovdb/cuda/DeviceBuffer.h>
#endif
#include <nanobind/operators.h>
#include <cstring>
#include <iostream>
#include <string>
#include <utility>
namespace nb = nanobind;
using namespace nb::literals;
@@ -18,7 +26,8 @@ namespace pynanovdb {
template<typename BufferT> void defineValidateGrids(nb::module_& m)
{
m.def("validateGrids", &tools::validateGrids<GridHandle<BufferT>>, "handle"_a, "mode"_a, "verbose"_a);
m.def("validateGrids", &tools::validateGrids<GridHandle<BufferT>>,
"handle"_a, "mode"_a, "verbose"_a);
}
template void defineValidateGrids<HostBuffer>(nb::module_&);
@@ -26,4 +35,126 @@ template void defineValidateGrids<HostBuffer>(nb::module_&);
template void defineValidateGrids<cuda::DeviceBuffer>(nb::module_&);
#endif
namespace {
// callNanoGrid op for checkGrid: writes into a fixed-size buffer and returns
// it as a std::pair<bool, std::string> for nb::make_tuple consumption.
//
// tools::checkGrid writes error messages with util::sprint / util::strcpy,
// neither of which is bounded — they trust the caller's buffer is big
// enough. Size the buffer well above what any current error message
// produces (the longest formatted message in GridValidator.h is on the
// order of ~80 characters once both GridType and GridClass enumerator
// names are stringified) and leave generous headroom for future
// additions.
struct CheckGridOp
{
static constexpr size_t kErrorBufSize = 4096;
template<typename BuildT>
static std::pair<bool, std::string> known(const GridData* gridData,
CheckMode mode)
{
char buf[kErrorBufSize];
buf[0] = '\0';
tools::checkGrid<BuildT>(
static_cast<const NanoGrid<BuildT>*>(gridData), buf, mode);
const bool ok = (buf[0] == '\0');
return {ok, std::string(buf)};
}
static std::pair<bool, std::string> unknown(const GridData* gridData,
CheckMode /*mode*/)
{
(void)gridData;
return {false, "Unsupported GridType for checkGrid"};
}
};
// callNanoGrid op for isValid — wraps tools::isValid<BuildT> for every
// switched-over BuildT.
struct IsValidOp
{
template<typename BuildT>
static bool known(const GridData* gridData, CheckMode mode, bool verbose)
{
return tools::isValid<BuildT>(
static_cast<const NanoGrid<BuildT>*>(gridData), mode, verbose);
}
static bool unknown(const GridData* gridData, CheckMode /*mode*/, bool verbose)
{
if (verbose && gridData != nullptr) {
char str[16];
std::cerr << "Validation failed: Unsupported GridType: \""
<< toStr(str, gridData->mGridType) << "\""
<< std::endl;
}
return false;
}
};
} // namespace
void defineGridValidatorModule(nb::module_& toolsModule)
{
// Single-grid validate. Takes a handle, a grid index, and the
// usual mode + verbose flags. Returns true iff the grid passes all
// tests for the given mode. Bound for both host and device handles
// (when CUDA is enabled), matching validateGrids' coverage —
// tools::validateGrid does host-side dispatch via callNanoGrid on
// the host-resident gridData() pointer that DeviceGridHandle also
// exposes, so the same overload pair is appropriate.
toolsModule.def("validateGrid",
&tools::validateGrid<GridHandle<HostBuffer>>,
"handle"_a, "gridID"_a,
"mode"_a = CheckMode::Default, "verbose"_a = false,
nb::call_guard<nb::gil_scoped_release>(),
"Validate the gridID'th grid in the handle against the given "
"CheckMode. Returns False (without raising) if gridID is out "
"of range or the grid fails any check. CheckMode.Disable is a "
"short-circuit that always returns True without inspecting "
"the grid (even when gridID is out of range), matching the "
"C++ behavior. Complements validateGrids() which checks the "
"whole handle.");
#ifdef NANOVDB_USE_CUDA
toolsModule.def("validateGrid",
&tools::validateGrid<GridHandle<cuda::DeviceBuffer>>,
"handle"_a, "gridID"_a,
"mode"_a = CheckMode::Default, "verbose"_a = false,
nb::call_guard<nb::gil_scoped_release>(),
"Validate the gridID'th grid in the device handle (uses the "
"host-resident copy of the grid metadata for the actual "
"checks). Same semantics as the host-handle overload.");
#endif
// Polymorphic checkGrid — returns (ok, error_message). Mirrors the C++
// char-buffer-out signature, but the buffer is hidden inside the
// binding so Python callers get a Python str.
toolsModule.def("checkGrid",
[](const GridData* gridData, CheckMode mode)
-> std::pair<bool, std::string> {
if (gridData == nullptr) {
return {false, "Grid is None"};
}
return callNanoGrid<CheckGridOp>(gridData, mode);
},
"grid"_a, "mode"_a = CheckMode::Full,
nb::call_guard<nb::gil_scoped_release>(),
"Run structural validation checks on the grid for the given "
"CheckMode. Returns a (ok, error_message) tuple — error_message "
"is empty when ok is True.");
// Polymorphic isValid — convenience wrapper. Same as checkGrid + a
// checksum check, returning just the bool.
toolsModule.def("isValid",
[](const GridData* gridData, CheckMode mode, bool verbose) {
if (gridData == nullptr) return false;
return callNanoGrid<IsValidOp>(gridData, mode, verbose);
},
"grid"_a, "mode"_a = CheckMode::Default, "verbose"_a = false,
nb::call_guard<nb::gil_scoped_release>(),
"Return True iff the grid passes structural validation AND its "
"stored checksum matches a freshly computed one for the given "
"CheckMode.");
}
} // namespace pynanovdb
+6
View File
@@ -11,6 +11,12 @@ namespace pynanovdb {
template<typename BufferT> void defineValidateGrids(nb::module_& m);
/// @brief Register tools.validateGrid (single grid in a handle),
/// tools.checkGrid (polymorphic, returns (bool, error_str)) and
/// tools.isValid (polymorphic shortcut) under the nanovdb.tools
/// submodule. Bound once; polymorphic dispatch uses callNanoGrid.
void defineGridValidatorModule(nb::module_& toolsModule);
} // namespace pynanovdb
#endif
+4 -1
View File
@@ -11,7 +11,10 @@ namespace pynanovdb {
void defineHostBuffer(nb::module_& m)
{
nb::class_<HostBuffer>(m, "HostBuffer");
nb::class_<HostBuffer>(m, "HostBuffer",
"Default host-side buffer used to back a GridHandle. Memory is "
"owned by this buffer and freed when the handle (and therefore "
"the buffer) is destroyed.");
}
} // namespace pynanovdb
+80 -40
View File
@@ -21,40 +21,67 @@ namespace {
void defineFileGridMetaData(nb::module_& m)
{
nb::class_<io::FileMetaData>(m, "FileMetaData")
.def_ro("gridSize", &io::FileMetaData::gridSize)
.def_ro("fileSize", &io::FileMetaData::fileSize)
.def_ro("nameKey", &io::FileMetaData::nameKey)
.def_ro("voxelCount", &io::FileMetaData::voxelCount)
.def_ro("gridType", &io::FileMetaData::gridType)
.def_ro("gridClass", &io::FileMetaData::gridClass)
.def_ro("indexBBox", &io::FileMetaData::indexBBox)
.def_ro("worldBBox", &io::FileMetaData::worldBBox)
.def_ro("voxelSize", &io::FileMetaData::voxelSize)
.def_ro("nameSize", &io::FileMetaData::nameSize)
nb::class_<io::FileMetaData>(m, "FileMetaData",
"Per-grid header read from the .nvdb file index. Mirrors the C++ "
"io::FileMetaData layout; subclassed by FileGridMetaData which adds "
"the grid name string.")
.def_ro("gridSize", &io::FileMetaData::gridSize,
"Uncompressed grid size in bytes.")
.def_ro("fileSize", &io::FileMetaData::fileSize,
"On-disk byte size of this grid (post-codec).")
.def_ro("nameKey", &io::FileMetaData::nameKey,
"Hash of the grid name used as a fast lookup key.")
.def_ro("voxelCount", &io::FileMetaData::voxelCount,
"Number of active voxels in this grid.")
.def_ro("gridType", &io::FileMetaData::gridType,
"GridType enumerator naming the BuildT of this grid.")
.def_ro("gridClass", &io::FileMetaData::gridClass,
"GridClass enumerator (LevelSet, FogVolume, ...).")
.def_ro("indexBBox", &io::FileMetaData::indexBBox,
"Axis-aligned bounding box of active voxels in index space.")
.def_ro("worldBBox", &io::FileMetaData::worldBBox,
"Axis-aligned bounding box of active voxels in world space.")
.def_ro("voxelSize", &io::FileMetaData::voxelSize,
"World-space size of a single voxel.")
.def_ro("nameSize", &io::FileMetaData::nameSize,
"Length of the grid name string including the null terminator.")
.def_prop_ro("nodeCount",
[](io::FileMetaData& metaData) {
return std::make_tuple(metaData.nodeCount[0], metaData.nodeCount[1], metaData.nodeCount[2], metaData.nodeCount[3]);
})
},
"Tuple (leaf, lower, upper, root) of node counts in this grid.")
.def_prop_ro("tileCount",
[](io::FileMetaData& metaData) { return std::make_tuple(metaData.tileCount[0], metaData.tileCount[1], metaData.tileCount[2]); })
.def_ro("codec", &io::FileMetaData::codec)
.def_ro("blindDataCount", &io::FileMetaData::blindDataCount)
.def_ro("version", &io::FileMetaData::version);
[](io::FileMetaData& metaData) { return std::make_tuple(metaData.tileCount[0], metaData.tileCount[1], metaData.tileCount[2]); },
"Tuple (lower-tile, upper-tile, root-tile) of active-tile counts.")
.def_ro("codec", &io::FileMetaData::codec,
"Codec used to compress this grid on disk.")
.def_ro("blindDataCount", &io::FileMetaData::blindDataCount,
"Number of blind-data channels attached to this grid.")
.def_ro("version", &io::FileMetaData::version,
"NanoVDB version stored in the file when this grid was written.");
nb::bind_vector<std::vector<io::FileMetaData>>(m, "FileMetaDataVector");
nb::bind_vector<std::vector<io::FileMetaData>>(m, "FileMetaDataVector",
"List of FileMetaData entries, one per grid in a .nvdb file.");
nb::class_<io::FileGridMetaData, io::FileMetaData>(m, "FileGridMetaData")
.def_ro("gridName", &io::FileGridMetaData::gridName)
.def("memUsage", &io::FileGridMetaData::memUsage);
nb::class_<io::FileGridMetaData, io::FileMetaData>(m, "FileGridMetaData",
"FileMetaData extended with the grid name. Returned by "
"readGridMetaData() so callers can identify grids by name without "
"materializing them.")
.def_ro("gridName", &io::FileGridMetaData::gridName,
"Grid name as a Python string.")
.def("memUsage", &io::FileGridMetaData::memUsage,
"Byte size of this metadata record in memory.");
nb::bind_vector<std::vector<io::FileGridMetaData>>(m, "FileGridMetaDataVector");
nb::bind_vector<std::vector<io::FileGridMetaData>>(m, "FileGridMetaDataVector",
"List of FileGridMetaData entries, one per grid in a .nvdb file.");
}
template<typename BufferT> void defineReadWriteGrid(nb::module_& m)
{
m.def("hasGrid", nb::overload_cast<const std::string&, const std::string&>(&io::hasGrid), "fileName"_a, "gridName"_a);
m.def("readGridMetaData", nb::overload_cast<const std::string&>(&io::readGridMetaData), "fileName"_a);
m.def("hasGrid", nb::overload_cast<const std::string&, const std::string&>(&io::hasGrid), "fileName"_a, "gridName"_a,
"Return True iff the .nvdb file at fileName contains a grid named gridName.");
m.def("readGridMetaData", nb::overload_cast<const std::string&>(&io::readGridMetaData), "fileName"_a,
"Return a FileGridMetaDataVector describing every grid stored in the .nvdb file.");
}
template<typename BufferT> nb::list readGrids(const std::string& fileName, int verbose, const BufferT& buffer)
@@ -91,21 +118,26 @@ void defineHostReadWriteGrid(nb::module_& m)
"fileName"_a,
"handle"_a,
"codec"_a = io::Codec::NONE,
"verbose"_a = 0);
m.def("writeGrids", &writeGrids<BufferT>, "fileName"_a, "handles"_a, "codec"_a = io::Codec::NONE, "verbose"_a = 0);
"verbose"_a = 0,
"Write the grids in handle to the .nvdb file at fileName using the given codec.");
m.def("writeGrids", &writeGrids<BufferT>, "fileName"_a, "handles"_a, "codec"_a = io::Codec::NONE, "verbose"_a = 0,
"Write every GridHandle in the handles list to the .nvdb file at fileName.");
m.def("readGrid",
nb::overload_cast<const std::string&, int, int, const BufferT&>(&io::template readGrid<BufferT>),
"fileName"_a,
"n"_a = 0,
"verbose"_a = 0,
"buffer"_a = BufferT());
"buffer"_a = BufferT(),
"Read the n-th grid from the .nvdb file at fileName into a fresh GridHandle.");
m.def("readGrid",
nb::overload_cast<const std::string&, const std::string&, int, const BufferT&>(&io::template readGrid<BufferT>),
"fileName"_a,
"gridName"_a,
"verbose"_a = 0,
"buffer"_a = BufferT());
m.def("readGrids", &readGrids<BufferT>, "fileName"_a, "verbose"_a = 0, "buffer"_a = BufferT());
"buffer"_a = BufferT(),
"Read the grid named gridName from the .nvdb file at fileName into a fresh GridHandle.");
m.def("readGrids", &readGrids<BufferT>, "fileName"_a, "verbose"_a = 0, "buffer"_a = BufferT(),
"Read every grid from the .nvdb file at fileName, returning a list of GridHandles.");
}
#ifdef NANOVDB_USE_CUDA
@@ -119,21 +151,26 @@ void defineDeviceReadWriteGrid(nb::module_& m)
"fileName"_a,
"handle"_a,
"codec"_a = io::Codec::NONE,
"verbose"_a = 0);
m.def("deviceWriteGrids", &writeGrids<BufferT>, "fileName"_a, "handles"_a, "codec"_a = io::Codec::NONE, "verbose"_a = 0);
"verbose"_a = 0,
"Write the grids in a device-backed handle to the .nvdb file at fileName.");
m.def("deviceWriteGrids", &writeGrids<BufferT>, "fileName"_a, "handles"_a, "codec"_a = io::Codec::NONE, "verbose"_a = 0,
"Write every device-backed GridHandle in handles to the .nvdb file at fileName.");
m.def("deviceReadGrid",
nb::overload_cast<const std::string&, int, int, const BufferT&>(&io::template readGrid<BufferT>),
"fileName"_a,
"n"_a = 0,
"verbose"_a = 0,
"buffer"_a = BufferT());
"buffer"_a = BufferT(),
"Read the n-th grid from the .nvdb file at fileName into a fresh DeviceGridHandle.");
m.def("deviceReadGrid",
nb::overload_cast<const std::string&, const std::string&, int, const BufferT&>(&io::template readGrid<BufferT>),
"fileName"_a,
"gridName"_a,
"verbose"_a = 0,
"buffer"_a = BufferT());
m.def("deviceReadGrids", &readGrids<BufferT>, "fileName"_a, "verbose"_a = 0, "buffer"_a = BufferT());
"buffer"_a = BufferT(),
"Read the grid named gridName from the .nvdb file at fileName into a fresh DeviceGridHandle.");
m.def("deviceReadGrids", &readGrids<BufferT>, "fileName"_a, "verbose"_a = 0, "buffer"_a = BufferT(),
"Read every grid from the .nvdb file at fileName into device-backed handles.");
}
#endif
@@ -141,16 +178,19 @@ void defineDeviceReadWriteGrid(nb::module_& m)
void defineIOModule(nb::module_& m)
{
nb::enum_<io::Codec>(m, "Codec")
nb::enum_<io::Codec>(m, "Codec",
"Compression codec selector used when writing a .nvdb file. NONE "
"writes raw bytes; ZIP uses zlib; BLOSC uses the blosc codec when "
"compiled in.")
.value("NONE", io::Codec::NONE)
.value("ZIP", io::Codec::ZIP)
.value("BLOSC", io::Codec::BLOSC)
.export_values();
// .def("__repr__", [](const io::Codec& codec) {
// char str[strlen<io::Codec>()];
// toStr(str, codec);
// return std::string(str);
// });
.export_values()
.def("__repr__", [](const io::Codec& codec) {
char str[strlen<io::Codec>()];
toStr(str, codec);
return std::string(str);
});
defineFileGridMetaData(m);
defineHostReadWriteGrid(m);
+325 -187
View File
@@ -11,7 +11,6 @@
#include <nanobind/stl/bind_vector.h>
#include "PySampleFromVoxels.h"
#include "cuda/PySampleFromVoxels.h"
namespace nb = nanobind;
using namespace nb::literals;
@@ -26,18 +25,27 @@ void defineCoord(nb::module_& m)
using ValueType = math::Coord::ValueType;
nb::class_<math::Coord>(m, "Coord", "Signed (i, j, k) 32-bit integer coordinate class, similar to openvdb::math::Coord")
.def(nb::init<>())
.def(nb::init<ValueType>(), "n"_a)
.def(nb::init<ValueType, ValueType, ValueType>(), "i"_a, "j"_a, "k"_a)
.def(nb::init<>(),
"Construct (0, 0, 0).")
.def(nb::init<ValueType>(), "n"_a,
"Construct (n, n, n).")
.def(nb::init<ValueType, ValueType, ValueType>(), "i"_a, "j"_a, "k"_a,
"Construct (i, j, k).")
.def_prop_rw(
"x", [](const math::Coord& ijk) { return ijk.x(); }, [](math::Coord& ijk, int32_t i) { ijk.x() = i; })
"x", [](const math::Coord& ijk) { return ijk.x(); }, [](math::Coord& ijk, int32_t i) { ijk.x() = i; },
"First component of the (i, j, k) triple.")
.def_prop_rw(
"y", [](const math::Coord& ijk) { return ijk.y(); }, [](math::Coord& ijk, int32_t j) { ijk.y() = j; })
"y", [](const math::Coord& ijk) { return ijk.y(); }, [](math::Coord& ijk, int32_t j) { ijk.y() = j; },
"Second component of the (i, j, k) triple.")
.def_prop_rw(
"z", [](const math::Coord& ijk) { return ijk.z(); }, [](math::Coord& ijk, int32_t k) { ijk.z() = k; })
.def_static("max", &math::Coord::max)
.def_static("min", &math::Coord::min)
.def_static("memUsage", &math::Coord::memUsage)
"z", [](const math::Coord& ijk) { return ijk.z(); }, [](math::Coord& ijk, int32_t k) { ijk.z() = k; },
"Third component of the (i, j, k) triple.")
.def_static("max", &math::Coord::max,
"Largest representable Coord (INT32_MAX in every component).")
.def_static("min", &math::Coord::min,
"Smallest representable Coord (INT32_MIN in every component).")
.def_static("memUsage", &math::Coord::memUsage,
"Byte size of a Coord instance.")
.def(
"__getitem__",
[](const math::Coord& ijk, size_t i) {
@@ -46,7 +54,8 @@ void defineCoord(nb::module_& m)
}
return ijk[static_cast<math::Coord::IndexType>(i)];
},
"i"_a)
"i"_a,
"Read the i-th component (0=x, 1=y, 2=z).")
.def(
"__setitem__",
[](math::Coord& ijk, size_t i, ValueType value) {
@@ -56,41 +65,69 @@ void defineCoord(nb::module_& m)
ijk[static_cast<math::Coord::IndexType>(i)] = value;
},
"i"_a,
"value"_a)
"value"_a,
"Write the i-th component (0=x, 1=y, 2=z).")
.def(
"__and__", [](const math::Coord& a, math::Coord::IndexType b) { return a & b; }, nb::is_operator(), "n"_a)
"__and__", [](const math::Coord& a, math::Coord::IndexType b) { return a & b; }, nb::is_operator(), "n"_a,
"Component-wise bitwise AND with the scalar n.")
.def(
"__lshift__", [](const math::Coord& a, math::Coord::IndexType b) { return a << b; }, nb::is_operator(), "n"_a)
"__lshift__", [](const math::Coord& a, math::Coord::IndexType b) { return a << b; }, nb::is_operator(), "n"_a,
"Component-wise left shift by n bits.")
.def(
"__rshift__", [](const math::Coord& a, math::Coord::IndexType b) { return a >> b; }, nb::is_operator(), "n"_a)
.def(nb::self < nb::self, "rhs"_a)
.def(nb::self == nb::self, "rhs"_a)
.def(nb::self != nb::self, "rhs"_a)
"__rshift__", [](const math::Coord& a, math::Coord::IndexType b) { return a >> b; }, nb::is_operator(), "n"_a,
"Component-wise right shift by n bits.")
.def(nb::self < nb::self, "rhs"_a,
"Lexicographic less-than comparison.")
.def(nb::self == nb::self, "rhs"_a,
"Equality of all three components.")
.def(nb::self != nb::self, "rhs"_a,
"Inequality of any one component.")
.def(
"__iand__", [](math::Coord& a, int b) { return a &= b; }, nb::is_operator(), "n"_a)
"__iand__", [](math::Coord& a, int b) { return a &= b; }, nb::is_operator(), "n"_a,
"In-place component-wise bitwise AND with the scalar n.")
.def(
"__ilshift__", [](math::Coord& a, uint32_t b) { return a <<= b; }, nb::is_operator(), "n"_a)
"__ilshift__", [](math::Coord& a, uint32_t b) { return a <<= b; }, nb::is_operator(), "n"_a,
"In-place component-wise left shift by n bits.")
.def(
"__irshift__", [](math::Coord& a, uint32_t b) { return a >>= b; }, nb::is_operator(), "n"_a)
"__irshift__", [](math::Coord& a, uint32_t b) { return a >>= b; }, nb::is_operator(), "n"_a,
"In-place component-wise right shift by n bits.")
.def(
"__iadd__", [](math::Coord& a, int b) { return a += b; }, nb::is_operator(), "n"_a)
.def(nb::self + nb::self, "rhs"_a)
.def(nb::self - nb::self, "rhs"_a)
.def(-nb::self)
.def(nb::self += nb::self, "rhs"_a)
.def(nb::self -= nb::self, "rhs"_a)
.def("minComponent", &math::Coord::minComponent, "other"_a)
.def("maxComponent", &math::Coord::maxComponent, "other"_a)
.def("offsetBy", nb::overload_cast<ValueType, ValueType, ValueType>(&math::Coord::offsetBy, nb::const_), "dx"_a, "dy"_a, "dz"_a)
.def("offsetBy", nb::overload_cast<ValueType>(&math::Coord::offsetBy, nb::const_), "n"_a)
.def_static("lessThan", &math::Coord::lessThan, "a"_a, "b"_a)
.def_static("Floor", &math::Coord::template Floor<math::Vec3<float>>, "xyz"_a)
.def_static("Floor", &math::Coord::template Floor<math::Vec3<double>>, "xyz"_a)
.def("hash", &math::Coord::template hash<12>)
.def("octant", &math::Coord::octant)
.def("asVec3s", &math::Coord::asVec3s)
.def("asVec3d", &math::Coord::asVec3d)
.def("round", &math::Coord::round)
"__iadd__", [](math::Coord& a, int b) { return a += b; }, nb::is_operator(), "n"_a,
"In-place add the scalar n to every component.")
.def(nb::self + nb::self, "rhs"_a,
"Component-wise addition.")
.def(nb::self - nb::self, "rhs"_a,
"Component-wise subtraction.")
.def(-nb::self,
"Negate every component.")
.def(nb::self += nb::self, "rhs"_a,
"In-place component-wise addition.")
.def(nb::self -= nb::self, "rhs"_a,
"In-place component-wise subtraction.")
.def("minComponent", &math::Coord::minComponent, "other"_a,
"Component-wise minimum with other. See nanovdb::math::Coord::minComponent in NanoVDB.h.")
.def("maxComponent", &math::Coord::maxComponent, "other"_a,
"Component-wise maximum with other. See nanovdb::math::Coord::maxComponent in NanoVDB.h.")
.def("offsetBy", nb::overload_cast<ValueType, ValueType, ValueType>(&math::Coord::offsetBy, nb::const_), "dx"_a, "dy"_a, "dz"_a,
"Return a new Coord offset by (dx, dy, dz). See nanovdb::math::Coord::offsetBy in NanoVDB.h.")
.def("offsetBy", nb::overload_cast<ValueType>(&math::Coord::offsetBy, nb::const_), "n"_a,
"Return a new Coord offset by n in every component.")
.def_static("lessThan", &math::Coord::lessThan, "a"_a, "b"_a,
"Component-wise a < b returning a Coord of 0 / 1 flags.")
.def_static("Floor", &math::Coord::template Floor<math::Vec3<float>>, "xyz"_a,
"Floor each component of a Vec3f to produce an integer Coord.")
.def_static("Floor", &math::Coord::template Floor<math::Vec3<double>>, "xyz"_a,
"Floor each component of a Vec3d to produce an integer Coord.")
.def("hash", &math::Coord::template hash<12>,
"Spatial hash of this coordinate suited for hashed root-table lookups.")
.def("octant", &math::Coord::octant,
"Return the 0..7 octant index of this coordinate's sign bits.")
.def("asVec3s", &math::Coord::asVec3s,
"Convert to a Vec3f (float) with no scaling.")
.def("asVec3d", &math::Coord::asVec3d,
"Convert to a Vec3d (double) with no scaling.")
.def("round", &math::Coord::round,
"Component-wise round; for an integer Coord this is the identity.")
.def("__repr__", [](const math::Coord& ijk) {
std::stringstream ostr;
ostr << ijk;
@@ -101,13 +138,20 @@ void defineCoord(nb::module_& m)
template<typename T> void defineVec3(nb::module_& m, const char* name, const char* doc)
{
nb::class_<math::Vec3<T>>(m, name, doc)
.def(nb::init<>())
.def(nb::init<T>(), "x"_a)
.def(nb::init<T, T, T>(), "x"_a, "y"_a, "z"_a)
.def(nb::init<math::Vec3<T>>(), "v"_a)
.def(nb::init<math::Coord>(), "ijk"_a)
.def(nb::self == nb::self, "rhs"_a)
.def(nb::self != nb::self, "rhs"_a)
.def(nb::init<>(),
"Construct a zero-initialized vector.")
.def(nb::init<T>(), "x"_a,
"Construct (x, x, x).")
.def(nb::init<T, T, T>(), "x"_a, "y"_a, "z"_a,
"Construct (x, y, z).")
.def(nb::init<math::Vec3<T>>(), "v"_a,
"Copy-construct from another Vec3.")
.def(nb::init<math::Coord>(), "ijk"_a,
"Construct from an integer Coord, casting each component.")
.def(nb::self == nb::self, "rhs"_a,
"Component-wise equality.")
.def(nb::self != nb::self, "rhs"_a,
"Component-wise inequality.")
.def(
"__getitem__",
[](const math::Vec3<T>& v, size_t i) {
@@ -116,7 +160,8 @@ template<typename T> void defineVec3(nb::module_& m, const char* name, const cha
}
return v[static_cast<int>(i)];
},
"i"_a)
"i"_a,
"Read the i-th component (0=x, 1=y, 2=z).")
.def(
"__setitem__",
[](math::Vec3<T>& v, size_t i, T value) {
@@ -126,38 +171,68 @@ template<typename T> void defineVec3(nb::module_& m, const char* name, const cha
v[static_cast<int>(i)] = value;
},
"i"_a,
"value"_a)
.def("dot", &math::Vec3<T>::template dot<math::Vec3<T>>, "v"_a)
.def("cross", &math::Vec3<T>::template cross<math::Vec3<T>>, "v"_a)
.def("lengthSqr", &math::Vec3<T>::lengthSqr)
.def("length", &math::Vec3<T>::length)
.def(-nb::self)
.def(nb::self * nb::self, "v"_a)
.def(nb::self / nb::self, "v"_a)
.def(nb::self + nb::self, "v"_a)
.def(nb::self - nb::self, "v"_a)
.def(nb::self + math::Coord(), "ijk"_a)
.def(nb::self - math::Coord(), "ijk"_a)
.def(nb::self * T(), "s"_a)
.def(nb::self / T(), "s"_a)
.def(nb::self += nb::self, "v"_a)
.def(nb::self += math::Coord(), "ijk"_a)
.def(nb::self -= nb::self, "v"_a)
.def(nb::self -= math::Coord(), "ijk"_a)
.def(nb::self *= T(), "s"_a)
.def(nb::self /= T(), "s"_a)
.def("normalize", &math::Vec3<T>::normalize)
.def("minComponent", &math::Vec3<T>::minComponent, "other"_a)
.def("maxComponent", &math::Vec3<T>::maxComponent, "other"_a)
.def("min", &math::Vec3<T>::min)
.def("max", &math::Vec3<T>::max)
.def("floor", &math::Vec3<T>::floor)
.def("ceil", &math::Vec3<T>::ceil)
.def("round", &math::Vec3<T>::round)
"value"_a,
"Write the i-th component (0=x, 1=y, 2=z).")
.def("dot", &math::Vec3<T>::template dot<math::Vec3<T>>, "v"_a,
"Dot product with another vector.")
.def("cross", &math::Vec3<T>::template cross<math::Vec3<T>>, "v"_a,
"Cross product with another vector.")
.def("lengthSqr", &math::Vec3<T>::lengthSqr,
"Squared Euclidean length (cheaper than length()).")
.def("length", &math::Vec3<T>::length,
"Euclidean length of this vector.")
.def(-nb::self,
"Negate every component.")
.def(nb::self * nb::self, "v"_a,
"Component-wise multiplication.")
.def(nb::self / nb::self, "v"_a,
"Component-wise division.")
.def(nb::self + nb::self, "v"_a,
"Component-wise addition.")
.def(nb::self - nb::self, "v"_a,
"Component-wise subtraction.")
.def(nb::self + math::Coord(), "ijk"_a,
"Add an integer Coord component-wise.")
.def(nb::self - math::Coord(), "ijk"_a,
"Subtract an integer Coord component-wise.")
.def(nb::self * T(), "s"_a,
"Multiply every component by the scalar s.")
.def(nb::self / T(), "s"_a,
"Divide every component by the scalar s.")
.def(nb::self += nb::self, "v"_a,
"In-place component-wise addition.")
.def(nb::self += math::Coord(), "ijk"_a,
"In-place add an integer Coord.")
.def(nb::self -= nb::self, "v"_a,
"In-place component-wise subtraction.")
.def(nb::self -= math::Coord(), "ijk"_a,
"In-place subtract an integer Coord.")
.def(nb::self *= T(), "s"_a,
"In-place scalar multiply.")
.def(nb::self /= T(), "s"_a,
"In-place scalar divide.")
.def("normalize", &math::Vec3<T>::normalize,
"Scale this vector to unit length in place.")
.def("minComponent", &math::Vec3<T>::minComponent, "other"_a,
"Component-wise minimum with other.")
.def("maxComponent", &math::Vec3<T>::maxComponent, "other"_a,
"Component-wise maximum with other.")
.def("min", &math::Vec3<T>::min,
"Smallest single component of this vector.")
.def("max", &math::Vec3<T>::max,
"Largest single component of this vector.")
.def("floor", &math::Vec3<T>::floor,
"Component-wise floor.")
.def("ceil", &math::Vec3<T>::ceil,
"Component-wise ceiling.")
.def("round", &math::Vec3<T>::round,
"Component-wise round.")
.def(
"__mul__", [](const T& a, math::Vec3<T> b) { return a * b; }, nb::is_operator(), "b"_a)
"__mul__", [](const T& a, math::Vec3<T> b) { return a * b; }, nb::is_operator(), "b"_a,
"Right-multiply: scalar * Vec3.")
.def(
"__truediv__", [](const T& a, math::Vec3<T> b) { return a / b; }, nb::is_operator(), "b"_a)
"__truediv__", [](const T& a, math::Vec3<T> b) { return a / b; }, nb::is_operator(), "b"_a,
"Right-divide: scalar / Vec3, component-wise.")
.def("__repr__", [](const math::Vec3<T>& v) {
std::stringstream ostr;
ostr << v;
@@ -168,12 +243,18 @@ template<typename T> void defineVec3(nb::module_& m, const char* name, const cha
template<typename T> void defineVec4(nb::module_& m, const char* name, const char* doc)
{
nb::class_<math::Vec4<T>>(m, name, doc)
.def(nb::init<>())
.def(nb::init<T>(), "x"_a)
.def(nb::init<T, T, T, T>(), "x"_a, "y"_a, "z"_a, "w"_a)
.def(nb::init<math::Vec4<T>>(), "v"_a)
.def(nb::self == nb::self, "rhs"_a)
.def(nb::self != nb::self, "rhs"_a)
.def(nb::init<>(),
"Construct a zero-initialized vector.")
.def(nb::init<T>(), "x"_a,
"Construct (x, x, x, x).")
.def(nb::init<T, T, T, T>(), "x"_a, "y"_a, "z"_a, "w"_a,
"Construct (x, y, z, w).")
.def(nb::init<math::Vec4<T>>(), "v"_a,
"Copy-construct from another Vec4.")
.def(nb::self == nb::self, "rhs"_a,
"Component-wise equality.")
.def(nb::self != nb::self, "rhs"_a,
"Component-wise inequality.")
.def(
"__getitem__",
[](const math::Vec4<T>& v, size_t i) {
@@ -182,7 +263,8 @@ template<typename T> void defineVec4(nb::module_& m, const char* name, const cha
}
return v[static_cast<int>(i)];
},
"i"_a)
"i"_a,
"Read the i-th component (0=x, 1=y, 2=z, 3=w).")
.def(
"__setitem__",
[](math::Vec4<T>& v, size_t i, T value) {
@@ -192,28 +274,48 @@ template<typename T> void defineVec4(nb::module_& m, const char* name, const cha
v[static_cast<int>(i)] = value;
},
"i"_a,
"value"_a)
.def("dot", &math::Vec4<T>::template dot<math::Vec4<T>>, "v"_a)
.def("lengthSqr", &math::Vec4<T>::lengthSqr)
.def("length", &math::Vec4<T>::length)
.def(-nb::self)
.def(nb::self * nb::self, "v"_a)
.def(nb::self / nb::self, "v"_a)
.def(nb::self + nb::self, "v"_a)
.def(nb::self - nb::self, "v"_a)
.def(nb::self * T(), "s"_a)
.def(nb::self / T(), "s"_a)
.def(nb::self += nb::self, "v"_a)
.def(nb::self -= nb::self, "v"_a)
.def(nb::self *= T(), "s"_a)
.def(nb::self /= T(), "s"_a)
.def("normalize", &math::Vec4<T>::normalize)
.def("minComponent", &math::Vec4<T>::minComponent, "other"_a)
.def("maxComponent", &math::Vec4<T>::maxComponent, "other"_a)
"value"_a,
"Write the i-th component (0=x, 1=y, 2=z, 3=w).")
.def("dot", &math::Vec4<T>::template dot<math::Vec4<T>>, "v"_a,
"Dot product with another vector.")
.def("lengthSqr", &math::Vec4<T>::lengthSqr,
"Squared Euclidean length (cheaper than length()).")
.def("length", &math::Vec4<T>::length,
"Euclidean length of this vector.")
.def(-nb::self,
"Negate every component.")
.def(nb::self * nb::self, "v"_a,
"Component-wise multiplication.")
.def(nb::self / nb::self, "v"_a,
"Component-wise division.")
.def(nb::self + nb::self, "v"_a,
"Component-wise addition.")
.def(nb::self - nb::self, "v"_a,
"Component-wise subtraction.")
.def(nb::self * T(), "s"_a,
"Multiply every component by the scalar s.")
.def(nb::self / T(), "s"_a,
"Divide every component by the scalar s.")
.def(nb::self += nb::self, "v"_a,
"In-place component-wise addition.")
.def(nb::self -= nb::self, "v"_a,
"In-place component-wise subtraction.")
.def(nb::self *= T(), "s"_a,
"In-place scalar multiply.")
.def(nb::self /= T(), "s"_a,
"In-place scalar divide.")
.def("normalize", &math::Vec4<T>::normalize,
"Scale this vector to unit length in place.")
.def("minComponent", &math::Vec4<T>::minComponent, "other"_a,
"Component-wise minimum with other.")
.def("maxComponent", &math::Vec4<T>::maxComponent, "other"_a,
"Component-wise maximum with other.")
.def(
"__mul__", [](const T& a, math::Vec4<T> b) { return a * b; }, nb::is_operator(), "b"_a)
"__mul__", [](const T& a, math::Vec4<T> b) { return a * b; }, nb::is_operator(), "b"_a,
"Right-multiply: scalar * Vec4.")
.def(
"__truediv__", [](const T& a, math::Vec4<T> b) { return a / b; }, nb::is_operator(), "b"_a)
"__truediv__", [](const T& a, math::Vec4<T> b) { return a / b; }, nb::is_operator(), "b"_a,
"Right-divide: scalar / Vec4, component-wise.")
.def("__repr__", [](const math::Vec4<T>& v) {
std::stringstream ostr;
ostr << v;
@@ -226,18 +328,30 @@ void defineRgba8(nb::module_& m)
using ValueType = math::Rgba8::ValueType;
nb::class_<math::Rgba8>(m, "Rgba8", "8-bit red, green, blue, alpha packed into 32 bit unsigned int")
.def(nb::init<>())
.def(nb::init<const math::Rgba8&>(), "other"_a)
.def(nb::init<uint8_t, uint8_t, uint8_t, uint8_t>(), "r"_a, "g"_a, "b"_a, "a"_a = 255)
.def(nb::init<uint8_t>(), "v"_a)
.def(nb::init<float, float, float, float>(), "r"_a, "g"_a, "b"_a, "a"_a = 1.0)
.def(nb::init<Vec3f>(), "rgb"_a)
.def(nb::init<Vec4f>(), "rgba"_a)
.def(nb::self < nb::self, "rhs"_a)
.def(nb::self == nb::self, "rhs"_a)
.def("lengthSqr", &math::Rgba8::lengthSqr)
.def("length", &math::Rgba8::length)
.def("asFloat", &math::Rgba8::asFloat, "n"_a)
.def(nb::init<>(),
"Construct a fully transparent black Rgba8 (all components 0).")
.def(nb::init<const math::Rgba8&>(), "other"_a,
"Copy-construct from another Rgba8.")
.def(nb::init<uint8_t, uint8_t, uint8_t, uint8_t>(), "r"_a, "g"_a, "b"_a, "a"_a = 255,
"Construct from four 0..255 uint8 channels; a defaults to fully opaque.")
.def(nb::init<uint8_t>(), "v"_a,
"Construct a gray Rgba8 with every channel set to v.")
.def(nb::init<float, float, float, float>(), "r"_a, "g"_a, "b"_a, "a"_a = 1.0,
"Construct from four 0..1 floats, clamped and quantized to uint8.")
.def(nb::init<Vec3f>(), "rgb"_a,
"Construct from an RGB float triple; alpha defaults to opaque.")
.def(nb::init<Vec4f>(), "rgba"_a,
"Construct from an RGBA float quadruple.")
.def(nb::self < nb::self, "rhs"_a,
"Less-than comparison on the packed uint32 representation.")
.def(nb::self == nb::self, "rhs"_a,
"Equality on the packed uint32 representation.")
.def("lengthSqr", &math::Rgba8::lengthSqr,
"Squared length over (r, g, b, a) as integers.")
.def("length", &math::Rgba8::length,
"Euclidean length over (r, g, b, a) as floats.")
.def("asFloat", &math::Rgba8::asFloat, "n"_a,
"Return the n-th channel as a 0..1 float.")
.def(
"__getitem__",
[](const math::Rgba8& rgba, size_t i) {
@@ -246,7 +360,8 @@ void defineRgba8(nb::module_& m)
}
return rgba[static_cast<int>(i)];
},
"i"_a)
"i"_a,
"Read the i-th channel as a uint8 (0=r, 1=g, 2=b, 3=a).")
.def(
"__setitem__",
[](math::Rgba8& rgba, size_t i, ValueType value) {
@@ -256,26 +371,38 @@ void defineRgba8(nb::module_& m)
rgba[static_cast<int>(i)] = value;
},
"i"_a,
"value"_a)
"value"_a,
"Write the i-th channel as a uint8 (0=r, 1=g, 2=b, 3=a).")
.def_prop_rw(
"packed", [](const math::Rgba8& rgba) { return rgba.packed(); }, [](math::Rgba8& rgba, uint32_t packed) { rgba.packed() = packed; })
"packed", [](const math::Rgba8& rgba) { return rgba.packed(); }, [](math::Rgba8& rgba, uint32_t packed) { rgba.packed() = packed; },
"The raw 32-bit packed RGBA representation.")
.def_prop_rw(
"r", [](const math::Rgba8& rgba) { return rgba.r(); }, [](math::Rgba8& rgba, uint8_t r) { rgba.r() = r; })
"r", [](const math::Rgba8& rgba) { return rgba.r(); }, [](math::Rgba8& rgba, uint8_t r) { rgba.r() = r; },
"Red channel as a uint8 0..255.")
.def_prop_rw(
"g", [](const math::Rgba8& rgba) { return rgba.g(); }, [](math::Rgba8& rgba, uint8_t g) { rgba.g() = g; })
"g", [](const math::Rgba8& rgba) { return rgba.g(); }, [](math::Rgba8& rgba, uint8_t g) { rgba.g() = g; },
"Green channel as a uint8 0..255.")
.def_prop_rw(
"b", [](const math::Rgba8& rgba) { return rgba.b(); }, [](math::Rgba8& rgba, uint8_t b) { rgba.b() = b; })
"b", [](const math::Rgba8& rgba) { return rgba.b(); }, [](math::Rgba8& rgba, uint8_t b) { rgba.b() = b; },
"Blue channel as a uint8 0..255.")
.def_prop_rw(
"a", [](const math::Rgba8& rgba) { return rgba.a(); }, [](math::Rgba8& rgba, uint8_t a) { rgba.a() = a; })
.def("asVec3f", [](const math::Rgba8& rgba) { return Vec3f(rgba); })
.def("asVec4f", [](const math::Rgba8& rgba) { return Vec4f(rgba); });
"a", [](const math::Rgba8& rgba) { return rgba.a(); }, [](math::Rgba8& rgba, uint8_t a) { rgba.a() = a; },
"Alpha channel as a uint8 0..255.")
.def("asVec3f", [](const math::Rgba8& rgba) { return Vec3f(rgba); },
"Convert RGB channels to a Vec3f of 0..1 floats (alpha dropped).")
.def("asVec4f", [](const math::Rgba8& rgba) { return Vec4f(rgba); },
"Convert RGBA channels to a Vec4f of 0..1 floats.");
}
template<typename Vec3T> void defineBaseBBox(nb::module_& m, const char* name)
{
nb::class_<math::BaseBBox<Vec3T>>(m, name)
.def(nb::self == nb::self, "rhs"_a)
.def(nb::self != nb::self, "rhs"_a)
nb::class_<math::BaseBBox<Vec3T>>(m, name,
"Axis-aligned bounding-box base class. Stores a min / max corner; "
"concrete subclasses add the open-interval vs closed-interval semantics.")
.def(nb::self == nb::self, "rhs"_a,
"Equality of both min and max corners.")
.def(nb::self != nb::self, "rhs"_a,
"Inequality of either min or max corner.")
.def(
"__getitem__",
[](const math::BaseBBox<Vec3T>& bbox, size_t i) {
@@ -284,7 +411,8 @@ template<typename Vec3T> void defineBaseBBox(nb::module_& m, const char* name)
}
return bbox[static_cast<int>(i)];
},
"i"_a)
"i"_a,
"Read corner 0 (min) or corner 1 (max).")
.def(
"__setitem__",
[](math::BaseBBox<Vec3T>& bbox, size_t i, const Vec3T& value) {
@@ -294,29 +422,45 @@ template<typename Vec3T> void defineBaseBBox(nb::module_& m, const char* name)
bbox[static_cast<int>(i)] = value;
},
"i"_a,
"value"_a)
"value"_a,
"Write corner 0 (min) or corner 1 (max).")
.def_prop_rw(
"min", [](const math::BaseBBox<Vec3T>& bbox) { return bbox.min(); }, [](math::BaseBBox<Vec3T>& bbox, const Vec3T& min) { bbox.min() = min; })
"min", [](const math::BaseBBox<Vec3T>& bbox) { return bbox.min(); }, [](math::BaseBBox<Vec3T>& bbox, const Vec3T& min) { bbox.min() = min; },
"Minimum corner of the bounding box.")
.def_prop_rw(
"max", [](const math::BaseBBox<Vec3T>& bbox) { return bbox.max(); }, [](math::BaseBBox<Vec3T>& bbox, const Vec3T& max) { bbox.max() = max; })
.def("translate", &math::BaseBBox<Vec3T>::translate, "xyz"_a)
.def("expand", nb::overload_cast<const Vec3T&>(&math::BaseBBox<Vec3T>::expand), "xyz"_a)
.def("expand", nb::overload_cast<const math::BaseBBox<Vec3T>&>(&math::BaseBBox<Vec3T>::expand), "bbox"_a)
.def("intersect", &math::BaseBBox<Vec3T>::intersect, "bbox"_a)
.def("isInside", &math::BaseBBox<Vec3T>::isInside, "xyz"_a);
"max", [](const math::BaseBBox<Vec3T>& bbox) { return bbox.max(); }, [](math::BaseBBox<Vec3T>& bbox, const Vec3T& max) { bbox.max() = max; },
"Maximum corner of the bounding box.")
.def("translate", &math::BaseBBox<Vec3T>::translate, "xyz"_a,
"Translate this bounding box by xyz in place.")
.def("expand", nb::overload_cast<const Vec3T&>(&math::BaseBBox<Vec3T>::expand), "xyz"_a,
"Grow this bounding box to include the point xyz.")
.def("expand", nb::overload_cast<const math::BaseBBox<Vec3T>&>(&math::BaseBBox<Vec3T>::expand), "bbox"_a,
"Grow this bounding box to include another bbox in its entirety.")
.def("intersect", &math::BaseBBox<Vec3T>::intersect, "bbox"_a,
"Shrink this bounding box to the intersection with bbox.")
.def("isInside", &math::BaseBBox<Vec3T>::isInside, "xyz"_a,
"True iff xyz lies inside this bounding box.");
}
template<typename Vec3T> void defineBBoxFloatingPoint(nb::module_& m, const char* name, const char* doc)
{
nb::class_<math::BBox<Vec3T, true>, math::BaseBBox<Vec3T>>(m, name, doc)
.def(nb::init<>())
.def(nb::init<const Vec3T&, const Vec3T&>(), "min"_a, "max"_a)
.def(nb::init<const math::Coord&, const math::Coord&>(), "min"_a, "max"_a)
.def_static("createCube", &math::BBox<Vec3T>::createCube, "min"_a, "dim"_a)
.def(nb::init<const math::BaseBBox<math::Coord>&>(), "bbox"_a)
.def("empty", &math::BBox<Vec3T>::empty)
.def("dim", &math::BBox<Vec3T>::dim)
.def("isInside", &math::BBox<Vec3T>::isInside, "p"_a)
.def(nb::init<>(),
"Construct an empty bounding box (min > max sentinel).")
.def(nb::init<const Vec3T&, const Vec3T&>(), "min"_a, "max"_a,
"Construct from explicit min and max corners.")
.def(nb::init<const math::Coord&, const math::Coord&>(), "min"_a, "max"_a,
"Construct from integer Coord corners, cast to floating-point.")
.def_static("createCube", &math::BBox<Vec3T>::createCube, "min"_a, "dim"_a,
"Construct an axis-aligned cube of side dim anchored at min.")
.def(nb::init<const math::BaseBBox<math::Coord>&>(), "bbox"_a,
"Construct from an integer CoordBBox, cast to floating-point.")
.def("empty", &math::BBox<Vec3T>::empty,
"True iff this bbox is empty (any min component > the matching max).")
.def("dim", &math::BBox<Vec3T>::dim,
"Return max - min as a Vec3 of side lengths.")
.def("isInside", &math::BBox<Vec3T>::isInside, "p"_a,
"True iff p lies inside this bounding box (half-open interval).")
.def("__repr__", [](const math::BBox<Vec3T>& b) {
std::stringstream ostr;
ostr << b;
@@ -329,24 +473,39 @@ template<typename CoordT> void defineBBoxInteger(nb::module_& m, const char* nam
using ValueType = typename CoordT::ValueType;
nb::class_<math::BBox<CoordT, false>, math::BaseBBox<CoordT>>(m, name, doc)
.def(nb::init<>())
.def(nb::init<const CoordT&, const CoordT&>(), "min"_a, "max"_a)
.def(nb::init<>(),
"Construct an empty CoordBBox (min > max sentinel).")
.def(nb::init<const CoordT&, const CoordT&>(), "min"_a, "max"_a,
"Construct from explicit min and max Coord corners (inclusive).")
.def(
"__iter__",
[](const math::BBox<CoordT>& b) { return nb::make_iterator(nb::type<math::BBox<CoordT>>(), "CoordBBoxIterator", b.begin(), b.end()); },
nb::keep_alive<0, 1>())
.def_static("createCube", nb::overload_cast<const CoordT&, ValueType>(&math::BBox<CoordT>::createCube), "min"_a, "dim"_a)
.def_static("createCube", nb::overload_cast<ValueType, ValueType>(&math::BBox<CoordT>::createCube), "min"_a, "max"_a)
.def("is_divisible", &math::BBox<CoordT>::is_divisible)
.def("empty", &math::BBox<CoordT>::empty)
.def("dim", &math::BBox<CoordT>::dim)
.def("volume", &math::BBox<CoordT>::volume)
.def("isInside", nb::overload_cast<const CoordT&>(&math::BBox<CoordT>::isInside, nb::const_), "p"_a)
.def("isInside", nb::overload_cast<const math::BBox<CoordT>&>(&math::BBox<CoordT>::isInside, nb::const_), "b"_a)
.def("asFloat", &math::BBox<CoordT>::template asReal<float>)
.def("asDouble", &math::BBox<CoordT>::template asReal<double>)
.def("hasOverlap", &math::BBox<CoordT>::hasOverlap, "b"_a)
.def("expandBy", &math::BBox<CoordT>::expandBy, "padding"_a)
nb::keep_alive<0, 1>(),
"Iterate over every Coord in this CoordBBox in row-major order.")
.def_static("createCube", nb::overload_cast<const CoordT&, ValueType>(&math::BBox<CoordT>::createCube), "min"_a, "dim"_a,
"Construct a cube of side dim voxels anchored at the min Coord.")
.def_static("createCube", nb::overload_cast<ValueType, ValueType>(&math::BBox<CoordT>::createCube), "min"_a, "max"_a,
"Construct a cube spanning [min, max] in every axis.")
.def("is_divisible", &math::BBox<CoordT>::is_divisible,
"True iff this CoordBBox has more than one voxel in every axis.")
.def("empty", &math::BBox<CoordT>::empty,
"True iff this CoordBBox is empty (any min component > the matching max).")
.def("dim", &math::BBox<CoordT>::dim,
"Return max - min + 1 as a Coord of side lengths (inclusive).")
.def("volume", &math::BBox<CoordT>::volume,
"Total number of voxels enclosed by this CoordBBox.")
.def("isInside", nb::overload_cast<const CoordT&>(&math::BBox<CoordT>::isInside, nb::const_), "p"_a,
"True iff the integer point p lies inside this CoordBBox.")
.def("isInside", nb::overload_cast<const math::BBox<CoordT>&>(&math::BBox<CoordT>::isInside, nb::const_), "b"_a,
"True iff b lies entirely inside this CoordBBox.")
.def("asFloat", &math::BBox<CoordT>::template asReal<float>,
"Convert this CoordBBox to a floating-point BBox of Vec3f.")
.def("asDouble", &math::BBox<CoordT>::template asReal<double>,
"Convert this CoordBBox to a floating-point BBox of Vec3d.")
.def("hasOverlap", &math::BBox<CoordT>::hasOverlap, "b"_a,
"True iff this CoordBBox shares any voxel with b.")
.def("expandBy", &math::BBox<CoordT>::expandBy, "padding"_a,
"Grow this CoordBBox by the given padding in every direction.")
.def("__repr__", [](const CoordBBox& b) {
std::stringstream ostr;
ostr << b;
@@ -377,33 +536,12 @@ void defineMathModule(nb::module_& m)
defineBaseBBox<math::Coord>(m, "CoordBaseBBox");
defineBBoxInteger<math::Coord>(m, "CoordBBox", "Bounding box for Coord minimum and maximum");
defineNearestNeighborSampler<float>(m, "FloatNearestNeighborSampler");
defineTrilinearSampler<float>(m, "FloatTrilinearSampler");
defineTriquadraticSampler<float>(m, "FloatTriquadraticSampler");
defineTricubicSampler<float>(m, "FloatTricubicSampler");
defineNearestNeighborSampler<double>(m, "DoubleNearestNeighborSampler");
defineTrilinearSampler<double>(m, "DoubleTrilinearSampler");
defineTriquadraticSampler<double>(m, "DoubleTriquadraticSampler");
defineTricubicSampler<double>(m, "DoubleTricubicSampler");
defineNearestNeighborSampler<int32_t>(m, "Int32NearestNeighborSampler");
defineTrilinearSampler<int32_t>(m, "Int32TrilinearSampler");
defineTriquadraticSampler<int32_t>(m, "Int32TriquadraticSampler");
defineTricubicSampler<int32_t>(m, "Int32TricubicSampler");
defineNearestNeighborSampler<Vec3f>(m, "Vec3fNearestNeighborSampler");
defineTrilinearSampler<Vec3f>(m, "Vec3fTrilinearSampler");
defineTriquadraticSampler<Vec3f>(m, "Vec3fTriquadraticSampler");
defineTricubicSampler<Vec3f>(m, "Vec3fTricubicSampler");
#ifdef NANOVDB_USE_CUDA
nb::module_ cudaModule = m.def_submodule("cuda");
cudaModule.doc() = "A submodule that implements CUDA-accelerated math functions";
defineSampleFromVoxels<float>(cudaModule, "sampleFromVoxels");
defineSampleFromVoxels<double>(cudaModule, "sampleFromVoxels");
#endif
#define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) \
defineNearestNeighborSampler<T>(m, #Suffix "NearestNeighborSampler"); \
defineTrilinearSampler<T>(m, #Suffix "TrilinearSampler"); \
defineTriquadraticSampler<T>(m, #Suffix "TriquadraticSampler"); \
defineTricubicSampler<T>(m, #Suffix "TricubicSampler");
#include "BuildTypes.def"
}
} // namespace pynanovdb
+356 -23
View File
@@ -32,11 +32,9 @@ GridHandle<BufferT> createLevelSetSphere(GridType gridType,
switch (gridType) {
case GridType::Float: return createLevelSetSphere<float, BufferT>(radius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
case GridType::Double: return createLevelSetSphere<double, BufferT>(radius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
default: {
std::stringstream ss;
// ss << "Cannot createLevelSetSphere for grid of type \"" << toStr(gridType);
throw std::runtime_error(ss.str() + "\"");
}
default:
throw std::runtime_error(
"createLevelSetSphere: only float and double grid types are supported");
}
}
@@ -58,11 +56,9 @@ GridHandle<BufferT> createLevelSetTorus(GridType gridType,
return createLevelSetTorus<float, BufferT>(majorRadius, minorRadius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
case GridType::Double:
return createLevelSetTorus<double, BufferT>(majorRadius, minorRadius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
default: {
std::stringstream ss;
// ss << "Cannot createLevelSetTorus for grid of type \"" << toStr(gridType);
throw std::runtime_error(ss.str() + "\"");
}
default:
throw std::runtime_error(
"createLevelSetTorus: only float and double grid types are supported");
}
}
@@ -81,11 +77,9 @@ GridHandle<BufferT> createFogVolumeSphere(GridType gridType,
switch (gridType) {
case GridType::Float: return createFogVolumeSphere<float, BufferT>(radius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
case GridType::Double: return createFogVolumeSphere<double, BufferT>(radius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
default: {
std::stringstream ss;
// ss << "Cannot createFogVolumeSphere for grid of type \"" << toStr(gridType);
throw std::runtime_error(ss.str() + "\"");
}
default:
throw std::runtime_error(
"createFogVolumeSphere: only float and double grid types are supported");
}
}
@@ -107,14 +101,218 @@ GridHandle<BufferT> createFogVolumeTorus(GridType gridType,
return createFogVolumeTorus<float, BufferT>(majorRadius, minorRadius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
case GridType::Double:
return createFogVolumeTorus<double, BufferT>(majorRadius, minorRadius, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
default: {
std::stringstream ss;
// ss << "Cannot createFogVolumeTorus for grid of type \"" << toStr(gridType);
throw std::runtime_error(ss.str() + "\"");
default:
throw std::runtime_error(
"createFogVolumeTorus: only float and double grid types are supported");
}
}
// ---------- New primitives ----------
// Same float/double switch pattern as the four existing primitives above.
// The C++ templates also accept Fp4/Fp8/Fp16/FpN; those flavors live on
// the createNanoGrid path with explicit oracle and dither parameters
// rather than being expressed as primitive overloads here.
template<typename BufferT>
GridHandle<BufferT> createLevelSetBox(GridType gridType,
double width,
double height,
double depth,
const Vec3d& center,
double voxelSize,
double halfWidth,
const Vec3d& origin,
const std::string& name,
tools::StatsMode sMode,
CheckMode cMode,
const BufferT& buffer)
{
switch (gridType) {
case GridType::Float:
return tools::createLevelSetBox<float, BufferT>(
width, height, depth, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
case GridType::Double:
return tools::createLevelSetBox<double, BufferT>(
width, height, depth, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
default:
throw std::runtime_error(
"createLevelSetBox: only float and double grid types are supported");
}
}
template<typename BufferT>
GridHandle<BufferT> createLevelSetBBox(GridType gridType,
double width,
double height,
double depth,
double thickness,
const Vec3d& center,
double voxelSize,
double halfWidth,
const Vec3d& origin,
const std::string& name,
tools::StatsMode sMode,
CheckMode cMode,
const BufferT& buffer)
{
switch (gridType) {
case GridType::Float:
return tools::createLevelSetBBox<float, BufferT>(
width, height, depth, thickness, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
case GridType::Double:
return tools::createLevelSetBBox<double, BufferT>(
width, height, depth, thickness, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
default:
throw std::runtime_error(
"createLevelSetBBox: only float and double grid types are supported");
}
}
template<typename BufferT>
GridHandle<BufferT> createLevelSetOctahedron(GridType gridType,
double scale,
const Vec3d& center,
double voxelSize,
double halfWidth,
const Vec3d& origin,
const std::string& name,
tools::StatsMode sMode,
CheckMode cMode,
const BufferT& buffer)
{
switch (gridType) {
case GridType::Float:
return tools::createLevelSetOctahedron<float, BufferT>(
scale, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
case GridType::Double:
return tools::createLevelSetOctahedron<double, BufferT>(
scale, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
default:
throw std::runtime_error(
"createLevelSetOctahedron: only float and double grid types are supported");
}
}
template<typename BufferT>
GridHandle<BufferT> createFogVolumeBox(GridType gridType,
double width,
double height,
double depth,
const Vec3d& center,
double voxelSize,
double halfWidth,
const Vec3d& origin,
const std::string& name,
tools::StatsMode sMode,
CheckMode cMode,
const BufferT& buffer)
{
switch (gridType) {
case GridType::Float:
return tools::createFogVolumeBox<float, BufferT>(
width, height, depth, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
case GridType::Double:
return tools::createFogVolumeBox<double, BufferT>(
width, height, depth, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
default:
throw std::runtime_error(
"createFogVolumeBox: only float and double grid types are supported");
}
}
template<typename BufferT>
GridHandle<BufferT> createFogVolumeOctahedron(GridType gridType,
double scale,
const Vec3d& center,
double voxelSize,
double halfWidth,
const Vec3d& origin,
const std::string& name,
tools::StatsMode sMode,
CheckMode cMode,
const BufferT& buffer)
{
switch (gridType) {
case GridType::Float:
return tools::createFogVolumeOctahedron<float, BufferT>(
scale, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
case GridType::Double:
return tools::createFogVolumeOctahedron<double, BufferT>(
scale, center, voxelSize, halfWidth, origin, name, sMode, cMode, buffer);
default:
throw std::runtime_error(
"createFogVolumeOctahedron: only float and double grid types are supported");
}
}
// Point primitives. The result is always a PointDataGrid (uint32 storage),
// so unlike the level-set / fog-volume primitives there's no value-type
// dispatch worth exposing — the intermediate level-set's precision is
// not user-controllable in this binding. The C++ template also accepts
// BuildT=double, but that path segfaults during scatter at least with the
// current C++ implementation, so the binding stays on the float-only
// instantiation that's exercised by the C++ unit tests.
template<typename BufferT>
GridHandle<BufferT> createPointSphere(int pointsPerVoxel,
double radius,
const Vec3d& center,
double voxelSize,
const Vec3d& origin,
const std::string& name,
CheckMode mode,
const BufferT& buffer)
{
return tools::createPointSphere<float, BufferT>(
pointsPerVoxel, radius, center, voxelSize, origin, name, mode, buffer);
}
template<typename BufferT>
GridHandle<BufferT> createPointTorus(int pointsPerVoxel,
double majorRadius,
double minorRadius,
const Vec3d& center,
double voxelSize,
const Vec3d& origin,
const std::string& name,
CheckMode cMode,
const BufferT& buffer)
{
return tools::createPointTorus<float, BufferT>(
pointsPerVoxel, majorRadius, minorRadius, center, voxelSize, origin, name, cMode, buffer);
}
template<typename BufferT>
GridHandle<BufferT> createPointBox(int pointsPerVoxel,
double width,
double height,
double depth,
const Vec3d& center,
double voxelSize,
const Vec3d& origin,
const std::string& name,
CheckMode mode,
const BufferT& buffer)
{
return tools::createPointBox<float, BufferT>(
pointsPerVoxel, width, height, depth, center, voxelSize, origin, name, mode, buffer);
}
// createPointScatter takes an existing level set as its source. We bind
// the float source variant — the C++ template also accepts double, but
// the existing primitives and tests use float, and the source grid is
// the runtime-typed nanovdb::NanoGrid<float>, so a single overload keeps
// the Python surface simple.
template<typename BufferT>
GridHandle<BufferT> createPointScatter(const NanoGrid<float>& srcGrid,
int pointsPerVoxel,
const std::string& name,
CheckMode mode,
const BufferT& buffer)
{
return tools::createPointScatter<float, BufferT>(
srcGrid, pointsPerVoxel, name, mode, buffer);
}
} // namespace
template<typename BufferT> void definePrimitives(nb::module_& m)
@@ -131,7 +329,8 @@ template<typename BufferT> void definePrimitives(nb::module_& m)
"name"_a = "sphere_ls",
"sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default,
"buffer"_a = BufferT());
"buffer"_a = BufferT(),
"Narrow-band level set of a sphere of the given radius and center.");
m.def("createLevelSetTorus",
nb::overload_cast<GridType,
@@ -155,7 +354,8 @@ template<typename BufferT> void definePrimitives(nb::module_& m)
"name"_a = "torus_ls",
"sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default,
"buffer"_a = BufferT());
"buffer"_a = BufferT(),
"Narrow-band level set of a torus with the given major and minor radii.");
m.def("createFogVolumeSphere",
nb::overload_cast<GridType, double, const Vec3d&, double, double, const Vec3d&, const std::string&, tools::StatsMode, CheckMode, const BufferT&>(
@@ -169,7 +369,8 @@ template<typename BufferT> void definePrimitives(nb::module_& m)
"name"_a = "sphere_fog",
"sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default,
"buffer"_a = BufferT());
"buffer"_a = BufferT(),
"Sparse fog volume of a sphere of the given radius and center.");
m.def("createFogVolumeTorus",
nb::overload_cast<GridType,
@@ -193,7 +394,139 @@ template<typename BufferT> void definePrimitives(nb::module_& m)
"name"_a = "torus_fog",
"sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default,
"buffer"_a = BufferT());
"buffer"_a = BufferT(),
"Sparse fog volume of a torus with the given major and minor radii.");
// ---------- Level-set / fog-volume box / bbox / octahedron primitives ----
m.def("createLevelSetBox", &createLevelSetBox<BufferT>,
"gridType"_a = GridType::Float,
"width"_a = 40.0,
"height"_a = 60.0,
"depth"_a = 100.0,
"center"_a = Vec3d(0.0),
"voxelSize"_a = 1.0,
"halfWidth"_a = 3.0,
"origin"_a = Vec3d(0.0),
"name"_a = "box_ls",
"sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default,
"buffer"_a = BufferT(),
"Narrow-band level set of an axis-aligned box.");
m.def("createLevelSetBBox", &createLevelSetBBox<BufferT>,
"gridType"_a = GridType::Float,
"width"_a = 40.0,
"height"_a = 60.0,
"depth"_a = 100.0,
"thickness"_a = 10.0,
"center"_a = Vec3d(0.0),
"voxelSize"_a = 1.0,
"halfWidth"_a = 3.0,
"origin"_a = Vec3d(0.0),
"name"_a = "bbox_ls",
"sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default,
"buffer"_a = BufferT(),
"Narrow-band level set of a hollow box wireframe (BBox = bounding "
"box edges with the given thickness).");
m.def("createLevelSetOctahedron", &createLevelSetOctahedron<BufferT>,
"gridType"_a = GridType::Float,
"scale"_a = 100.0,
"center"_a = Vec3d(0.0),
"voxelSize"_a = 1.0,
"halfWidth"_a = 3.0,
"origin"_a = Vec3d(0.0),
// Default name spells the shape correctly even though the
// upstream C++ default still carries the historical
// "octadedron_ls" typo. Callers can override either way.
"name"_a = "octahedron_ls",
"sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default,
"buffer"_a = BufferT(),
"Narrow-band level set of an octahedron.");
m.def("createFogVolumeBox", &createFogVolumeBox<BufferT>,
"gridType"_a = GridType::Float,
"width"_a = 40.0,
"height"_a = 60.0,
"depth"_a = 100.0,
"center"_a = Vec3d(0.0),
"voxelSize"_a = 1.0,
"halfWidth"_a = 3.0,
"origin"_a = Vec3d(0.0),
"name"_a = "box_fog",
"sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default,
"buffer"_a = BufferT(),
"Sparse fog volume of a box (exterior 0/inactive, interior active "
"with values smoothly varying from 0 at the surface to 1 inside).");
m.def("createFogVolumeOctahedron", &createFogVolumeOctahedron<BufferT>,
"gridType"_a = GridType::Float,
"scale"_a = 100.0,
"center"_a = Vec3d(0.0),
"voxelSize"_a = 1.0,
"halfWidth"_a = 3.0,
"origin"_a = Vec3d(0.0),
"name"_a = "octahedron_fog",
"sMode"_a = tools::StatsMode::Default,
"cMode"_a = CheckMode::Default,
"buffer"_a = BufferT(),
"Sparse fog volume of an octahedron.");
// ---------- Point primitives ----------
m.def("createPointSphere", &createPointSphere<BufferT>,
"pointsPerVoxel"_a = 1,
"radius"_a = 100.0,
"center"_a = Vec3d(0.0),
"voxelSize"_a = 1.0,
"origin"_a = Vec3d(0.0),
"name"_a = "sphere_points",
"cMode"_a = CheckMode::Default,
"buffer"_a = BufferT(),
"PointDataGrid of points scattered on the surface of a sphere. "
"The output grid is always a UInt32 PointDataGrid; the "
"intermediate level-set's value type is hard-coded to float.");
m.def("createPointTorus", &createPointTorus<BufferT>,
"pointsPerVoxel"_a = 1,
"majorRadius"_a = 100.0,
"minorRadius"_a = 50.0,
"center"_a = Vec3d(0.0),
"voxelSize"_a = 1.0,
"origin"_a = Vec3d(0.0),
"name"_a = "torus_points",
"cMode"_a = CheckMode::Default,
"buffer"_a = BufferT(),
"PointDataGrid of points scattered on the surface of a torus. "
"Always returns a UInt32 PointDataGrid.");
m.def("createPointBox", &createPointBox<BufferT>,
"pointsPerVoxel"_a = 1,
"width"_a = 40.0,
"height"_a = 60.0,
"depth"_a = 100.0,
"center"_a = Vec3d(0.0),
"voxelSize"_a = 1.0,
"origin"_a = Vec3d(0.0),
"name"_a = "box_points",
"cMode"_a = CheckMode::Default,
"buffer"_a = BufferT(),
"PointDataGrid of points scattered on the surface of a box. "
"Always returns a UInt32 PointDataGrid.");
m.def("createPointScatter", &createPointScatter<BufferT>,
"srcGrid"_a,
"pointsPerVoxel"_a = 1,
"name"_a = "point_scatter",
"cMode"_a = CheckMode::Default,
"buffer"_a = BufferT(),
"Scatter a PointDataGrid into the active voxels of a "
"NanoGrid<float> level set. The source grid must satisfy "
"srcGrid.isLevelSet() and have an active bounding box; "
"non-level-set sources (e.g. fog volumes) raise RuntimeError. "
"Point coordinates are stored as blind data in world space.");
}
template void definePrimitives<HostBuffer>(nb::module_&);
+45 -25
View File
@@ -15,20 +15,53 @@ namespace {
template<typename TreeT, int Order> void defineSampleFromVoxels(nb::module_& m, const char* name)
{
using CoordT = typename TreeT::CoordType;
nb::class_<math::SampleFromVoxels<TreeT, Order, false>>(m, name)
using CoordT = typename TreeT::CoordType;
using ValueT = typename TreeT::ValueType;
using SamplerT = math::SampleFromVoxels<TreeT, Order, false>;
auto cls = nb::class_<SamplerT>(m, name,
"Callable sampler that reconstructs a grid value at an arbitrary "
"index-space position. Build via the matching create*Sampler() factory.")
.def(
"__call__", [](const math::SampleFromVoxels<TreeT, Order, false>& sampler, const CoordT& ijk) { return sampler(ijk); }, nb::is_operator(), "ijk"_a)
"__call__", [](const SamplerT& sampler, const CoordT& ijk) { return sampler(ijk); }, nb::is_operator(), "ijk"_a,
"Sample the grid at integer voxel coordinate ijk.")
.def(
"__call__", [](const math::SampleFromVoxels<TreeT, Order, false>& sampler, const Vec3f& xyz) { return sampler(xyz); }, nb::is_operator(), "xyz"_a)
"__call__", [](const SamplerT& sampler, const Vec3f& xyz) { return sampler(xyz); }, nb::is_operator(), "xyz"_a,
"Sample the grid at fractional index-space position xyz.")
.def(
"__call__", [](const math::SampleFromVoxels<TreeT, Order, false>& sampler, const Vec3d& xyz) { return sampler(xyz); }, nb::is_operator(), "xyz"_a);
"__call__", [](const SamplerT& sampler, const Vec3d& xyz) { return sampler(xyz); }, nb::is_operator(), "xyz"_a,
"Sample the grid at fractional index-space position xyz (double).");
// gradient() exists on the trilinear sampler only, zeroCrossing() on the
// trilinear and triquadratic samplers, and both static_assert a
// floating-point ValueT in C++ — mirror that gating here.
if constexpr (Order == 1 && util::is_floating_point<ValueT>::value) {
cls.def(
"gradient", [](const SamplerT& sampler, const Vec3f& xyz) { return sampler.gradient(xyz); }, "xyz"_a,
"Return the index-space gradient of the trilinear reconstruction "
"at fractional index-space position xyz. Use "
"grid.indexToWorldGrad() to move it to world space.")
.def(
"gradient", [](const SamplerT& sampler, const Vec3d& xyz) { return sampler.gradient(xyz); }, "xyz"_a,
"Return the index-space gradient at fractional index-space "
"position xyz (double).");
}
if constexpr ((Order == 1 || Order == 2) && util::is_floating_point<ValueT>::value) {
cls.def(
"zeroCrossing", [](const SamplerT& sampler, const Vec3f& xyz) { return sampler.zeroCrossing(xyz); }, "xyz"_a,
"True iff the reconstruction stencil at fractional index-space "
"position xyz straddles the zero iso-surface.")
.def(
"zeroCrossing", [](const SamplerT& sampler, const Vec3d& xyz) { return sampler.zeroCrossing(xyz); }, "xyz"_a,
"True iff the reconstruction stencil at fractional index-space "
"position xyz (double) straddles the zero iso-surface.");
}
}
template<typename TreeT, int Order> void defineCreateSampler(nb::module_& m, const char* name)
{
m.def(
name, [](const Grid<TreeT>& grid) { return math::createSampler<Order, TreeT, false>(grid.tree()); }, "grid"_a);
name, [](const Grid<TreeT>& grid) { return math::createSampler<Order, TreeT, false>(grid.tree()); }, "grid"_a,
"Build a sampler of the matching order that reads values from the "
"given grid's tree.");
}
} // namespace
@@ -65,24 +98,11 @@ template<typename BuildT> void defineTricubicSampler(nb::module_& m, const char*
defineCreateSampler<TreeT, 3>(m, "createTricubicSampler");
}
template void defineNearestNeighborSampler<float>(nb::module_&, const char*);
template void defineTrilinearSampler<float>(nb::module_&, const char*);
template void defineTriquadraticSampler<float>(nb::module_&, const char*);
template void defineTricubicSampler<float>(nb::module_&, const char*);
template void defineNearestNeighborSampler<double>(nb::module_&, const char*);
template void defineTrilinearSampler<double>(nb::module_&, const char*);
template void defineTriquadraticSampler<double>(nb::module_&, const char*);
template void defineTricubicSampler<double>(nb::module_&, const char*);
template void defineNearestNeighborSampler<int32_t>(nb::module_&, const char*);
template void defineTrilinearSampler<int32_t>(nb::module_&, const char*);
template void defineTriquadraticSampler<int32_t>(nb::module_&, const char*);
template void defineTricubicSampler<int32_t>(nb::module_&, const char*);
template void defineNearestNeighborSampler<Vec3f>(nb::module_&, const char*);
template void defineTrilinearSampler<Vec3f>(nb::module_&, const char*);
template void defineTriquadraticSampler<Vec3f>(nb::module_&, const char*);
template void defineTricubicSampler<Vec3f>(nb::module_&, const char*);
#define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) \
template void defineNearestNeighborSampler<T>(nb::module_&, const char*); \
template void defineTrilinearSampler<T>(nb::module_&, const char*); \
template void defineTriquadraticSampler<T>(nb::module_&, const char*); \
template void defineTricubicSampler<T>(nb::module_&, const char*);
#include "BuildTypes.def"
} // namespace pynanovdb
+13 -4
View File
@@ -16,6 +16,7 @@
#include "PyNanoToOpenVDB.h"
#ifdef NANOVDB_USE_CUDA
#include "cuda/PyPointsToGrid.h"
#include "cuda/PySampleFromVoxels.h"
#include "cuda/PySignedFloodFill.h"
#endif
@@ -31,12 +32,17 @@ void defineToolsModule(nb::module_& m)
defineStatsMode(m);
defineGridStatsModule(m);
defineGridValidatorModule(m);
defineEvalChecksumModule(m);
defineCreateNanoGridConversions(m);
definePrimitives<HostBuffer>(m);
defineCreateNanoGrid<float>(m, "createFloatGrid");
defineCreateNanoGrid<double>(m, "createDoubleGrid");
defineCreateNanoGrid<int32_t>(m, "createInt32Grid");
defineCreateNanoGrid<Vec3f>(m, "createVec3fGrid");
#define NANOVDB_PY_FOR_EACH_SAMPLEABLE_BUILDT(T, Suffix) \
defineCreateNanoGrid<T>(m, "create" #Suffix "Grid");
#include "BuildTypes.def"
#ifdef NANOVDB_USE_OPENVDB
defineOpenToNanoVDB<HostBuffer>(m);
@@ -55,6 +61,9 @@ void defineToolsModule(nb::module_& m)
defineSignedFloodFill<double>(cudaModule, "signedFloodFill");
definePointsToGrid<math::Rgba8>(cudaModule, "pointsToRGBA8Grid");
defineSampleFromVoxels<float>(cudaModule, "sampleFromVoxels");
defineSampleFromVoxels<double>(cudaModule, "sampleFromVoxels");
#endif
}
+121
View File
@@ -0,0 +1,121 @@
// Copyright Contributors to the OpenVDB Project
// SPDX-License-Identifier: Apache-2.0
#include "PyTree.h"
#include <nanobind/stl/string.h>
namespace nb = nanobind;
using namespace nb::literals;
using namespace nanovdb;
namespace pynanovdb {
// Polymorphic mgr() that returns the right typed NodeManager based on the
// handle's stored gridType. Dispatch follows the same X-macro pattern as
// pyHostGrid / pyDeviceGrid; unbound BuildTs return None rather than the
// generic getMgr<BuildT>() ptr that would be reinterpreted.
template<typename BufferT>
static nb::object pyNodeMgr(nb::handle py_self)
{
using HandleT = NodeManagerHandle<BufferT>;
auto& handle = nb::cast<HandleT&>(py_self);
if (!handle.data()) return nb::none();
// We need to read the stored gridType, but it's private. The public
// mgr<BuildT>() returns NULL for type mismatch, so iterate by BuildT.
// The X-macro produces one case per bound BuildT; first non-null wins.
#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \
if (auto* m = handle.template mgr<T>()) { \
return nb::cast(m, nb::rv_policy::reference, py_self); \
}
#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \
if (auto* m = handle.template mgr<T>()) { \
return nb::cast(m, nb::rv_policy::reference, py_self); \
}
#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \
if (auto* m = handle.template mgr<T>()) { \
return nb::cast(m, nb::rv_policy::reference, py_self); \
}
#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) \
if (auto* m = handle.template mgr<T>()) { \
return nb::cast(m, nb::rv_policy::reference, py_self); \
}
#include "BuildTypes.def"
return nb::none();
}
void defineNodeManagerHandle(nb::module_& m)
{
using HandleT = NodeManagerHandle<HostBuffer>;
nb::class_<HandleT>(m, "NodeManagerHandle",
"Owns the memory backing a NodeManager. Move-only. "
"Obtain via nanovdb.createNodeManager(grid).")
.def("size",
[](const HandleT& h) { return h.size(); },
"Byte size of the buffer backing this NodeManagerHandle.")
.def(
"__bool__",
[](const HandleT& h) { return h.data() != nullptr; },
nb::is_operator(),
"True iff this handle owns a non-empty buffer.")
.def("mgr", &pyNodeMgr<HostBuffer>,
nb::keep_alive<0, 1>(),
"Return the typed NodeManager for the grid this handle was "
"built from, or None if the BuildT is not Python-visible. The "
"returned NodeManager keeps this handle alive.");
}
// createNodeManager has one template instantiation per BuildT. We expose a
// single polymorphic `createNodeManager(grid)` that picks the right one
// based on the runtime type of `grid` (any nb::class_-bound NanoGrid<T>).
// nb::isinstance is a fast type check that avoids the exception-on-mismatch
// overhead that would come from trying nb::cast and catching cast_error for
// every non-matching BuildT.
template<typename BuildT>
static nb::object tryCreateNodeManager(nb::handle py_grid)
{
using GridT = NanoGrid<BuildT>;
if (!nb::isinstance<GridT>(py_grid)) {
return nb::object(); // sentinel: "not this BuildT, try next"
}
auto& grid = nb::cast<GridT&>(py_grid);
return nb::cast(createNodeManager<BuildT, HostBuffer>(grid));
}
void defineCreateNodeManager(nb::module_& m)
{
m.def("createNodeManager",
[](nb::handle py_grid) -> nb::object {
// Try every bound BuildT; first successful cast wins.
#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \
if (auto obj = tryCreateNodeManager<T>(py_grid); obj.is_valid()) { \
return obj; \
}
#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \
if (auto obj = tryCreateNodeManager<T>(py_grid); obj.is_valid()) { \
return obj; \
}
#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \
if (auto obj = tryCreateNodeManager<T>(py_grid); obj.is_valid()) { \
return obj; \
}
#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) \
if (auto obj = tryCreateNodeManager<T>(py_grid); obj.is_valid()) { \
return obj; \
}
#include "BuildTypes.def"
throw nb::type_error(
"createNodeManager: argument is not a NanoVDB grid of any "
"bound BuildT");
},
"grid"_a,
// The constructed NodeManager stores a raw pointer back to the
// grid; the handle must therefore keep the grid (and transitively
// the GridHandle that owns the grid's buffer) alive.
nb::keep_alive<0, 1>(),
"Build a NodeManager for the given grid, returning a "
"NodeManagerHandle that owns the underlying buffer. The handle's "
"mgr() method returns the typed NodeManager. The handle keeps the "
"source grid alive for as long as it lives.");
}
} // namespace pynanovdb
+486
View File
@@ -0,0 +1,486 @@
// Copyright Contributors to the OpenVDB Project
// SPDX-License-Identifier: Apache-2.0
#ifndef NANOVDB_PYTREE_HAS_BEEN_INCLUDED
#define NANOVDB_PYTREE_HAS_BEEN_INCLUDED
#include <nanobind/nanobind.h>
#include <nanobind/ndarray.h>
#include <nanobind/stl/tuple.h>
#include <nanovdb/NanoVDB.h>
#include <nanovdb/NodeManager.h>
#include <nanovdb/HostBuffer.h>
#include <type_traits> // std::is_arithmetic_v, std::enable_if
namespace nb = nanobind;
namespace pynanovdb {
// -------------------- NanoLeaf<BuildT> --------------------
//
// Binds the 8^3 leaf node. Methods that don't depend on whether the leaf
// stores a contiguous T[512] array are bound unconditionally; the
// zero-copy 512-element NumPy values() view is only bound when the leaf
// layout actually carries T mValues[512] (the BuildTraits<T>::is_special
// types use packed / index / mask layouts where a 512-element T view is
// either impossible or misleading).
template<typename BuildT> void defineNanoLeaf(nb::module_& m, const char* name)
{
using LeafT = nanovdb::NanoLeaf<BuildT>;
using ValueT = typename LeafT::ValueType;
using CoordT = typename LeafT::CoordType;
auto cls = nb::class_<LeafT>(m, name,
"Leaf node — 8x8x8 voxels. Inherits stats and bbox from the same "
"leaf-data block bound across BuildTs.");
cls.def("origin", &LeafT::origin,
"Index-space origin (minimum corner) of this leaf node.")
.def("bbox", &LeafT::bbox,
"Index-space bounding box of this leaf's active voxels.")
.def("hasBBox", &LeafT::hasBBox,
"True iff this leaf carries a cached active bounding box.")
.def_static("dim", &LeafT::dim,
"Side length of a leaf node in voxels (always 8).")
.def_static("voxelCount", &LeafT::voxelCount,
"Total voxel count in a leaf (always 512).")
.def("memUsage", &LeafT::memUsage,
"Byte size of this leaf node.")
.def("isActive",
nb::overload_cast<const CoordT&>(&LeafT::isActive, nb::const_),
nb::arg("ijk"),
"True iff the voxel at index-space ijk is active.")
.def("isActive",
[](const LeafT& leaf, uint32_t n) {
// Underlying mValueMask.isOn(n) is unchecked; release builds
// skip the C++ NANOVDB_ASSERT and would silently read OOB.
if (n >= LeafT::voxelCount()) {
throw nb::index_error(
"Leaf.isActive(n): n out of range [0, voxelCount)");
}
return leaf.isActive(n);
},
nb::arg("n"),
"True iff the n-th voxel (linear index into the 512-element leaf) is active.")
.def("getValue",
[](const LeafT& leaf, uint32_t offset) {
// mValues[offset] is unchecked in C++; guard the Python side.
if (offset >= LeafT::voxelCount()) {
throw nb::index_error(
"Leaf.getValue(offset): offset out of range [0, voxelCount)");
}
return leaf.getValue(offset);
},
nb::arg("offset"),
"Return the value at linear offset (0..511) within this leaf.")
.def("getValue",
nb::overload_cast<const CoordT&>(&LeafT::getValue, nb::const_),
nb::arg("ijk"),
"Return the value at index-space ijk; ijk must lie inside this leaf.")
.def("getFirstValue", &LeafT::getFirstValue,
"Value at the leaf's first voxel (linear offset 0).")
.def("getLastValue", &LeafT::getLastValue,
"Value at the leaf's last voxel (linear offset 511).")
.def("minimum", &LeafT::minimum,
"Minimum active value within this leaf.")
.def("maximum", &LeafT::maximum,
"Maximum active value within this leaf.")
.def("average", &LeafT::average,
"Average of active values within this leaf.")
.def("stdDeviation", &LeafT::stdDeviation,
"Standard deviation of active values within this leaf.")
// NOTE: variance() omitted — NanoVDB.h line 4388 uses unqualified
// Pow2() which fails ADL for non-float ValueTs (ValueIndex /
// ValueMask / etc.). Users can compute it as stdDeviation() ** 2.
.def("flags", &LeafT::flags,
"Raw leaf flag bits.")
.def("valueMask", &LeafT::valueMask, nb::rv_policy::reference_internal,
"Reference to the leaf's 512-bit active-value mask.")
.def("probeValue",
[](const LeafT& leaf, const CoordT& ijk) {
ValueT v;
bool on = leaf.probeValue(ijk, v);
return std::make_tuple(v, on);
},
nb::arg("ijk"),
"Return (value, isActive) for the voxel at index-space ijk.");
// Zero-copy 512-element NumPy view of mValues. Only enabled for
// BuildTs whose ValueType is a primitive arithmetic type (float,
// double, int*). For Fp* / Index / Mask / bool / Point the leaf uses
// packed / mask / void layouts. For Vec3f / Vec3d / Vec4f / Vec4d /
// Vec3u8 / Vec3u16 / Rgba8 the leaf carries a struct array that
// nanobind's ndarray can't represent directly — those would need a
// flattened (count, dim) component-typed view which a follow-up can
// add. Users can still walk every leaf via the bound getValue().
if constexpr (std::is_arithmetic_v<ValueT>
&& !nanovdb::BuildTraits<BuildT>::is_special) {
cls.def("values",
[](nb::handle py_self) {
auto& leaf = nb::cast<LeafT&>(py_self);
size_t shape[1] = {LeafT::voxelCount()};
return nb::cast(
nb::ndarray<nb::numpy, ValueT, nb::ndim<1>, nb::c_contig, nb::device::cpu>(
static_cast<void*>(leaf.data()->mValues),
size_t(1), shape, py_self),
nb::rv_policy::reference);
},
nb::keep_alive<0, 1>(),
"Return a zero-copy NumPy view of the 512 leaf values. The view "
"keeps the leaf (and transitively the GridHandle that owns the "
"underlying buffer) alive.");
}
}
// -------------------- NanoUpper / NanoLower<BuildT> --------------------
//
// Both internal node levels share the same C++ API surface (just different
// LOG2DIM). One helper templated on the concrete InternalNode type.
template<typename InternalT>
void defineInternalNodeBase(nb::class_<InternalT>& cls)
{
using ValueT = typename InternalT::ValueType;
using CoordT = typename InternalT::CoordType;
cls.def("origin", &InternalT::origin,
"Index-space origin (minimum corner) of this internal node.")
.def("bbox", &InternalT::bbox,
"Index-space bounding box of this internal node's active voxels.")
.def_static("dim", &InternalT::dim,
"Side length in voxels covered by this internal node.")
.def_static("memUsage", []() { return InternalT::memUsage(); },
"Byte size of an internal node.")
.def("minimum", &InternalT::minimum,
"Minimum active value within this node's subtree.")
.def("maximum", &InternalT::maximum,
"Maximum active value within this node's subtree.")
.def("average", &InternalT::average,
"Average of active values within this node's subtree.")
.def("stdDeviation", &InternalT::stdDeviation,
"Standard deviation of active values within this node's subtree.")
// variance() omitted for parity with the leaf binding; compute as
// stdDeviation() ** 2 in Python.
.def("valueMask", &InternalT::valueMask, nb::rv_policy::reference_internal,
"Reference to the node's active-tile mask.")
.def("childMask", &InternalT::childMask, nb::rv_policy::reference_internal,
"Reference to the node's child-pointer mask (1 where a child node exists).")
.def("getValue",
nb::overload_cast<const CoordT&>(&InternalT::getValue, nb::const_),
nb::arg("ijk"),
"Return the value at index-space ijk by descending into the subtree.")
.def("getFirstValue", &InternalT::getFirstValue,
"Value at the first (lowest-indexed) tile in this node.")
.def("getLastValue", &InternalT::getLastValue,
"Value at the last (highest-indexed) tile in this node.")
.def("isActive",
nb::overload_cast<const CoordT&>(&InternalT::isActive, nb::const_),
nb::arg("ijk"),
"True iff the voxel at index-space ijk is active.")
.def("probeValue",
[](const InternalT& node, const CoordT& ijk) {
ValueT v;
bool on = node.probeValue(ijk, v);
return std::make_tuple(v, on);
},
nb::arg("ijk"),
"Return (value, isActive) for the voxel at index-space ijk.");
}
template<typename BuildT> void defineNanoUpper(nb::module_& m, const char* name)
{
using UpperT = nanovdb::NanoUpper<BuildT>;
nb::class_<UpperT> cls(m, name,
"Upper internal node — 32x32x32 (covers a 4096^3 region in index space).");
defineInternalNodeBase(cls);
}
template<typename BuildT> void defineNanoLower(nb::module_& m, const char* name)
{
using LowerT = nanovdb::NanoLower<BuildT>;
nb::class_<LowerT> cls(m, name,
"Lower internal node — 16x16x16 (covers a 128^3 region in index space).");
defineInternalNodeBase(cls);
}
// -------------------- NanoRoot<BuildT> --------------------
template<typename BuildT> void defineNanoRoot(nb::module_& m, const char* name)
{
using RootT = nanovdb::NanoRoot<BuildT>;
using ValueT = typename RootT::ValueType;
using CoordT = typename RootT::CoordType;
nb::class_<RootT>(m, name, "Root node — top of the tree, holds the tile table.")
.def("background", &RootT::background, nb::rv_policy::reference_internal,
"Background value returned for inactive voxels.")
.def("tileCount", &RootT::tileCount,
"Number of active tiles in the root tile table.")
.def("getTableSize", &RootT::getTableSize,
"Total number of entries in the root tile table.")
.def("isEmpty", &RootT::isEmpty,
"True iff the root has no active tiles.")
.def("bbox", &RootT::bbox, nb::rv_policy::reference_internal,
"Index-space bounding box of every active value in this tree.")
.def("minimum", &RootT::minimum, nb::rv_policy::reference_internal,
"Minimum active value in the tree.")
.def("maximum", &RootT::maximum, nb::rv_policy::reference_internal,
"Maximum active value in the tree.")
.def("average", &RootT::average, nb::rv_policy::reference_internal,
"Average of active values in the tree.")
.def("stdDeviation", &RootT::stdDeviation, nb::rv_policy::reference_internal,
"Standard deviation of active values in the tree.")
.def("memUsage",
nb::overload_cast<>(&RootT::memUsage, nb::const_),
"Byte size of the root node and its tile table.")
.def("getValue",
nb::overload_cast<const CoordT&>(&RootT::getValue, nb::const_),
nb::arg("ijk"),
"Return the value at index-space ijk by walking from the root.")
.def("isActive",
nb::overload_cast<const CoordT&>(&RootT::isActive, nb::const_),
nb::arg("ijk"),
"True iff the voxel at index-space ijk is active.")
.def("probeValue",
[](const RootT& root, const CoordT& ijk) {
ValueT v;
bool on = root.probeValue(ijk, v);
return std::make_tuple(v, on);
},
nb::arg("ijk"),
"Return (value, isActive) for the voxel at index-space ijk.");
}
// -------------------- NanoTree<BuildT> --------------------
template<typename BuildT> void defineNanoTree(nb::module_& m, const char* name)
{
using TreeT = nanovdb::NanoTree<BuildT>;
using ValueT = typename TreeT::ValueType;
using CoordT = typename TreeT::CoordType;
using RootT = typename TreeT::RootType;
using UpperT = typename TreeT::UpperNodeType;
using LowerT = typename TreeT::LowerNodeType;
using LeafT = typename TreeT::LeafNodeType;
nb::class_<TreeT>(m, name,
"Tree — owns the root and provides bulk metadata queries "
"(node counts, active voxel count, extrema).")
.def("root",
nb::overload_cast<>(&TreeT::root, nb::const_),
nb::rv_policy::reference_internal,
"Root node of this tree. Lifetime is anchored to the tree.")
.def("background", &TreeT::background, nb::rv_policy::reference_internal,
"Background value returned for inactive voxels.")
.def("activeVoxelCount", &TreeT::activeVoxelCount,
"Total number of active voxels in this tree.")
// activeTileCount(level): valid range is 1..3 (lower / upper / root
// tile counts). C++ uses NANOVDB_ASSERT(level > 0 && level <= 3)
// which is a no-op in release builds — so guard explicitly.
.def("activeTileCount",
[](const TreeT& tree, uint32_t level) -> uint32_t {
if (level < 1 || level > 3) {
throw nb::value_error(
"Tree.activeTileCount(level): level must be 1, 2, or 3");
}
return tree.activeTileCount(level);
},
nb::arg("level"),
"Number of active tiles at the given tree level (1=lower, 2=upper, 3=root).")
// nodeCount(level): valid range is 0..2 (leaf / lower / upper).
// C++ uses NANOVDB_ASSERT(level < 3), again no-op in release.
// The lambda's `int level` argument disambiguates the call against
// Tree's templated nodeCount<NodeT>() overload at the C++ level,
// so we don't need an overload_cast / static_cast wrapper here.
.def("nodeCount",
[](const TreeT& tree, int level) -> uint32_t {
if (level < 0 || level >= 3) {
throw nb::value_error(
"Tree.nodeCount(level): level must be 0, 1, or 2");
}
return tree.nodeCount(level);
},
nb::arg("level"),
"Number of nodes at the given tree level (0=leaf, 1=lower, 2=upper).")
.def("totalNodeCount", &TreeT::totalNodeCount,
"Sum of node counts across every tree level.")
.def_static("memUsage", &TreeT::memUsage,
"Byte size of a single tree structure (header only).")
.def("getValue",
nb::overload_cast<const CoordT&>(&TreeT::getValue, nb::const_),
nb::arg("ijk"),
"Return the value at index-space ijk by descending from the root.")
.def("isActive", &TreeT::isActive, nb::arg("ijk"),
"True iff the voxel at index-space ijk is active.")
.def("probeValue",
[](const TreeT& tree, const CoordT& ijk) {
ValueT v;
bool on = tree.probeValue(ijk, v);
return std::make_tuple(v, on);
},
nb::arg("ijk"),
"Return (value, isActive) for the voxel at index-space ijk.")
.def("extrema",
[](const TreeT& tree) {
ValueT mn, mx;
tree.extrema(mn, mx);
return std::make_tuple(mn, mx);
},
"Return (min, max) of the active values over the whole tree.")
.def("getFirstLeaf",
nb::overload_cast<>(&TreeT::getFirstLeaf, nb::const_),
nb::rv_policy::reference_internal,
"First leaf node in breadth-first order, or None if the tree is empty.")
.def("getFirstLower",
nb::overload_cast<>(&TreeT::getFirstLower, nb::const_),
nb::rv_policy::reference_internal,
"First lower internal node in breadth-first order, or None if none exists.")
.def("getFirstUpper",
nb::overload_cast<>(&TreeT::getFirstUpper, nb::const_),
nb::rv_policy::reference_internal,
"First upper internal node in breadth-first order, or None if none exists.");
}
// -------------------- NodeManager<BuildT> --------------------
//
// NodeManager is heap-managed by a NodeManagerHandle (move-only, owns the
// underlying memory). We bind one NodeManager class per BuildT and one
// host-side NodeManagerHandle class. Users get a handle from
// nanovdb.createNodeManager(grid); they then call handle.mgr() to obtain a
// borrowed pointer to the typed NodeManager — its lifetime is anchored to
// the handle via reference_internal.
template<typename BuildT> void defineNodeManager(nb::module_& m, const char* name)
{
using NMT = nanovdb::NodeManager<BuildT>;
nb::class_<NMT>(m, name,
"Sequential breadth-first accessor for the leaf / lower / upper "
"internal nodes of a NanoGrid. Construct via "
"nanovdb.createNodeManager(grid).")
.def("isLinear",
nb::overload_cast<>(&NMT::isLinear, nb::const_),
"True iff this NodeManager is laid out as a linear offset table over a breadth-first grid.")
.def("memUsage",
nb::overload_cast<>(&NMT::memUsage, nb::const_),
"Byte size of this NodeManager.")
.def("nodeCount",
[](const NMT& nm, int level) -> uint64_t {
// Mirror Tree.nodeCount bounds (NodeManager forwards to Tree).
if (level < 0 || level >= 3) {
throw nb::value_error(
"NodeManager.nodeCount(level): level must be 0, 1, or 2");
}
return nm.nodeCount(level);
},
nb::arg("level"),
"Number of nodes at the given tree level (0=leaf, 1=lower, 2=upper).")
.def("leafCount", &NMT::leafCount,
"Number of leaf nodes managed by this NodeManager.")
.def("lowerCount", &NMT::lowerCount,
"Number of lower internal nodes managed by this NodeManager.")
.def("upperCount", &NMT::upperCount,
"Number of upper internal nodes managed by this NodeManager.")
// leaf / lower / upper: NANOVDB_ASSERT(i < nodeCount(LEVEL)) in C++ is
// no-op in release, so guard explicitly to convert OOB access into a
// Python IndexError instead of memory corruption.
.def("leaf",
[](const NMT& nm, uint32_t i) -> const nanovdb::NanoLeaf<BuildT>& {
if (i >= nm.leafCount()) {
throw nb::index_error(
"NodeManager.leaf(i): i out of range [0, leafCount)");
}
return nm.leaf(i);
},
nb::rv_policy::reference_internal, nb::arg("i"),
"Return the i-th leaf node in breadth-first order.")
.def("lower",
[](const NMT& nm, uint32_t i) -> const nanovdb::NanoLower<BuildT>& {
if (i >= nm.lowerCount()) {
throw nb::index_error(
"NodeManager.lower(i): i out of range [0, lowerCount)");
}
return nm.lower(i);
},
nb::rv_policy::reference_internal, nb::arg("i"),
"Return the i-th lower internal node in breadth-first order.")
.def("upper",
[](const NMT& nm, uint32_t i) -> const nanovdb::NanoUpper<BuildT>& {
if (i >= nm.upperCount()) {
throw nb::index_error(
"NodeManager.upper(i): i out of range [0, upperCount)");
}
return nm.upper(i);
},
nb::rv_policy::reference_internal, nb::arg("i"),
"Return the i-th upper internal node in breadth-first order.");
}
void defineNodeManagerHandle(nb::module_& m);
void defineCreateNodeManager(nb::module_& m);
// -------------------- grid.leaf_values() bulk extractor --------------------
//
// For non-special BuildTs with breadth-first, fixed-size leaves, the leaf
// values can be reached as a contiguous (N_leaves, 512) array — every leaf
// occupies sizeof(NanoLeaf<T>) bytes and mValues starts at a known offset
// inside each leaf. We bind this on NanoGrid<T> as leaf_values() for
// efficient bulk analytics from Python.
template<typename BuildT, typename = void>
struct PyLeafValuesBinder
{
template<typename ClsT> static void apply(ClsT&) {}
};
template<typename BuildT>
struct PyLeafValuesBinder<BuildT,
typename std::enable_if<
std::is_arithmetic_v<typename nanovdb::NanoLeaf<BuildT>::ValueType>
&& !nanovdb::BuildTraits<BuildT>::is_special>::type>
{
template<typename ClsT>
static void apply(ClsT& cls)
{
using GridT = nanovdb::NanoGrid<BuildT>;
using LeafT = nanovdb::NanoLeaf<BuildT>;
using ValueT = typename LeafT::ValueType;
cls.def("leaf_values",
[](nb::handle py_self) -> nb::object {
auto& grid = nb::cast<GridT&>(py_self);
const auto& tree = grid.tree();
const uint32_t nLeaves = tree.template nodeCount<LeafT>();
if (!grid.isBreadthFirst()) {
throw nb::value_error(
"leaf_values() requires a breadth-first grid "
"layout; rebuild via "
"nanovdb.tools.createNanoGrid(...).");
}
// For an empty grid (no leaves) we still return an ndarray
// — shape (0, 512) — so callers can iterate / np.asarray()
// / shape-test without branching on a None sentinel.
LeafT* first = const_cast<LeafT*>(tree.getFirstLeaf());
size_t shape[2] = {nLeaves, LeafT::voxelCount()};
int64_t strides[2] = {
static_cast<int64_t>(sizeof(LeafT) / sizeof(ValueT)),
1
};
// first is non-null whenever nLeaves > 0; when nLeaves == 0
// we pass a dummy non-null aligned pointer (the grid itself)
// so nanobind has something to base the empty array on.
// Nothing will be read since the leading shape is 0.
void* data = (first != nullptr)
? static_cast<void*>(first->data()->mValues)
: static_cast<void*>(&grid);
return nb::cast(
nb::ndarray<nb::numpy, ValueT, nb::ndim<2>, nb::device::cpu>(
data, size_t(2), shape, py_self, strides),
nb::rv_policy::reference);
},
nb::keep_alive<0, 1>(),
"Return a zero-copy (N_leaves, 512) NumPy view of every leaf's "
"values, in breadth-first leaf order. Available only for "
"BuildTs whose leaf layout carries T mValues[512] (i.e. not "
"Fp*, Index, Mask, bool, or Point) and only on breadth-first "
"grids. Returns an empty (0, 512) array for grids with no "
"leaves. The view keeps the grid alive.");
}
};
} // namespace pynanovdb
#endif
@@ -0,0 +1,547 @@
// Copyright Contributors to the OpenVDB Project
// SPDX-License-Identifier: Apache-2.0
#include "PyVoxelBlockManager.h"
#include <nanobind/ndarray.h>
#include <nanobind/stl/tuple.h>
#include <nanovdb/NanoVDB.h>
#include <nanovdb/HostBuffer.h>
#include <nanovdb/tools/CreateNanoGrid.h>
#include <nanovdb/tools/VoxelBlockManager.h>
#include <nanovdb/util/Util.h>
#include <memory>
#include <string>
#include <type_traits>
#include <utility>
namespace nb = nanobind;
using namespace nb::literals;
using namespace nanovdb;
using nanovdb::tools::VoxelBlockManager;
using nanovdb::tools::VoxelBlockManagerBase;
using nanovdb::tools::VoxelBlockManagerHandle;
using nanovdb::tools::buildVoxelBlockManager;
namespace pynanovdb {
// ----------------------- Log2BlockWidth dispatch --------------------------
//
// Log2BlockWidth is a compile-time template parameter on every VBM helper.
// We expose it to Python as a runtime int and dispatch via a switch that
// instantiates the four useful widths (BlockWidth = 64, 128, 256, 512).
// Larger widths are not bound by default; callers who need them can add a
// new case below.
template<typename F>
static auto dispatchLog2BlockWidth(int log2BlockWidth, F&& fn)
{
switch (log2BlockWidth) {
case 6: return fn(std::integral_constant<int, 6>{});
case 7: return fn(std::integral_constant<int, 7>{});
case 8: return fn(std::integral_constant<int, 8>{});
case 9: return fn(std::integral_constant<int, 9>{});
default:
throw nb::value_error(
"VoxelBlockManager: log2_block_width must be 6, 7, 8, or 9 "
"(BlockWidth = 64, 128, 256, or 512). Larger widths are not "
"bound in Python by default.");
}
}
// PyVBMHandle wraps the C++ VoxelBlockManagerHandle and carries the
// log2_block_width the handle was built with. The C++ handle does NOT store
// log2_block_width itself, so without this wrapper the Python binding would
// have to ask the caller every time — which the user can lie about and
// trigger out-of-bounds reads of the metadata buffers. Storing it once at
// build time and consulting it in every accessor closes that hole.
struct PyVBMHandle
{
VoxelBlockManagerHandle<HostBuffer> handle;
int log2BlockWidth = 6;
PyVBMHandle() = default;
PyVBMHandle(VoxelBlockManagerHandle<HostBuffer>&& h, int lbw) noexcept
: handle(std::move(h)), log2BlockWidth(lbw) {}
PyVBMHandle(const PyVBMHandle&) = delete;
PyVBMHandle& operator=(const PyVBMHandle&) = delete;
PyVBMHandle(PyVBMHandle&&) = default;
PyVBMHandle& operator=(PyVBMHandle&&) = default;
uint64_t blockCount() const { return handle.blockCount(); }
uint64_t firstOffset() const { return handle.firstOffset(); }
uint64_t lastOffset() const { return handle.lastOffset(); }
void reset() { handle.reset(); }
int blockWidth() const { return 1 << log2BlockWidth; }
int jumpMapLength() const { return 1 << (log2BlockWidth - 6); }
};
// ----------------------- decodeInverseMaps helper -------------------------
//
// Common implementation used by both the free function and the
// handle.decodeBlock(i) method. Allocates fresh leafIndex (uint32) and
// voxelOffset (uint16) NumPy arrays of length BlockWidth and fills them.
template<int Log2BlockWidth>
static nb::object pyDecodeInverseMapsImpl(const NanoGrid<ValueOnIndex>& grid,
uint32_t firstLeafID,
const uint64_t* jumpMap,
uint64_t blockFirstOffset)
{
constexpr int BlockWidth = 1 << Log2BlockWidth;
constexpr int JumpMapLength =
VoxelBlockManagerBase<Log2BlockWidth>::JumpMapLength;
// The C++ decodeInverseMaps iterates leafID = firstLeafID ..
// firstLeafID + nExtraLeaves, where nExtraLeaves is the popcount of
// this block's jumpMap (each set bit marks an additional leaf
// boundary crossed within the block). If the jumpMap is corrupt or
// was built against a different grid, the loop could read past
// tree.getFirstNode<0>(). Pre-compute the upper bound and validate
// it against grid.tree().nodeCount(0) before any allocation.
uint32_t nExtraLeaves = 0;
for (int i = 0; i < JumpMapLength; ++i)
nExtraLeaves += util::countOn(jumpMap[i]);
const uint32_t nLeaves = grid.tree().nodeCount(0);
if (uint64_t(firstLeafID) + uint64_t(nExtraLeaves) >= nLeaves) {
throw nb::value_error(
"decodeInverseMaps: firstLeafID + popcount(jumpMap) would "
"index past grid.tree().nodeCount(0) — the jumpMap is "
"either corrupt or was paired with a different grid.");
}
// Each call allocates fresh BlockWidth-sized output arrays for the
// leaf-index and voxel-offset results. We use plain new[] (rather than
// a numpy-allocated buffer) because the produced ndarrays are returned
// by reference and Python owns them via the capsule deleters below —
// when the ndarray is destroyed, the capsule's deleter runs delete[].
//
// The raw pointers live in std::unique_ptr until the matching capsule
// has been constructed; that way if the second allocation, the
// decodeInverseMaps call, or either capsule construction throws, the
// unique_ptr unwinds the half-built state cleanly. After a capsule
// takes ownership we release() so the unique_ptr no longer double-frees.
std::unique_ptr<uint32_t[]> leafIndex(new uint32_t[BlockWidth]);
std::unique_ptr<uint16_t[]> voxelOffset(new uint16_t[BlockWidth]);
using VBM = VoxelBlockManager<Log2BlockWidth>;
{
// Release the GIL around the pure-C++ decode kernel — the heavy part,
// and the only part of this helper that touches no Python objects. The
// GIL is re-acquired on scope exit (including during exception unwind)
// before the capsules / ndarrays below are constructed.
nb::gil_scoped_release release;
VBM::template decodeInverseMaps<ValueOnIndex>(
&grid, firstLeafID, jumpMap, blockFirstOffset,
leafIndex.get(), voxelOffset.get());
}
// nb::capsule wraps the raw pointer + matching delete[] so it can serve
// as the ndarray's owner — the capsule lives as long as the ndarray and
// its destruction runs the deleter.
nb::capsule leafOwner(leafIndex.get(),
[](void* p) noexcept { delete[] static_cast<uint32_t*>(p); });
auto* leafRaw = leafIndex.release();
nb::capsule offsetOwner(voxelOffset.get(),
[](void* p) noexcept { delete[] static_cast<uint16_t*>(p); });
auto* offsetRaw = voxelOffset.release();
size_t shape[1] = {static_cast<size_t>(BlockWidth)};
nb::ndarray<nb::numpy, uint32_t, nb::ndim<1>, nb::c_contig, nb::device::cpu>
leafArr(leafRaw, size_t(1), shape, leafOwner);
nb::ndarray<nb::numpy, uint16_t, nb::ndim<1>, nb::c_contig, nb::device::cpu>
offsetArr(offsetRaw, size_t(1), shape, offsetOwner);
return nb::make_tuple(
nb::cast(leafArr, nb::rv_policy::reference),
nb::cast(offsetArr, nb::rv_policy::reference));
}
// ------------------- VoxelBlockManagerHandle binding ----------------------
static const NanoGrid<ValueOnIndex>* castOnIndexGrid(nb::handle py_grid,
const char* fn_name)
{
if (!nb::isinstance<NanoGrid<ValueOnIndex>>(py_grid)) {
std::string msg(fn_name);
msg += ": grid must be a NanoVDB grid of build type ValueOnIndex (OnIndexGrid)";
throw nb::type_error(msg.c_str());
}
return &nb::cast<const NanoGrid<ValueOnIndex>&>(py_grid);
}
static void defineHandle(nb::module_& toolsModule)
{
nb::class_<PyVBMHandle>(toolsModule, "VoxelBlockManagerHandle",
"Owns the firstLeafID / jumpMap metadata buffers backing a "
"VoxelBlockManager. Constructed by nanovdb.tools.buildVoxelBlockManager.")
.def(nb::init<>(),
"Construct an empty VoxelBlockManagerHandle with no buffers.")
.def("blockCount", &PyVBMHandle::blockCount,
"Number of voxel blocks managed by this handle.")
.def("firstOffset", &PyVBMHandle::firstOffset,
"Sequential voxel index of the first active voxel covered "
"by this handle (1 by default when the handle covers the "
"full grid).")
.def("lastOffset", &PyVBMHandle::lastOffset,
"Sequential voxel index of the last active voxel covered "
"by this handle.")
.def("reset", &PyVBMHandle::reset,
"Release this handle's buffers and reset it to the empty state.")
.def_prop_ro("log2_block_width", [](const PyVBMHandle& h) { return h.log2BlockWidth; },
"The log2_block_width this handle was built with. The jumpMap "
"and decodeBlock outputs derive their shapes from this value.")
.def_prop_ro("block_width", &PyVBMHandle::blockWidth,
"BlockWidth = 1 << log2_block_width (64, 128, 256, or 512).")
.def_prop_ro("jump_map_length", &PyVBMHandle::jumpMapLength,
"JumpMapLength = BlockWidth / 64 (1, 2, 4, or 8).")
.def(
"__bool__",
[](const PyVBMHandle& h) { return h.blockCount() > 0; },
nb::is_operator())
// Zero-copy view of the (blockCount,) firstLeafID array.
.def("firstLeafID",
[](nb::handle py_self) -> nb::object {
auto& h = nb::cast<PyVBMHandle&>(py_self);
size_t shape[1] = {static_cast<size_t>(h.blockCount())};
// A default-constructed or reset() handle has a null
// hostFirstLeafID(); we still return an empty (0,) ndarray
// so callers don't have to branch on a None sentinel. The
// dummy non-null pointer (the handle itself) keeps nanobind
// happy; nothing is read since the leading shape is 0.
uint32_t* raw = h.handle.hostFirstLeafID();
void* data = (raw != nullptr) ? static_cast<void*>(raw)
: static_cast<void*>(&h);
return nb::cast(
nb::ndarray<nb::numpy, uint32_t, nb::ndim<1>,
nb::c_contig, nb::device::cpu>(
data, size_t(1), shape, py_self),
nb::rv_policy::reference);
},
nb::keep_alive<0, 1>(),
"Return a zero-copy (blockCount,) uint32 NumPy view of the "
"firstLeafID array. Returns an empty (0,) array on a "
"default-constructed or reset() handle. The view keeps this "
"handle alive.")
// jumpMap is uint64_t[blockCount * JumpMapLength]. JumpMapLength is
// determined by the log2_block_width recorded on the handle, not by
// the caller — that way the returned view always covers exactly the
// allocated buffer, with no risk of OOB reads.
.def("jumpMap",
[](nb::handle py_self) -> nb::object {
auto& h = nb::cast<PyVBMHandle&>(py_self);
size_t shape[2] = {static_cast<size_t>(h.blockCount()),
static_cast<size_t>(h.jumpMapLength())};
// Same null-buffer guard as firstLeafID(): a
// default-constructed / reset() handle has a null
// hostJumpMap(); return an empty (0, jump_map_length)
// ndarray rather than passing nullptr to nanobind.
uint64_t* raw = h.handle.hostJumpMap();
void* data = (raw != nullptr) ? static_cast<void*>(raw)
: static_cast<void*>(&h);
return nb::cast(
nb::ndarray<nb::numpy, uint64_t, nb::ndim<2>,
nb::c_contig, nb::device::cpu>(
data, size_t(2), shape, py_self),
nb::rv_policy::reference);
},
nb::keep_alive<0, 1>(),
"Return a zero-copy (blockCount, jump_map_length) uint64 NumPy "
"view of the jumpMap. The shape is determined by the "
"log2_block_width the handle was built with. Returns an empty "
"(0, jump_map_length) array on a default-constructed or reset() "
"handle. The view keeps this handle alive.")
// Decode the inverse maps for a single block of this VBM. The
// log2_block_width is taken from the handle, so the caller cannot
// request a width that doesn't match what was built.
.def("decodeBlock",
[](PyVBMHandle& self,
nb::handle py_grid,
uint64_t block_index) -> nb::object {
const auto* grid = castOnIndexGrid(py_grid,
"VoxelBlockManagerHandle.decodeBlock");
if (block_index >= self.blockCount()) {
throw nb::index_error(
"VoxelBlockManagerHandle.decodeBlock(block_index): "
"block_index out of range [0, blockCount).");
}
// Defensive: NanoVDB's buildVoxelBlockManager doesn't always
// initialize firstLeafID for blocks where no leaf starts at
// a block boundary AND no leaf's iteration sweep reaches
// them (e.g. when the source grid is tile-compressed, so
// some sequential offsets correspond to tile values rather
// than leaf voxels). The slot is then uninitialized memory;
// passing it into decodeInverseMaps would lead to an OOB
// read of tree.getFirstNode<0>()[garbage]. Catch the case
// and raise rather than segfault.
const uint32_t firstLeafID =
self.handle.hostFirstLeafID()[block_index];
const uint32_t nLeaves = grid->tree().nodeCount(0);
if (firstLeafID >= nLeaves) {
throw nb::value_error(
"VoxelBlockManagerHandle.decodeBlock: the VBM's "
"firstLeafID for this block was not initialized by "
"buildVoxelBlockManager (the underlying algorithm "
"doesn't cover blocks that no leaf reaches via its "
"iteration). This typically happens on OnIndex "
"grids built from tile-compressed source grids; "
"until the issue is fixed upstream the workaround "
"is to build the source grid voxel-by-voxel with "
"build::Grid so it stays uncompressed.");
}
return dispatchLog2BlockWidth(self.log2BlockWidth, [&](auto W) {
constexpr int LBW = decltype(W)::value;
constexpr int BlockWidth = 1 << LBW;
constexpr int JumpMapLength =
VoxelBlockManagerBase<LBW>::JumpMapLength;
const uint64_t blockFirstOffset =
self.firstOffset() + block_index * BlockWidth;
return pyDecodeInverseMapsImpl<LBW>(
*grid,
firstLeafID,
self.handle.hostJumpMap() + block_index * JumpMapLength,
blockFirstOffset);
});
},
"grid"_a, "block_index"_a,
"Decode the inverse maps for the block_index-th block of this "
"VBM. Returns (leaf_index, voxel_offset) uint32 / uint16 NumPy "
"arrays of length BlockWidth = 1<<log2_block_width, using the "
"log2_block_width the handle was built with.");
}
// ------------------- buildVoxelBlockManager binding -----------------------
static void defineBuild(nb::module_& toolsModule)
{
toolsModule.def("buildVoxelBlockManager",
[](nb::handle py_grid,
int log2_block_width,
uint64_t first_offset,
uint64_t last_offset,
uint64_t n_blocks) -> PyVBMHandle {
const auto* grid = castOnIndexGrid(py_grid, "buildVoxelBlockManager");
// The C++ implementation only NANOVDB_ASSERTs these preconditions,
// which makes them no-ops in release builds. Validate them here
// so Python callers get a clear error instead of UB / abort.
if (!grid->isSequential()) {
throw nb::value_error(
"buildVoxelBlockManager: grid must satisfy "
"grid.isSequential() (fixed-size, breadth-first node "
"layout). NanoVDB grids constructed via "
"tools.createOnIndexGrid satisfy this by default.");
}
return dispatchLog2BlockWidth(log2_block_width, [&](auto W) {
constexpr int LBW = decltype(W)::value;
using Base = VoxelBlockManagerBase<LBW>;
constexpr uint64_t BlockWidth = Base::BlockWidth;
constexpr uint64_t JumpMapLength = Base::JumpMapLength;
// first_offset must be 1 (mod BlockWidth). The single-arg
// C++ helper would normalize a zero input to 1; we do the
// same here so the in-place builder below sees a valid
// value. Validate the nonzero case ourselves.
if (first_offset != 0 &&
((first_offset - 1) & (BlockWidth - 1)) != 0) {
throw nb::value_error(
"buildVoxelBlockManager: first_offset must satisfy "
"first_offset == 1 (mod BlockWidth). Pass 0 (the "
"default) to let the implementation use 1.");
}
if (first_offset == 0) first_offset = 1;
if (last_offset == 0) last_offset = grid->activeVoxelCount();
if (last_offset < first_offset) return PyVBMHandle();
// Capacity must hold at least ceil((last - first + 1) /
// BlockWidth) blocks; otherwise the handle's lastOffset
// would advertise more coverage than blockCount allows
// and decodeBlock would silently truncate. The formula
// below equals the ceil() above when BlockWidth is a
// power of two.
const uint64_t minBlocks =
(last_offset - first_offset + BlockWidth) >> LBW;
if (n_blocks != 0 && n_blocks < minBlocks) {
std::string msg(
"buildVoxelBlockManager: n_blocks must be at "
"least ceil((last_offset - first_offset + 1) / "
"BlockWidth) = ");
msg += std::to_string(minBlocks);
msg += ". Pass 0 (the default) to use the minimum "
"required capacity.";
throw nb::value_error(msg.c_str());
}
if (n_blocks == 0) n_blocks = minBlocks;
// Allocate the metadata buffers ourselves so we can
// pre-initialize firstLeafID with a sentinel value before
// the in-place builder runs. The C++ allocating overload
// calls HostBuffer::create() which returns uninitialized
// memory; blocks that the algorithm doesn't touch would
// then retain arbitrary values, and our decodeBlock guard
// (firstLeafID >= nLeaves) might miss any garbage value
// that happens to be < nLeaves. By prefilling with nLeaves
// up front, every untouched slot deterministically trips
// the guard.
auto firstLeafIDBuf = HostBuffer::create(
n_blocks * sizeof(uint32_t));
auto jumpMapBuf = HostBuffer::create(
n_blocks * JumpMapLength * sizeof(uint64_t));
const uint32_t nLeaves = grid->tree().nodeCount(0);
{
uint32_t* slots = static_cast<uint32_t*>(
firstLeafIDBuf.data());
for (uint64_t i = 0; i < n_blocks; ++i) {
slots[i] = nLeaves;
}
}
VoxelBlockManagerHandle<HostBuffer> handle(
std::move(firstLeafIDBuf), std::move(jumpMapBuf),
n_blocks, first_offset, last_offset);
// In-place builder zeros the jumpMap itself and only
// touches firstLeafID slots it actually visits. Release the
// GIL around it — it's pure C++ (touches no Python objects)
// and may parallelize internally via util::forEach.
{
nb::gil_scoped_release release;
buildVoxelBlockManager<LBW, HostBuffer>(grid, handle);
}
return PyVBMHandle(std::move(handle), LBW);
});
},
"grid"_a,
"log2_block_width"_a = 6,
"first_offset"_a = 0,
"last_offset"_a = 0,
"n_blocks"_a = 0,
"Build a host-side VoxelBlockManager from an OnIndexGrid. "
"log2_block_width selects the per-block active-voxel count "
"(6=64, 7=128, 8=256, 9=512). Pass 0 for first_offset / "
"last_offset / n_blocks to use the full grid (first active "
"voxel through grid.activeVoxelCount(), minimum block count). "
"first_offset, if nonzero, must satisfy first_offset == 1 "
"(mod BlockWidth).");
}
// --------------------- decodeInverseMaps binding --------------------------
static void defineDecode(nb::module_& toolsModule)
{
toolsModule.def("decodeInverseMaps",
[](nb::handle py_grid,
uint32_t first_leaf_id,
nb::ndarray<const uint64_t, nb::ndim<1>,
nb::c_contig, nb::device::cpu> jump_map,
uint64_t block_first_offset,
int log2_block_width) -> nb::object {
const auto* grid = castOnIndexGrid(py_grid, "decodeInverseMaps");
// The C++ helper indexes tree.getFirstNode<0>()[first_leaf_id]
// without a bounds check, so a stray first_leaf_id leads to an
// OOB read. Validate up front. (We also require isSequential();
// getFirstNode only makes sense on a sequential tree.)
if (!grid->isSequential()) {
throw nb::value_error(
"decodeInverseMaps: grid must satisfy "
"grid.isSequential().");
}
const uint32_t nLeaves = grid->tree().nodeCount(0);
if (first_leaf_id >= nLeaves) {
throw nb::index_error(
"decodeInverseMaps: first_leaf_id out of range "
"[0, grid.tree().nodeCount(0)).");
}
return dispatchLog2BlockWidth(log2_block_width, [&](auto W) {
constexpr int LBW = decltype(W)::value;
constexpr int JumpMapLength =
VoxelBlockManagerBase<LBW>::JumpMapLength;
if (jump_map.shape(0) != JumpMapLength) {
std::string msg("decodeInverseMaps: jump_map must have "
"length JumpMapLength = ");
msg += std::to_string(JumpMapLength);
msg += " for log2_block_width=";
msg += std::to_string(LBW);
throw nb::value_error(msg.c_str());
}
return pyDecodeInverseMapsImpl<LBW>(
*grid, first_leaf_id, jump_map.data(),
block_first_offset);
});
},
"grid"_a,
"first_leaf_id"_a,
"jump_map"_a,
"block_first_offset"_a,
"log2_block_width"_a = 6,
"Decode the inverse maps for a single voxel block of an OnIndexGrid. "
"Returns a (leaf_index, voxel_offset) tuple of fresh NumPy arrays of "
"length BlockWidth = 1<<log2_block_width. jump_map must have length "
"BlockWidth/64. first_leaf_id must be in [0, grid.tree().nodeCount(0)).");
}
// ----- createOnIndexGrid test-scaffold factory ----------------------------
//
// Narrow source-coverage factory used by the VoxelBlockManager unit tests.
// New code should prefer tools.createNanoGridOnIndex (in PyCreateNanoGrid.cc)
// which accepts a wider source set.
template<typename SrcBuildT>
static nb::object tryCreateOnIndexGrid(nb::handle py_grid,
uint32_t channels,
bool include_stats,
bool include_tiles,
int verbose)
{
using SrcGridT = NanoGrid<SrcBuildT>;
if (!nb::isinstance<SrcGridT>(py_grid)) {
return nb::object();
}
const SrcGridT& src = nb::cast<const SrcGridT&>(py_grid);
return nb::cast(
tools::createNanoGrid<SrcGridT, ValueOnIndex, HostBuffer>(
src, channels, include_stats, include_tiles, verbose));
}
static void defineCreateOnIndexGrid(nb::module_& toolsModule)
{
toolsModule.def("createOnIndexGrid",
[](nb::handle py_grid,
uint32_t channels,
bool include_stats,
bool include_tiles,
int verbose) -> nb::object {
// Try every source BuildT we accept.
if (auto r = tryCreateOnIndexGrid<float>(
py_grid, channels, include_stats, include_tiles, verbose);
r.is_valid()) return r;
if (auto r = tryCreateOnIndexGrid<double>(
py_grid, channels, include_stats, include_tiles, verbose);
r.is_valid()) return r;
if (auto r = tryCreateOnIndexGrid<int32_t>(
py_grid, channels, include_stats, include_tiles, verbose);
r.is_valid()) return r;
if (auto r = tryCreateOnIndexGrid<Vec3f>(
py_grid, channels, include_stats, include_tiles, verbose);
r.is_valid()) return r;
throw nb::type_error(
"createOnIndexGrid: source grid must be a FloatGrid, "
"DoubleGrid, Int32Grid, or Vec3fGrid (other source BuildTs "
"are not yet bound).");
},
"src_grid"_a,
"channels"_a = 0u,
"include_stats"_a = true,
"include_tiles"_a = true,
"verbose"_a = 0,
"Convert a source grid into a NanoGrid<ValueOnIndex> "
"(OnIndexGrid). Accepts FloatGrid / DoubleGrid / Int32Grid / "
"Vec3fGrid. This is a narrow helper kept alongside "
"buildVoxelBlockManager; for general index conversion (broader "
"source coverage, blind-data channels) prefer "
"nanovdb.tools.createNanoGridOnIndex.");
}
void defineVoxelBlockManagerModule(nb::module_& toolsModule)
{
defineHandle(toolsModule);
defineBuild(toolsModule);
defineDecode(toolsModule);
defineCreateOnIndexGrid(toolsModule);
}
} // namespace pynanovdb
@@ -0,0 +1,20 @@
// Copyright Contributors to the OpenVDB Project
// SPDX-License-Identifier: Apache-2.0
#ifndef NANOVDB_PYVOXELBLOCKMANAGER_HAS_BEEN_INCLUDED
#define NANOVDB_PYVOXELBLOCKMANAGER_HAS_BEEN_INCLUDED
#include <nanobind/nanobind.h>
namespace nb = nanobind;
namespace pynanovdb {
/// @brief Bind VoxelBlockManagerHandle<HostBuffer>,
/// tools.buildVoxelBlockManager, tools.decodeInverseMaps, and the
/// test-scaffold tools.createOnIndexGrid factory under the given
/// Python submodule (expected to be the existing nanovdb.tools).
void defineVoxelBlockManagerModule(nb::module_& toolsModule);
} // namespace pynanovdb
#endif
+22 -5
View File
@@ -1,9 +1,26 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
import os
import sys
if sys.platform == "win32":
import os
openvdb_dll_directory = os.path.join(os.path.dirname(os.path.abspath(__file__)), os.pardir, 'openvdb', 'lib')
os.add_dll_directory(directory)
from .lib.nanovdb import *
if sys.platform == "win32":
_openvdb_dll_directory = os.path.join(
os.path.dirname(os.path.abspath(__file__)), os.pardir, "openvdb", "lib"
)
if os.path.isdir(_openvdb_dll_directory):
os.add_dll_directory(_openvdb_dll_directory)
def get_include():
"""Return the absolute path to the bundled NanoVDB C/C++ include directory.
Use this from a downstream Python extension's build system so the extension
compiles against the same NanoVDB headers the installed wheel was built with::
import nanovdb
ext_kwargs = dict(include_dirs=[nanovdb.get_include()])
"""
return os.path.join(os.path.dirname(os.path.abspath(__file__)), "include")
from .lib.nanovdb import * # noqa: E402,F401,F403
@@ -13,7 +13,10 @@ namespace pynanovdb {
void defineDeviceBuffer(nb::module_& m)
{
nb::class_<cuda::DeviceBuffer>(m, "DeviceBuffer");
nb::class_<cuda::DeviceBuffer>(m, "DeviceBuffer",
"CUDA device-side buffer used to back a DeviceGridHandle. Holds a "
"host mirror and a device pointer; deviceUpload / deviceDownload on "
"the handle move bytes between the two.");
}
} // namespace pynanovdb
@@ -6,6 +6,7 @@
#include <nanobind/ndarray.h>
#include <nanovdb/cuda/DeviceBuffer.h>
#include <nanovdb/NanoVDB.h>
namespace nb = nanobind;
using namespace nb::literals;
@@ -13,6 +14,48 @@ using namespace nanovdb;
namespace pynanovdb {
// Device-side polymorphic deviceGrid(n) — same dispatch shape as
// pyHostGrid<BufferT> in PyGridHandle.h, but returns the device pointer.
// gridType(n) is read from the host-side GridData header (the handle keeps
// a host mirror), so this works whether or not the grid has been uploaded.
// Returns None if the device-side grid is null (i.e. no deviceUpload yet)
// or the BuildT is not Python-visible.
static nb::object pyDeviceGrid(nb::handle py_handle, uint32_t n)
{
using BufferT = nanovdb::cuda::DeviceBuffer;
auto& handle = nb::cast<GridHandle<BufferT>&>(py_handle);
if (n >= handle.gridCount()) return nb::none();
switch (handle.gridType(n)) {
#define NANOVDB_PY_FOR_EACH_SCALAR_BUILDT(T, Suffix, GridTypeEnum) \
case nanovdb::GridType::GridTypeEnum: { \
auto* grid = handle.template deviceGrid<T>(n); \
return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \
: nb::none(); \
}
#define NANOVDB_PY_FOR_EACH_VECTOR_BUILDT(T, Suffix, AccessorName, GridTypeEnum) \
case nanovdb::GridType::GridTypeEnum: { \
auto* grid = handle.template deviceGrid<T>(n); \
return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \
: nb::none(); \
}
#define NANOVDB_PY_FOR_EACH_POINT_BUILDT(T, Suffix, GridTypeEnum) \
case nanovdb::GridType::GridTypeEnum: { \
auto* grid = handle.template deviceGrid<T>(n); \
return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \
: nb::none(); \
}
#define NANOVDB_PY_FOR_EACH_READONLY_BUILDT(T, Suffix, GridTypeEnum) \
case nanovdb::GridType::GridTypeEnum: { \
auto* grid = handle.template deviceGrid<T>(n); \
return grid ? nb::cast(grid, nb::rv_policy::reference, py_handle) \
: nb::none(); \
}
#include "../BuildTypes.def"
default:
return nb::none();
}
}
void defineDeviceGridHandle(nb::module_& m)
{
using BufferT = nanovdb::cuda::DeviceBuffer;
@@ -27,19 +70,31 @@ void defineDeviceGridHandle(nb::module_& m)
new (&handle) GridHandle<BufferT>(std::move(buffer));
},
"cpu_t"_a.noconvert(),
"cuda_t"_a.noconvert())
.def("deviceFloatGrid", nb::overload_cast<uint32_t>(&GridHandle<BufferT>::template deviceGrid<float>), "n"_a = 0, nb::rv_policy::reference_internal)
.def("deviceDoubleGrid", nb::overload_cast<uint32_t>(&GridHandle<BufferT>::template deviceGrid<double>), "n"_a = 0, nb::rv_policy::reference_internal)
.def("deviceInt32Grid", nb::overload_cast<uint32_t>(&GridHandle<BufferT>::template deviceGrid<int32_t>), "n"_a = 0, nb::rv_policy::reference_internal)
.def("deviceVec3fGrid", nb::overload_cast<uint32_t>(&GridHandle<BufferT>::template deviceGrid<Vec3f>), "n"_a = 0, nb::rv_policy::reference_internal)
.def("deviceRGBA8Grid",
nb::overload_cast<uint32_t>(&GridHandle<BufferT>::template deviceGrid<math::Rgba8>),
"n"_a = 0,
nb::rv_policy::reference_internal)
"cuda_t"_a.noconvert(),
"Construct a DeviceGridHandle that wraps an existing pair of "
"host and device uint32 arrays of equal length.")
.def("deviceGrid", &pyDeviceGrid, "n"_a = 0,
nb::keep_alive<0, 1>(),
"Return the n-th device-resident grid as a typed Grid subclass "
"selected by gridType(n), or None if the BuildT is not bound in "
"Python or the device copy has not been uploaded yet. The "
"returned grid keeps this handle alive.")
.def(
"deviceUpload", [](GridHandle<BufferT>& handle, bool sync) { handle.deviceUpload(nullptr, sync); }, "sync"_a = true)
"deviceUpload", [](GridHandle<BufferT>& handle, bool sync) { handle.deviceUpload(nullptr, sync); }, "sync"_a = true,
"Copy the host-side buffer to the device. If sync is True the "
"call blocks until the transfer completes.")
.def(
"deviceDownload", [](GridHandle<BufferT>& handle, bool sync) { handle.deviceDownload(nullptr, sync); }, "sync"_a = true);
"deviceDownload", [](GridHandle<BufferT>& handle, bool sync) { handle.deviceDownload(nullptr, sync); }, "sync"_a = true,
"Copy the device-side buffer back to the host. If sync is True "
"the call blocks until the transfer completes.");
// NOTE: defineGridHandleUtilities<BufferT> intentionally NOT called for
// DeviceBuffer. Registering nanovdb.splitGrids / nanovdb.mergeGrids as a
// second overload taking a DeviceGridHandle list conflicts with the host
// overload because both signatures take nb::list, and nanobind's
// overload resolution can't disambiguate by element type — it picks one
// and the inner cast fails with std::bad_cast. A properly typed device
// variant (with its own name, or strongly-typed std::vector<HandleT>
// args via nanobind/stl/vector.h) can land later if it's needed.
}
} // namespace pynanovdb
@@ -50,7 +50,8 @@ template<typename BuildT> void definePointsToGrid(nb::module_& m, const char* na
auto handle = converter.getHandle(points, tensor.shape(0));
return handle;
},
"tensor"_a);
"tensor"_a,
"Rasterize the given (N, 3) int32 device tensor of points into a fresh GridHandle.");
}
template void definePointsToGrid<nanovdb::math::Rgba8>(nb::module_&, const char*);
+48
View File
@@ -0,0 +1,48 @@
# NanoVDB Python Examples
Runnable `.py` scripts demonstrating the NanoVDB Python bindings.
Each script is self-contained, builds its own input data, and prints
a small summary to stdout.
Run any example with:
```bash
python load_inspect.py
```
When working from the source tree, the `nanovdb` module needs to be
on `PYTHONPATH` (e.g. via the build output directory):
```bash
cd <build>/nanovdb/nanovdb/python
PYTHONPATH=. python /path/to/<example>.py
```
## Examples
| Script | What it shows |
| --- | --- |
| [`load_inspect.py`](load_inspect.py) | Polymorphic `handle.grid(n)` access, `GridMetaData` type-erased introspection, mixed-type handles via `mergeGrids`. |
| [`build_grid.py`](build_grid.py) | Voxel-by-voxel construction with `nanovdb.tools.build.FloatGrid`, the cached `ValueAccessor`, the thread-safe `WriteAccessor`, and `.toNanoVDB()` to bake into a host `GridHandle`. |
| [`bulk_leaf_numpy.py`](bulk_leaf_numpy.py) | Zero-copy `(N_leaves, 512)` NumPy view of every leaf's mValues via `grid.leaf_values()`. Includes a global-stats reduction and an in-place mutation that propagates back into the grid. Requires NumPy. |
| [`quantize.py`](quantize.py) | Quantize a `NanoGrid<float>` through `nanovdb.tools.createNanoGridFp{4,8,16,N}`. Shows fixed-width quantization with dithering and variable-width `FpN` with both `AbsDiff` and `RelDiff` oracles. |
| [`validate.py`](validate.py) | `nanovdb.tools.validateGrid` / `validateGrids`, `checkGrid`, `isValid`, and the `evalChecksum` / `validateChecksum` / `updateChecksum` round-trip. |
| [`io_roundtrip.py`](io_roundtrip.py) | `nanovdb.io` write/read round-trip over five primitives: `writeGrids` with codec fallback, `readGridMetaData`, `hasGrid`, `readGrid` by name, `splitGrids`, and zero-copy point positions via `getBlindData`. Port of `ex_write_nanovdb_grids` + `ex_read_nanovdb_sphere_accessor`. |
| [`make_funny_nanovdb.py`](make_funny_nanovdb.py) | Functor-based construction: `tools.createFloatGrid(background, name, gridClass, func, bbox)` evaluates a Python callback at every voxel. Port of `ex_make_funny_nanovdb` on a reduced domain. |
| [`make_typed_grids.py`](make_typed_grids.py) | One solid sphere per value type via `tools.build.{Float,Double,Int16,Int32,Int64,UInt32,Vec3f}Grid`, written to a single file and re-read with polymorphic `handle.grid()` dispatch. Port of `ex_make_typed_grids`. |
| [`raytrace_level_set.py`](raytrace_level_set.py) | CPU sphere-traced level-set render to PGM using `worldToIndex`, the trilinear sampler, `zeroCrossing()`, and normalized `gradient()` shading. Port of `ex_raytrace_level_set` (host path). |
| [`raytrace_fog_volume.py`](raytrace_fog_volume.py) | CPU transmittance ray-march of a fog volume to PGM using a `ReadAccessor` and `Coord.Floor` — the accessor-based sampling idiom. Port of `ex_raytrace_fog_volume` (host path). |
| [`collide_level_set.py`](collide_level_set.py) | Particles colliding with a level set: `worldToIndexF`, `tree.isActive` narrow-band test, accessor distance reads, and `sampler.gradient()` collision normals. Port of `ex_collide_level_set` (host path). |
| [`index_grid_channels.py`](index_grid_channels.py) | `tools.createNanoGridOnIndex(src, channels=1)`, `grid.valueCount()`, coordinate reads through `createChannelAccessor`, and blind-data authoring with `tools.CreateNanoGrid.addBlindData` + the writable `getBlindData` view. Extends the host half of `ex_index_grid_cuda`. Requires NumPy for the authoring section. |
| [`node_manager.py`](node_manager.py) | Linearized node iteration with `createNodeManager`: per-level counts, `leaf(i)` / `lower(i)` access, node origins, masks, and stats. Port of the host half of `ex_nodemanager_cuda`. |
| [`openvdb_interop.py`](openvdb_interop.py) | `tools.openToNanoVDB` / `nanoToOpenVDB` round-trip with accessor comparison on both sides. Self-skips unless built with `NANOVDB_USE_OPENVDB` and `openvdb` is importable. Port of `ex_openvdb_to_nanovdb_accessor`. |
Scripts that produce files write them to a fresh temporary directory
and print its path, so the source tree stays clean. The whole set is
smoke-tested by [`../test/TestExamples.py`](../test/TestExamples.py)
(ctest name `pytest_nanovdb_examples`) when the module is configured
with `NANOVDB_BUILD_PYTHON_UNITTESTS=ON`.
For full API signatures and per-argument docstrings, use Python's
`help()` on any symbol — e.g. `help(nanovdb.tools.createNanoGridFpN)`.
The bindings ship `.pyi` type stubs for IDE / type-checker support.
@@ -0,0 +1,80 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Construct a NanoVDB grid voxel-by-voxel in pure Python.
nanovdb.tools.build.<Suffix>Grid is the mutable CPU grid builder
that mirrors nanovdb::tools::build::Grid<T>. This example shows the
three construction loops you'll typically reach for (setValue
directly, the cached ValueAccessor, and the thread-safe
WriteAccessor), then bakes each build grid into a host NanoGrid via
.to_nanovdb() and reads it back through the regular polymorphic
handle.grid() API.
Run with: python build_grid.py
"""
import nanovdb
def fill_with_setValue():
"""Simplest path: setValue() directly on the build grid."""
g = nanovdb.tools.build.FloatGrid(
background=0.0, name="setValue_demo",
gridClass=nanovdb.GridClass.FogVolume)
# Plant five active voxels along the x axis.
for i in range(5):
g.setValue(nanovdb.math.Coord(i, 0, 0), float(i + 1))
return g
def fill_with_accessor():
"""Cached path: getAccessor() avoids re-walking the tree for
each write when consecutive coordinates share a leaf."""
g = nanovdb.tools.build.FloatGrid(background=0.0, name="accessor_demo")
acc = g.getAccessor()
# The accessor caches the last leaf / lower / upper node, so a
# burst of writes in a 16^3 neighborhood only walks the tree once.
for x in range(8):
for y in range(8):
for z in range(8):
if x + y + z == 7:
acc.setValue(nanovdb.math.Coord(x, y, z),
float(x * 64 + y * 8 + z))
return g
def fill_with_write_accessor():
"""Thread-safe path: getWriteAccessor() buffers writes into a
private root and merges them into the parent on destruction or
on an explicit .merge() call. Useful when fanning out to multiple
threads — one WriteAccessor per thread, no shared mutable state."""
g = nanovdb.tools.build.FloatGrid(background=0.0, name="write_accessor_demo")
wa = g.getWriteAccessor()
wa.setValue(nanovdb.math.Coord(50, 50, 50), 9.0)
# Before merge, the parent doesn't see the change yet.
assert g.getValue(nanovdb.math.Coord(50, 50, 50)) == 0.0
wa.merge()
assert g.getValue(nanovdb.math.Coord(50, 50, 50)) == 9.0
return g
def main():
for builder in (fill_with_setValue, fill_with_accessor,
fill_with_write_accessor):
g = builder()
print(f"=== {g.getName()} ===")
print(f" nodeCount (leaf, lower, upper) = {g.nodeCount()}")
# Bake into a host NanoGrid<float>.
handle = g.toNanoVDB(sMode=nanovdb.tools.StatsMode.All)
ng = handle.grid()
print(f" baked NanoGrid: type={ng.gridType()}, "
f"active={ng.activeVoxelCount()}, "
f"worldBBox={ng.worldBBox()}")
# The build grid is left untouched — we can bake again.
handle2 = g.toNanoVDB()
assert handle2.grid().activeVoxelCount() == ng.activeVoxelCount()
if __name__ == "__main__":
main()
@@ -0,0 +1,63 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Bulk per-leaf value access as a zero-copy NumPy array.
grid.leaf_values() is the highest-bandwidth path from NanoVDB into
NumPy. It returns an (N_leaves, 512) view of every leaf's mValues
without copying — modify it, slice it, feed it into a PyTorch tensor,
hash it for cache lookup, whatever you need.
Run with: python bulk_leaf_numpy.py
"""
import nanovdb
def main():
try:
import numpy as np
except ImportError:
print("This example requires numpy. Install it with: pip install numpy")
return
# Build a fog volume sphere with stats so the leaves have meaningful
# min/max attached (just for the printing below — not required by
# leaf_values itself).
handle = nanovdb.tools.createFogVolumeSphere(
radius=20.0, name="bulk_demo")
grid = handle.grid()
print(f"Grid: {grid.gridType()}, active voxels = {grid.activeVoxelCount()}, "
f"leaves = {grid.tree().nodeCount(0)}")
# leaf_values() is the zero-copy view. Modifying it modifies the grid.
bulk = grid.leaf_values()
# np.asarray adds a NumPy wrapper but doesn't copy.
arr = np.asarray(bulk)
print(f"leaf_values: shape={arr.shape}, dtype={arr.dtype}, "
f"backed by grid memory (no copy).")
# Global statistics across every voxel in every leaf, computed in C.
# 0.0 voxels (background) are excluded by using a mask.
nonzero = arr[arr != 0.0]
print(f" non-background voxels = {nonzero.size}")
print(f" min = {nonzero.min()}, max = {nonzero.max()}, "
f"mean = {nonzero.mean()}")
# Per-leaf reductions: each row of `arr` is one leaf's 512 voxels.
per_leaf_max = arr.max(axis=1)
print(f" per-leaf max (first 5): {per_leaf_max[:5]}")
# Zero-copy means writes propagate. Zero out the first leaf's values
# and read one back through the regular accessor to confirm the grid
# actually changed. (The active *mask* is unchanged — we wrote into
# mValues only — so activeVoxelCount() stays the same.)
arr[0] = 0.0
leaf = grid.tree().getFirstLeaf()
if leaf is not None:
first_value_after = leaf.getFirstValue()
print(f" zeroed first leaf's values in place: "
f"leaf.getFirstValue() = {first_value_after}, "
f"activeVoxelCount unchanged: {grid.activeVoxelCount()}")
if __name__ == "__main__":
main()
@@ -0,0 +1,85 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Collide falling particles against a level-set surface on the CPU.
Python port of the host path of ex_collide_level_set: particles fall
under gravity; each step transforms the candidate position to index
space with grid.worldToIndexF, tests the narrow band with
tree.isActive, reads the signed distance through a ReadAccessor, and
on penetration reflects the velocity about the surface normal. The C++
original builds the normal from 6-tap finite differences of the SDF;
here the bound sampler.gradient() provides the same quantity in one
call (index-space units — normalized before use).
Run with: python collide_level_set.py
"""
import random
import nanovdb
NUM_PARTICLES = 500
NUM_STEPS = 40
DT = 0.1
GRAVITY = -9.8
def main():
handle = nanovdb.tools.createLevelSetSphere(radius=100.0, name="sphere")
grid = handle.grid()
tree = grid.tree()
acc = grid.getAccessor()
sampler = nanovdb.math.createTrilinearSampler(grid)
# Seed particles above the north pole of the sphere, falling down.
rng = random.Random(42)
particles = []
for _ in range(NUM_PARTICLES):
p = [rng.uniform(-30.0, 30.0), rng.uniform(115.0, 140.0),
rng.uniform(-30.0, 30.0)]
particles.append((p, [0.0, -20.0, 0.0]))
total_collisions = 0
for step in range(NUM_STEPS):
collisions = 0
for p, v in particles:
v[1] += GRAVITY * DT
next_p = [p[i] + v[i] * DT for i in range(3)]
ijk = nanovdb.math.Coord.Floor(grid.worldToIndexF(
nanovdb.math.Vec3f(next_p[0], next_p[1], next_p[2])))
if tree.isActive(ijk): # inside the narrow band?
d = acc.getValue(ijk)
if d <= 0.0: # inside the level set?
ipos = grid.worldToIndexF(
nanovdb.math.Vec3f(next_p[0], next_p[1], next_p[2]))
n = sampler.gradient(ipos)
n.normalize()
# Project the position back to the surface and
# reflect the velocity, as in the C++ example.
for i in range(3):
next_p[i] -= d * n[i]
v_dot_n = sum(v[i] * n[i] for i in range(3))
for i in range(3):
v[i] -= 2.0 * v_dot_n * n[i]
collisions += 1
p[:] = next_p
total_collisions += collisions
if collisions:
print(f"step {step:2d}: {collisions} collisions")
print(f"total collisions over {NUM_STEPS} steps: {total_collisions}")
assert total_collisions > 0
# No particle should end up inside the surface.
worst = 0.0
for p, _ in particles:
ijk = nanovdb.math.Coord.Floor(grid.worldToIndexF(
nanovdb.math.Vec3f(p[0], p[1], p[2])))
if tree.isActive(ijk):
worst = min(worst, acc.getValue(ijk))
print(f"deepest final penetration: {worst:.3f} world units")
assert worst > -2.0 * grid.voxelSize()[0]
if __name__ == "__main__":
main()
@@ -0,0 +1,86 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Index grids, value channels, and blind-data authoring.
Part A ports the host half of ex_index_grid_cuda: a float level set is
re-encoded as a NanoGrid<ValueOnIndex> whose voxels store sequential
uint64 indices, with the original float values copied into a blind-data
channel — then read back by coordinate through a ChannelAccessor,
which the C++ example only does in a CUDA kernel. Part B authors a
brand-new blind-data channel on a grid with tools.CreateNanoGrid and
fills it in place through the writable getBlindData() NumPy view.
Run with: python index_grid_channels.py
"""
import nanovdb
def index_grid_with_channel():
src = nanovdb.tools.createLevelSetSphere(radius=50.0, name="sphere")
src_grid = src.grid()
# channels=1 copies the source values into blind-data channel 0,
# indexed by the per-voxel uint64 indices the OnIndex grid stores.
handle = nanovdb.tools.createNanoGridOnIndex(src_grid, channels=1)
grid = handle.grid()
print(f"OnIndex grid: valueCount={grid.valueCount()}, "
f"source activeVoxelCount={src_grid.activeVoxelCount()}")
assert grid.valueCount() >= src_grid.activeVoxelCount()
# createChannelAccessor inspects the channel's recorded dataType and
# returns the matching typed accessor (here: OnIndexFloat...).
channel = nanovdb.createChannelAccessor(grid, 0)
print(f"channel accessor: {type(channel).__name__}, "
f"valueCount={channel.valueCount()}")
src_acc = src_grid.getAccessor()
for ijk in (nanovdb.math.Coord(48, 0, 0), nanovdb.math.Coord(0, 50, 0),
nanovdb.math.Coord(0, 0, 52)):
via_channel = channel.getValue(ijk)
via_source = src_acc.getValue(ijk)
print(f" {ijk}: channel={via_channel:.3f} source={via_source:.3f} "
f"(linear offset {channel.getIndex(ijk)})")
assert via_channel == via_source
def author_blind_data():
try:
import numpy as np
except ImportError:
print("NumPy not found. Skipping the blind-data authoring section.")
return
src = nanovdb.tools.build.FloatGrid(0.0, "authored",
nanovdb.GridClass.Unknown)
for i in range(8):
src.setValue(nanovdb.math.Coord(i, 0, 0), float(i))
# Declare a channel at conversion time; it is allocated zero-filled
# in the baked grid and filled afterwards through the writable view.
conv = nanovdb.tools.CreateNanoGrid(src)
ch = conv.addBlindData("temperature", count=64,
dataType=nanovdb.GridType.Float)
handle = conv.getHandle()
grid = handle.grid()
print(f"authored grid has {grid.blindDataCount()} blind-data channel(s)")
view = grid.getBlindData(ch)
view[:] = np.linspace(273.0, 373.0, num=64, dtype=np.float32)
# Re-resolve the channel by name and confirm the writes persisted.
n = grid.findBlindData("temperature")
meta = grid.blindMetaData(n)
stored = grid.getBlindData(n)
print(f" {meta.name()!r}: {meta.valueCount} x {meta.dataType}, "
f"range [{stored.min():.1f}, {stored.max():.1f}]")
assert n == ch
assert stored[0] == np.float32(273.0)
def main():
index_grid_with_channel()
print()
author_blind_data()
if __name__ == "__main__":
main()
@@ -0,0 +1,103 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Write a bundle of primitive grids to a .nvdb file and read it back.
Python port of ex_write_nanovdb_grids, ex_read_nanovdb_sphere, and
ex_read_nanovdb_sphere_accessor, consolidated into one self-contained
round trip: bake five primitives, write them to a single file (with
codec fallback), inspect the file metadata without loading the grids,
re-read one grid by name and all grids at once, and split a merged
handle back apart. The point-sphere section reads the world-space
point positions through the zero-copy getBlindData() NumPy view.
Run with: python io_roundtrip.py
"""
import os
import tempfile
import nanovdb
def write_primitives(path):
handles = [
nanovdb.tools.createLevelSetSphere(radius=50.0, name="sphere_ls"),
nanovdb.tools.createLevelSetTorus(majorRadius=50.0, minorRadius=20.0,
name="torus_ls"),
nanovdb.tools.createLevelSetBox(width=40.0, height=60.0, depth=80.0,
name="box_ls"),
nanovdb.tools.createLevelSetBBox(width=40.0, height=60.0, depth=80.0,
thickness=10.0, name="bbox_ls"),
nanovdb.tools.createPointSphere(pointsPerVoxel=2, radius=50.0,
name="sphere_points"),
]
# BLOSC gives the best compression but is a build-time option; fall
# back to an uncompressed file when this module was built without it.
try:
nanovdb.io.writeGrids(path, handles, codec=nanovdb.io.Codec.BLOSC)
codec = "BLOSC"
except RuntimeError:
nanovdb.io.writeGrids(path, handles, codec=nanovdb.io.Codec.NONE)
codec = "NONE"
print(f"Wrote {len(handles)} grids to {path} (codec={codec})")
def inspect_file(path):
# readGridMetaData parses the per-grid file headers without loading
# any voxel data — the cheap way to answer "what's in this file?".
for meta in nanovdb.io.readGridMetaData(path):
print(f" {meta.gridName!r}: type={meta.gridType}, "
f"class={meta.gridClass}, voxels={meta.voxelCount}")
assert nanovdb.io.hasGrid(path, "sphere_ls")
assert not nanovdb.io.hasGrid(path, "no_such_grid")
def read_back(path):
# Read a single grid by name, then print the recognizable
# ex_read_nanovdb_sphere_accessor cross-section along the x-axis.
handle = nanovdb.io.readGrid(path, "sphere_ls")
acc = handle.grid().getAccessor()
for i in range(47, 54):
ijk = nanovdb.math.Coord(i, 0, 0)
print(f" sphere_ls({i},0,0) = {acc.getValue(ijk):.2f}")
# Read every grid, merge them into one multi-grid handle, and split
# that handle back into one handle per grid.
handles = nanovdb.io.readGrids(path)
merged = nanovdb.mergeGrids(handles)
print(f" merged handle holds {merged.gridCount()} grids")
parts = nanovdb.splitGrids(merged)
assert len(parts) == len(handles)
print(f" splitGrids -> {len(parts)} single-grid handles")
return handles
def point_positions(handles):
try:
import numpy as np
except ImportError:
print("NumPy not found. Skipping the point-positions section.")
return
# The point sphere stores its world-space positions as a blind-data
# channel; getBlindData() exposes it as a zero-copy (N, 3) view.
points = next(h.grid() for h in handles
if h.grid().gridName() == "sphere_points")
positions = points.getBlindData(0)
radii = np.linalg.norm(positions, axis=1)
print(f" {positions.shape[0]} points, "
f"|p| in [{radii.min():.2f}, {radii.max():.2f}]")
# Points are jittered within their voxel, so allow ~1.5 voxels.
assert np.all(np.abs(radii - 50.0) < 1.5)
def main():
out_dir = tempfile.mkdtemp(prefix="nanovdb_")
path = os.path.join(out_dir, "primitives.nvdb")
write_primitives(path)
inspect_file(path)
handles = read_back(path)
point_positions(handles)
print(f"Output left in {out_dir}")
if __name__ == "__main__":
main()
@@ -0,0 +1,67 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Load a NanoVDB grid and inspect it polymorphically.
handle.grid(i) returns the correct typed Python subclass for whatever
GridType the grid carries, so a single call site can handle a mixed
bundle of grid types. This example builds two grids of different
value types into one handle, then walks the handle inspecting each
grid via the polymorphic accessor plus GridMetaData (the type-erased
introspector that answers "what's in this buffer?" without knowing
BuildT at compile time).
Run with: python load_inspect.py
"""
import nanovdb
def describe_handle(handle):
print(f"Handle contains {handle.gridCount()} grid(s).")
for i in range(handle.gridCount()):
# GridType / gridSize are cheap to query on the handle itself.
gtype = handle.gridType(i)
gsize = handle.gridSize(i)
print(f" [{i}] type={gtype}, size={gsize} bytes")
# handle.grid(i) returns the matching <Suffix>Grid subclass
# at runtime — no isinstance dispatch needed at the call site.
# The grid name lives on the grid itself, not the handle.
grid = handle.grid(i)
print(f" name={grid.gridName()!r}, "
f"gridClass={grid.gridClass()}")
print(f" activeVoxelCount={grid.activeVoxelCount()}")
print(f" worldBBox={grid.worldBBox()}")
# GridMetaData is the type-erased introspector — answers
# "what's in this buffer?" without knowing BuildT.
meta = nanovdb.GridMetaData(grid)
print(f" gridSize={meta.gridSize()}, "
f"isLevelSet={meta.isLevelSet()}, "
f"isFogVolume={meta.isFogVolume()}")
def main():
# Build two grids of different types into one handle so we can
# exercise the polymorphic accessor.
h_float = nanovdb.tools.createLevelSetSphere(
radius=10.0, name="sphere_float")
h_double = nanovdb.tools.createLevelSetSphere(
gridType=nanovdb.GridType.Double, radius=10.0, name="sphere_double")
handle = nanovdb.mergeGrids([h_float, h_double])
describe_handle(handle)
print()
print("Polymorphic dispatch from a runtime GridType:")
for i in range(handle.gridCount()):
grid = handle.grid(i)
# Each typed grid carries a getAccessor() that returns the
# appropriate <Suffix>ReadAccessor — float for FloatGrid,
# double for DoubleGrid, etc.
acc = grid.getAccessor()
v = acc.getValue(nanovdb.math.Coord(0, 0, 0))
print(f" grid[{i}] accessor.getValue(0,0,0) = {v}")
if __name__ == "__main__":
main()
@@ -0,0 +1,61 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Build a level set from a Python function evaluated at every voxel.
Python port of ex_make_funny_nanovdb: a trigonometric interference
pattern is CSG-intersected with a sphere and clamped to a narrow band,
using the functor-based nanovdb.tools.createFloatGrid factory. The
callback is one Python call per voxel in the bbox, so this port uses a
65^3 domain (~275k calls, a few seconds) where the C++ original fills
[-500,500]^3 — scale `SIZE` up only if you are willing to wait.
Run with: python make_funny_nanovdb.py
"""
import math
import os
import tempfile
import nanovdb
SIZE = 32 # half-width of the cubic domain, in voxels
BACKGROUND = 5.0 # narrow-band half-width, in world units
FREQ = 0.8 # rescaled from the C++ 0.1 to fit the smaller domain
def funny(ijk):
v = 4.0 + 5.0 * (math.cos(ijk.x * FREQ) * math.sin(ijk.y * FREQ) +
math.cos(ijk.y * FREQ) * math.sin(ijk.z * FREQ) +
math.cos(ijk.z * FREQ) * math.sin(ijk.x * FREQ))
# CSG intersection with a sphere of radius SIZE.
r = math.sqrt(ijk.x ** 2 + ijk.y ** 2 + ijk.z ** 2)
v = max(v, r - SIZE)
# Clamp to the narrow band.
return max(-BACKGROUND, min(BACKGROUND, v))
def main():
bbox = nanovdb.math.CoordBBox(nanovdb.math.Coord(-SIZE),
nanovdb.math.Coord(SIZE))
print(f"Evaluating funny() over {bbox} ...")
handle = nanovdb.tools.createFloatGrid(
BACKGROUND, "funny", nanovdb.GridClass.LevelSet, funny, bbox)
grid = handle.grid()
print(f"activeVoxelCount = {grid.activeVoxelCount()}")
acc = grid.getAccessor()
probe = nanovdb.math.Coord(0, 0, 0)
print(f"value at {probe} = {acc.getValue(probe):.3f}")
assert grid.isLevelSet()
assert grid.activeVoxelCount() > 0
out_dir = tempfile.mkdtemp(prefix="nanovdb_")
path = os.path.join(out_dir, "funny.nvdb")
try:
nanovdb.io.writeGrid(path, handle, codec=nanovdb.io.Codec.BLOSC)
except RuntimeError:
nanovdb.io.writeGrid(path, handle, codec=nanovdb.io.Codec.NONE)
print(f"Wrote {path}")
if __name__ == "__main__":
main()
@@ -0,0 +1,72 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Build grids of many value types and read them back polymorphically.
Python port of ex_make_typed_grids: a small solid sphere is baked once
per value type through the matching nanovdb.tools.build.<Suffix>Grid
mutable builder, all handles are written to a single .nvdb file, and
the file is re-read with handle.grid() returning the correct typed
subclass for each grid at runtime.
Run with: python make_typed_grids.py
"""
import os
import tempfile
import nanovdb
RADIUS = 8 # voxels; ~2.1k active voxels per grid keeps this quick
def solid_sphere_coords():
r2 = RADIUS * RADIUS
for i in range(-RADIUS, RADIUS + 1):
for j in range(-RADIUS, RADIUS + 1):
for k in range(-RADIUS, RADIUS + 1):
if i * i + j * j + k * k < r2:
yield nanovdb.math.Coord(i, j, k)
def build_typed_grids():
# (builder class, grid name, background, voxel value)
specs = [
(nanovdb.tools.build.FloatGrid, "float_grid", 0.0, 1.0),
(nanovdb.tools.build.DoubleGrid, "double_grid", 0.0, 1.0),
(nanovdb.tools.build.Int16Grid, "int16_grid", 0, 1),
(nanovdb.tools.build.Int32Grid, "int32_grid", 0, 1),
(nanovdb.tools.build.Int64Grid, "int64_grid", 0, 1),
(nanovdb.tools.build.UInt32Grid, "uint32_grid", 0, 1),
(nanovdb.tools.build.Vec3fGrid, "vec3f_grid",
nanovdb.math.Vec3f(0.0), nanovdb.math.Vec3f(1.0, 0.0, 0.0)),
]
handles = []
for cls, name, background, value in specs:
grid = cls(background, name, nanovdb.GridClass.Unknown)
for ijk in solid_sphere_coords():
grid.setValue(ijk, value)
handles.append(grid.toNanoVDB())
print(f"built {name} ({cls.__name__})")
return handles
def main():
handles = build_typed_grids()
out_dir = tempfile.mkdtemp(prefix="nanovdb_")
path = os.path.join(out_dir, "custom_types.nvdb")
nanovdb.io.writeGrids(path, handles)
print(f"Wrote {len(handles)} grids to {path}")
# Re-read and dispatch: handle.grid() returns FloatGrid, Int16Grid,
# Vec3fGrid, ... according to the GridType each grid carries.
probe = nanovdb.math.Coord(0, 0, 0)
for handle in nanovdb.io.readGrids(path):
grid = handle.grid()
value = grid.getAccessor().getValue(probe)
print(f" {grid.gridName():<12} -> {type(grid).__name__:<12} "
f"active={grid.activeVoxelCount()} value(0,0,0)={value}")
assert grid.activeVoxelCount() > 0
if __name__ == "__main__":
main()
@@ -0,0 +1,50 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Iterate a grid's nodes linearly through a NodeManager.
Python port of the host half of ex_nodemanager_cuda: createNodeManager
builds linearized arrays of the tree's leaf, lower, and upper nodes so
they can be visited by index instead of by tree traversal. Each node
exposes its origin, per-node stats, and (on leaves) the raw 512-value
buffer. For bulk NumPy analytics over every leaf at once, see
bulk_leaf_numpy.py's grid.leaf_values() instead.
Run with: python node_manager.py
"""
import nanovdb
def main():
handle = nanovdb.tools.createLevelSetSphere(radius=50.0, name="sphere")
grid = handle.grid()
tree = grid.tree()
nmh = nanovdb.createNodeManager(grid)
nm = nmh.mgr()
print(f"NodeManager over {grid.gridName()!r} (linear={nm.isLinear()}):")
print(f" leaves={nm.leafCount()}, lower={nm.lowerCount()}, "
f"upper={nm.upperCount()}")
# The counts mirror the tree's per-level node counts.
assert nm.leafCount() == tree.nodeCount(0)
assert nm.lowerCount() == tree.nodeCount(1)
assert nm.upperCount() == tree.nodeCount(2)
# Linear access agrees with tree traversal.
assert nm.leaf(0).origin() == tree.getFirstLeaf().origin()
# Visit a few leaves by index: origin, activity, and value range.
for i in range(min(5, nm.leafCount())):
leaf = nm.leaf(i)
print(f" leaf[{i}] origin={leaf.origin()} "
f"on={leaf.valueMask().countOn()} "
f"min={leaf.minimum():.3f} max={leaf.maximum():.3f}")
# Internal nodes are reachable the same way.
lower = nm.lower(0)
print(f" lower[0] origin={lower.origin()} "
f"children={lower.childMask().countOn()}")
if __name__ == "__main__":
main()
@@ -0,0 +1,53 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Round-trip a grid between OpenVDB and NanoVDB.
Python port of ex_openvdb_to_nanovdb and ex_openvdb_to_nanovdb_accessor:
an OpenVDB level-set sphere is converted to NanoVDB with openToNanoVDB,
values are compared through both libraries' accessors, and the NanoVDB
grid is converted back with nanoToOpenVDB. Requires a build with
NANOVDB_USE_OPENVDB and an importable `openvdb` module; the script
skips (exit 0) when either is missing.
Run with: python openvdb_interop.py
"""
import nanovdb
def main():
if not hasattr(nanovdb.tools, "openToNanoVDB"):
print("nanovdb was built without NANOVDB_USE_OPENVDB. Skipping...")
return
try:
import openvdb
except ImportError:
print("openvdb not found. Skipping...")
return
sphere = openvdb.createLevelSetSphere(100.0)
sphere.name = "sphere"
handle = nanovdb.tools.openToNanoVDB(sphere)
grid = handle.grid()
print(f"openToNanoVDB: {grid.gridName()!r}, class={grid.gridClass()}, "
f"active={grid.activeVoxelCount()}")
assert grid.gridClass() == nanovdb.GridClass.LevelSet
# Compare a cross-section through both accessors, as the C++
# accessor example does.
open_acc = sphere.getAccessor()
nano_acc = grid.getAccessor()
for i in range(97, 104):
open_v = open_acc.getValue((i, 0, 0))
nano_v = nano_acc.getValue(nanovdb.math.Coord(i, 0, 0))
print(f" ({i},0,0): openvdb={open_v:.3f} nanovdb={nano_v:.3f}")
assert abs(open_v - nano_v) < 1e-5
# And back: NanoVDB -> OpenVDB.
back = nanovdb.tools.nanoToOpenVDB(handle)
print(f"nanoToOpenVDB: {back.name!r}, empty={back.empty()}")
assert back.name == "sphere"
assert not back.empty()
if __name__ == "__main__":
main()
@@ -0,0 +1,59 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Quantize a NanoGrid<float> down to a quantized BuildT.
nanovdb.tools.createNanoGridFp4 / Fp8 / Fp16 / FpN are the quantized
counterparts of the regular grid types. Fp4 / Fp8 / Fp16 use a fixed
bit-width per voxel. FpN picks the bit-width per leaf so each leaf
hits a user-supplied tolerance (the "oracle"). Smaller bit widths
give smaller files at the cost of approximation error.
Run with: python quantize.py
"""
import nanovdb
def gridSize_in_kb(handle):
return handle.gridSize(0) / 1024
def main():
# Source: a 50-radius sphere fog volume at full float precision.
src_handle = nanovdb.tools.createFogVolumeSphere(radius=50.0)
src_grid = src_handle.grid()
print(f"Source FloatGrid: {gridSize_in_kb(src_handle):.1f} KB, "
f"active voxels = {src_grid.activeVoxelCount()}")
# Fixed-width quantization. Each subsequent format roughly halves
# the per-voxel storage cost; dithering optional.
for fn_name, label in [
("createNanoGridFp16", "Fp16 (16-bit fixed)"),
("createNanoGridFp8", "Fp8 (8-bit fixed)"),
("createNanoGridFp4", "Fp4 (4-bit fixed)"),
]:
h = getattr(nanovdb.tools, fn_name)(src_grid, ditherOn=True)
print(f" {label}: {gridSize_in_kb(h):.1f} KB, "
f"active voxels = {h.grid().activeVoxelCount()}")
# Variable-width FpN. The oracle picks the per-leaf bit-width to
# meet a tolerance — AbsDiff for absolute error, RelDiff for
# relative error. -1 tolerance means "uninitialized" so we pass
# an explicit value.
abs_oracle = nanovdb.tools.AbsDiff(0.05) # ±0.05 per voxel
h_fpn_abs = nanovdb.tools.createNanoGridFpN(src_grid, abs_oracle)
print(f" FpN (AbsDiff 0.05): {gridSize_in_kb(h_fpn_abs):.1f} KB")
rel_oracle = nanovdb.tools.RelDiff(0.1) # 10% relative error
h_fpn_rel = nanovdb.tools.createNanoGridFpN(src_grid, rel_oracle)
print(f" FpN (RelDiff 0.10): {gridSize_in_kb(h_fpn_rel):.1f} KB")
# The output is a regular NanoGrid<FpN> — read-only, but exposes
# the standard surface (gridType, activeVoxelCount, accessor.getValue
# returning a decoded float, etc).
fpn = h_fpn_abs.grid()
print(f"\nFpN grid type: {fpn.gridType()}, "
f"voxel at (0,0,0) = {fpn.getAccessor().getValue(nanovdb.math.Coord(0, 0, 0))}")
if __name__ == "__main__":
main()
@@ -0,0 +1,95 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Ray-march a fog volume on the CPU and write a PGM image.
Python port of the host path of ex_raytrace_fog_volume: each ray is
clipped to the grid's index bounding box, then integrated with fixed
steps, accumulating transmittance from the density sampled through a
ReadAccessor at Coord.Floor of the march position — the accessor-based
idiom, in contrast to the sampler-based raytrace_level_set.py. Every
step is one Python-to-C++ call, so the default RES is modest; raise it
for a nicer image.
Run with: python raytrace_fog_volume.py
"""
import math
import os
import tempfile
import nanovdb
RES = 64 # image is RES x RES pixels
FOV_DEG = 45.0
DT = 1.0 # march step, in voxels
SIGMA = 0.2 # extinction scale applied to the sampled density
def clip_to_bbox(eye, direction, bbox):
"""Slab-clip a ray against a CoordBBox; returns (t0, t1) or None."""
t0, t1 = 0.0, math.inf
for axis in range(3):
lo = bbox.min[axis] - eye[axis]
hi = bbox.max[axis] + 1.0 - eye[axis]
d = direction[axis]
if abs(d) < 1e-12:
if lo > 0.0 or hi < 0.0:
return None
continue
near, far = lo / d, hi / d
if near > far:
near, far = far, near
t0, t1 = max(t0, near), min(t1, far)
if t0 > t1:
return None
return t0, t1
def main():
handle = nanovdb.tools.createFogVolumeSphere(radius=50.0, name="fog")
grid = handle.grid()
bbox = grid.indexBBox()
acc = grid.getAccessor()
# Perspective camera looking down -z, as in the C++ RayGenOp,
# working directly in index space (the grid transform is uniform).
dim = [bbox.max[i] + 1 - bbox.min[i] for i in range(3)]
center = [bbox.min[i] + 0.5 * dim[i] for i in range(3)]
eye = (center[0], center[1], center[2] + 2.0 * dim[2])
tan_fov = math.tan(math.radians(FOV_DEG) * 0.5)
pixels = bytearray(RES * RES)
for y in range(RES):
for x in range(RES):
px = (2.0 * (x + 0.5) / RES - 1.0) * tan_fov
py = (2.0 * (y + 0.5) / RES - 1.0) * tan_fov
norm = math.sqrt(px * px + py * py + 1.0)
direction = (px / norm, py / norm, -1.0 / norm)
span = clip_to_bbox(eye, direction, bbox)
if span is None:
continue
transmittance = 1.0
t = span[0]
while t < span[1]:
pos = nanovdb.math.Vec3d(eye[0] + t * direction[0],
eye[1] + t * direction[1],
eye[2] + t * direction[2])
density = acc.getValue(nanovdb.math.Coord.Floor(pos))
transmittance *= 1.0 - density * SIGMA * DT
if transmittance < 0.005:
break
t += DT
pixels[y * RES + x] = int(255 * (1.0 - transmittance))
out_dir = tempfile.mkdtemp(prefix="nanovdb_")
path = os.path.join(out_dir, "raytrace_fog_volume.pgm")
with open(path, "wb") as f:
f.write(f"P5\n{RES} {RES}\n255\n".encode("ascii"))
f.write(bytes(pixels))
lit = sum(1 for p in pixels if p > 0)
print(f"Rendered {RES}x{RES} image, {lit} foggy pixels -> {path}")
assert lit > 0
if __name__ == "__main__":
main()
@@ -0,0 +1,120 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Ray-trace a narrow-band level set on the CPU and write a PGM image.
Python port of the host path of ex_raytrace_level_set. The C++ example
finds surface hits with math::Ray + HDDA ZeroCrossing, which are not
bound in Python; this port re-expresses the search as sphere tracing —
the clamped SDF value itself bounds the safe step size — using the
trilinear sampler, with sampler.zeroCrossing() confirming the interval
and sampler.gradient() shading the hit. Every sample is one
Python-to-C++ call, so the default RES is modest; raise it for a nicer
image.
Run with: python raytrace_level_set.py
"""
import math
import os
import tempfile
import nanovdb
RES = 64 # image is RES x RES pixels
FOV_DEG = 45.0
LIGHT = (0.577, 0.577, 0.577) # unit vector toward the light
def clip_to_bbox(eye, direction, bbox):
"""Slab-clip a ray against a CoordBBox; returns (t0, t1) or None."""
t0, t1 = 0.0, math.inf
for axis in range(3):
lo = bbox.min[axis] - eye[axis]
hi = bbox.max[axis] + 1.0 - eye[axis]
d = direction[axis]
if abs(d) < 1e-12:
if lo > 0.0 or hi < 0.0:
return None
continue
near, far = lo / d, hi / d
if near > far:
near, far = far, near
t0, t1 = max(t0, near), min(t1, far)
if t0 > t1:
return None
return t0, t1
def trace(sampler, eye, direction, t0, t1, voxel_size):
"""Sphere-trace from t0 to t1; returns (hit t, crossing seen) or None."""
t = t0
while t < t1:
pos = nanovdb.math.Vec3d(eye[0] + t * direction[0],
eye[1] + t * direction[1],
eye[2] + t * direction[2])
d = sampler(pos) # world-unit SDF, clamped to the narrow band
if d <= 0.0:
# zeroCrossing() reports whether the reconstruction stencil
# here straddles the iso-surface — the sampler-level analog
# of the HDDA ZeroCrossing test in the C++ example.
return t, sampler.zeroCrossing(pos)
# The clamped SDF bounds the distance to the surface, so it is
# a safe (index-space) step size; never step below half a voxel.
t += max(d / voxel_size, 0.5)
return None
def main():
handle = nanovdb.tools.createLevelSetSphere(radius=100.0, name="sphere")
grid = handle.grid()
bbox = grid.indexBBox()
voxel_size = grid.voxelSize()[0]
sampler = nanovdb.math.createTrilinearSampler(grid)
# Perspective camera looking down -z, as in the C++ RayGenOp,
# working directly in index space (the grid transform is uniform).
dim = [bbox.max[i] + 1 - bbox.min[i] for i in range(3)]
center = [bbox.min[i] + 0.5 * dim[i] for i in range(3)]
eye = nanovdb.math.Vec3d(center[0], center[1], center[2] + 2.0 * dim[2])
tan_fov = math.tan(math.radians(FOV_DEG) * 0.5)
pixels = bytearray(RES * RES)
crossings = 0
for y in range(RES):
for x in range(RES):
px = (2.0 * (x + 0.5) / RES - 1.0) * tan_fov
py = (2.0 * (y + 0.5) / RES - 1.0) * tan_fov
norm = math.sqrt(px * px + py * py + 1.0)
direction = (px / norm, py / norm, -1.0 / norm)
span = clip_to_bbox(eye, direction, bbox)
if span is None:
continue
hit = trace(sampler, eye, direction, span[0], span[1],
voxel_size)
if hit is None:
continue
t_hit, crossed = hit
crossings += crossed
pos = nanovdb.math.Vec3d(eye[0] + t_hit * direction[0],
eye[1] + t_hit * direction[1],
eye[2] + t_hit * direction[2])
# gradient() is in index-space units — normalize before use.
n = sampler.gradient(pos)
n.normalize()
shade = max(0.0, n[0] * LIGHT[0] + n[1] * LIGHT[1]
+ n[2] * LIGHT[2])
pixels[y * RES + x] = int(255 * shade)
out_dir = tempfile.mkdtemp(prefix="nanovdb_")
path = os.path.join(out_dir, "raytrace_level_set.pgm")
with open(path, "wb") as f:
f.write(f"P5\n{RES} {RES}\n255\n".encode("ascii"))
f.write(bytes(pixels))
lit = sum(1 for p in pixels if p > 0)
print(f"Rendered {RES}x{RES} image, {lit} lit pixels "
f"({crossings} confirmed zero-crossings) -> {path}")
assert lit > 0 and crossings > 0
if __name__ == "__main__":
main()
@@ -0,0 +1,61 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Validate a NanoVDB grid and round-trip its checksum.
nanovdb.tools exposes the grid-quality surface: validateGrid (single
grid), checkGrid -> (ok, error), isValid, plus the checksum helpers
evalChecksum / validateChecksum / updateChecksum. This example walks
through a typical "load and verify" workflow.
Run with: python validate.py
"""
import nanovdb
def main():
# A well-formed grid: createLevelSetSphere returns a NanoGrid<float>
# with stats and checksum populated to the defaults.
handle = nanovdb.tools.createLevelSetSphere(radius=10.0)
grid = handle.grid()
# Whole-handle validation. validateGrids returns a single bool.
# The verbose=True flag prints failure details to std::cerr
# (visible from Python via std::cerr -> sys.stderr on most stdlibs).
all_ok = nanovdb.tools.validateGrids(
handle, nanovdb.CheckMode.Default, verbose=False)
print(f"validateGrids(handle, Default) = {all_ok}")
# Per-grid validation, with a helpful message on failure.
one_ok = nanovdb.tools.validateGrid(handle, 0, nanovdb.CheckMode.Full)
print(f"validateGrid(handle, 0, Full) = {one_ok}")
# validateGrid returns False (no raise) on out-of-range gridID;
# CheckMode.Disable short-circuits and always returns True.
print(f"validateGrid(handle, 99) = "
f"{nanovdb.tools.validateGrid(handle, 99)} (out of range)")
print(f"validateGrid(handle, 99, Disable) = "
f"{nanovdb.tools.validateGrid(handle, 99, nanovdb.CheckMode.Disable)}"
f" (Disable short-circuit)")
# tools.checkGrid returns the structural check result plus a
# human-readable error message (empty on success).
ok, msg = nanovdb.tools.checkGrid(grid, nanovdb.CheckMode.Full)
print(f"checkGrid(grid, Full) = ok={ok}, msg={msg!r}")
# tools.isValid is checkGrid + checksum verification, returning
# one bool.
print(f"isValid(grid, Default) = "
f"{nanovdb.tools.isValid(grid, nanovdb.CheckMode.Default)}")
# Checksum round-trip. evalChecksum is non-mutating; updateChecksum
# writes back into the grid header.
cs1 = nanovdb.tools.evalChecksum(grid, nanovdb.CheckMode.Full)
nanovdb.tools.updateChecksum(grid, nanovdb.CheckMode.Full)
cs2 = nanovdb.tools.evalChecksum(grid, nanovdb.CheckMode.Full)
print(f"evalChecksum equal after no-op update: {cs1 == cs2}")
print(f"validateChecksum(grid, Full) = "
f"{nanovdb.tools.validateChecksum(grid, nanovdb.CheckMode.Full)}")
if __name__ == "__main__":
main()
+105
View File
@@ -0,0 +1,105 @@
#!/usr/bin/env python
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Smoke test: run every example script and require a clean exit.
The examples are self-contained and degrade gracefully (exit 0 with a
skip message) when an optional dependency such as NumPy or OpenVDB is
missing, so a non-zero exit or an uncaught exception always indicates
a real breakage — typically the bindings drifting under the examples.
The script list is deliberately explicit rather than a glob so that a
missing listed example fails loudly and unrelated files placed under
examples/ are not executed.
"""
import os
import subprocess
import sys
import tempfile
import unittest
EXAMPLES_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)),
os.pardir, "examples")
EXAMPLE_SCRIPTS = [
"build_grid.py",
"bulk_leaf_numpy.py",
"collide_level_set.py",
"index_grid_channels.py",
"io_roundtrip.py",
"load_inspect.py",
"make_funny_nanovdb.py",
"make_typed_grids.py",
"node_manager.py",
"openvdb_interop.py",
"quantize.py",
"raytrace_fog_volume.py",
"raytrace_level_set.py",
"validate.py",
]
class TestExamples(unittest.TestCase):
pass
def _make_test(script_name):
def test(self):
script = os.path.join(EXAMPLES_DIR, script_name)
self.assertTrue(os.path.isfile(script),
f"example script is missing: {script}")
command = [sys.executable, script]
env = os.environ.copy()
# Preserve output up to a native crash and ask Python to report the
# active frame for fatal signals and Windows exceptions.
env["PYTHONFAULTHANDLER"] = "1"
env["PYTHONUNBUFFERED"] = "1"
if hasattr(os, "add_dll_directory"):
# Match TestNanoVDB.py's in-tree OpenVDB DLL lookup. The child
# starts in a temporary directory, so resolve this while the
# parent is still in the CMake configuration directory.
config = os.path.basename(os.getcwd())
openvdb_dll_directory = os.path.abspath(os.path.join(
os.getcwd(), os.pardir, os.pardir, os.pardir, os.pardir,
"openvdb", "openvdb", config))
dll_directories = [openvdb_dll_directory]
cuda_root = env.get("CUDA_PATH")
if cuda_root:
dll_directories.append(os.path.join(cuda_root, "bin"))
env["NANOVDB_TEST_DLL_DIRECTORIES"] = os.pathsep.join(
dll_directories)
# Python imports sitecustomize during startup. Add this test
# directory to the child's module path so sitecustomize.py can
# register only the required OpenVDB and CUDA DLL directories
# before the example imports nanovdb, while still running the
# example as a normal script.
test_directory = os.path.dirname(os.path.abspath(__file__))
env["PYTHONPATH"] = os.pathsep.join(
(test_directory, env.get("PYTHONPATH", "")))
# A fresh cwd per run keeps any output files out of the source
# tree (the examples themselves also write to tempdirs).
result = subprocess.run(
command,
cwd=tempfile.mkdtemp(prefix="nanovdb_example_"),
env=env,
capture_output=True,
text=True,
timeout=120,
)
self.assertEqual(
result.returncode, 0,
f"{script_name} exited with {result.returncode}\n"
f"--- stdout ---\n{result.stdout}\n"
f"--- stderr ---\n{result.stderr}")
return test
for _name in EXAMPLE_SCRIPTS:
_test_name = "test_" + _name.replace(".py", "")
setattr(TestExamples, _test_name, _make_test(_name))
if __name__ == "__main__":
unittest.main()
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,19 @@
#!/usr/bin/env python
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
"""Configure dependent-DLL lookup for in-tree Windows Python tests."""
import os
# Keep these handles alive for the lifetime of the child Python process.
_dll_directory_handles = []
if hasattr(os, "add_dll_directory"):
for _path in os.environ.get(
"NANOVDB_TEST_DLL_DIRECTORIES", "").split(os.pathsep):
if os.path.isdir(_path):
try:
_dll_directory_handles.append(os.add_dll_directory(_path))
except OSError:
pass
+10 -1
View File
@@ -142,7 +142,16 @@ public:
~VoxelBlockManagerHandle() { this->reset(); }
/// @brief clear the buffer
void reset() { mFirstLeafID.clear(); mJumpMap.clear(); mBlockCount = 0; }
void reset() {
if constexpr (BufferHasDestroy<BufferT>::value) {
mFirstLeafID.destroy();
mJumpMap.destroy();
} else {
mFirstLeafID.clear();
mJumpMap.clear();
}
mBlockCount = 0;
}
/// @brief Returns a non-const pointer to the firstLeafID device-hosted data
///
+13 -4
View File
@@ -18,6 +18,7 @@
#define NANOVDB_TOOLS_CUDA_ADDBLINDDATA_CUH_HAS_BEEN_INCLUDED
#include <nanovdb/NanoVDB.h>
#include <nanovdb/cuda/Buffer.h>
#include <nanovdb/cuda/DeviceBuffer.h>
#include <nanovdb/GridHandle.h>
#include <nanovdb/util/cuda/Util.h>
@@ -34,6 +35,7 @@ namespace tools::cuda {// ============================================
/// @tparam BuildT Build type of the grid
/// @tparam BlindDataT Type of the blind data
/// @tparam BufferT Type of the buffer used for allocation
/// @tparam ResourceT Template type of optional resource used for internal temporary memory
/// @param d_grid Pointer to device grid
/// @param d_blindData Pointer to device blind data
/// @param valueCount number of values in the blind data
@@ -43,7 +45,7 @@ namespace tools::cuda {// ============================================
/// @param pool optional pool used for allocation
/// @param stream optional CUDA stream (defaults to CUDA stream 0)
/// @return GridHandle with blind data appended
template<typename BuildT, typename BlindDataT, typename BufferT = nanovdb::cuda::DeviceBuffer>
template<typename BuildT, typename BlindDataT, typename BufferT = nanovdb::cuda::DeviceBuffer, typename ResourceT = nanovdb::cuda::DeviceResource>
GridHandle<BufferT>
addBlindData(const NanoGrid<BuildT> *d_grid,
const BlindDataT *d_blindData,
@@ -60,11 +62,16 @@ addBlindData(const NanoGrid<BuildT> *d_grid,
// old grid old meta new meta old data new data
static_assert(BufferTraits<BufferT>::hasDeviceDual, "Expected BufferT to support device allocation");
static_assert(nanovdb::cuda::is_async_resource<ResourceT>::value,
"addBlindData allocates stream-ordered scratch and requires an AsyncResource");
// extract byte sizes of the grid, blind meta data and blind data
enum {GRID=0, META=1, DATA=2, CHECKSUM=3};
uint64_t tmp[4], *d_tmp;
cudaCheck(util::cuda::mallocAsync((void**)&d_tmp, 4*sizeof(uint64_t), stream));
using ScratchT = nanovdb::cuda::Buffer<uint64_t, nanovdb::cuda::ResourceRef<ResourceT>>;
auto &resource = nanovdb::cuda::default_resource<ResourceT>();
uint64_t tmp[4];
ScratchT tmpBuf(stream, nanovdb::cuda::ResourceRef<ResourceT>(resource), 4, nanovdb::cuda::noInit);
uint64_t *d_tmp = tmpBuf.data();
util::cuda::lambdaKernel<<<1, 1, 0, stream>>>(1, [=] __device__(size_t) {
if (auto count = d_grid->blindDataCount()) {
d_tmp[GRID] = util::PtrDiff(&d_grid->blindMetaData(0), d_grid);
@@ -110,13 +117,15 @@ addBlindData(const NanoGrid<BuildT> *d_grid,
// increment grid size and blind data counter in output grid
util::cuda::lambdaKernel<<<1, 1, 0, stream>>>(1, [=] __device__(size_t) {
auto &grid = *reinterpret_cast<NanoGrid<BuildT>*>(d_data);
grid.mGridIndex = 0u; // Possibly overwriting input; returned grid has batch size 1
grid.mGridCount = 1u;
grid.mBlindMetadataCount += 1;
grid.mBlindMetadataOffset = d_tmp[GRID];
auto *meta = util::PtrAdd<GridBlindMetaData>(d_data, grid.mBlindMetadataOffset);// points to first blind meta data
for (uint32_t i=0, n=grid.mBlindMetadataCount-1; i<n; ++i, ++meta) meta->mDataOffset += sizeof(GridBlindMetaData);
grid.mGridSize += sizeof(GridBlindMetaData) + meta->blindDataSize();// expansion with 32 byte alignment
}); cudaCheckError();
cudaCheck(util::cuda::freeAsync(d_tmp, stream));
tmpBuf.destroy(stream);
Checksum cs(tmp[CHECKSUM]);
cuda::updateChecksum(reinterpret_cast<GridData*>(d_data), cs.mode(), stream);
+27 -21
View File
@@ -30,9 +30,12 @@ namespace nanovdb {
namespace tools::cuda {
template <typename BuildT>
template <typename BuildT, typename ResourceT = nanovdb::cuda::DeviceResource>
class CoarsenGrid
{
static_assert(nanovdb::cuda::is_async_resource<ResourceT>::value,
"CoarsenGrid allocates stream-ordered scratch and requires an AsyncResource");
using GridT = NanoGrid<BuildT>;
using TreeT = NanoTree<BuildT>;
using RootT = NanoRoot<BuildT>;
@@ -43,8 +46,11 @@ public:
/// @brief Constructor
/// @param deviceGrid source device grid to be coarsened
/// @param stream optional CUDA stream (defaults to CUDA stream 0)
CoarsenGrid(const GridT* d_srcGrid, cudaStream_t stream = 0)
: mBuilder(stream), mStream(stream), mTimer(stream), mDeviceSrcGrid(d_srcGrid) {}
/// @param resource resource instance all device scratch is allocated from;
/// must outlive this operator (defaults to the per-type default resource)
CoarsenGrid(const GridT* d_srcGrid, cudaStream_t stream = 0,
ResourceT& resource = nanovdb::cuda::default_resource<ResourceT>())
: mBuilder(stream, resource), mStream(stream), mTimer(stream), mDeviceSrcGrid(d_srcGrid) {}
/// @brief Toggle on and off verbose mode
/// @param level Verbose level: 0=quiet, 1=timing, 2=benchmarking
@@ -74,20 +80,20 @@ private:
static constexpr unsigned int mNumThreads = 128;// for kernels spawned via lambdaKernel (others may specialize)
static unsigned int numBlocks(unsigned int n) {return (n + mNumThreads - 1) / mNumThreads;}
TopologyBuilder<BuildT> mBuilder;
TopologyBuilder<BuildT, ResourceT> mBuilder;
cudaStream_t mStream{0};
util::cuda::Timer mTimer;
int mVerbose{0};
const GridT *mDeviceSrcGrid;
TreeData mSrcTreeData;
};// tools::cuda::CoarsenGrid<BuildT>
};// tools::cuda::CoarsenGrid<BuildT, ResourceT>
//-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
template<typename BuildT>
template<typename BuildT, typename ResourceT>
template<typename BufferT>
GridHandle<BufferT>
CoarsenGrid<BuildT>::getHandle(const BufferT &pool)
CoarsenGrid<BuildT, ResourceT>::getHandle(const BufferT &pool)
{
// Copy TreeData from GPU -> CPU
cudaStreamSynchronize(mStream);
@@ -147,12 +153,12 @@ CoarsenGrid<BuildT>::getHandle(const BufferT &pool)
cudaStreamSynchronize(mStream);
return GridHandle<BufferT>(std::move(buffer));
}// CoarsenGrid<BuildT>::getHandle
}// CoarsenGrid<BuildT, ResourceT>::getHandle
//-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
template<typename BuildT>
void CoarsenGrid<BuildT>::coarsenRoot()
template<typename BuildT, typename ResourceT>
void CoarsenGrid<BuildT, ResourceT>::coarsenRoot()
{
// This method coarsens the root tiles, to accommodate for the overall downsamping operation.
@@ -197,12 +203,12 @@ void CoarsenGrid<BuildT>::coarsenRoot()
for (const auto& [key, tile] : coarsenedTiles)
*coarsenedRootPtr->tile(t++) = tile;
mBuilder.mProcessedRoot.deviceUpload(device, mStream, false);
}// CoarsenGrid<BuildT>::coarsenRoot
}// CoarsenGrid<BuildT, ResourceT>::coarsenRoot
//-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
template<typename BuildT>
void CoarsenGrid<BuildT>::coarsenInternalNodes()
template<typename BuildT, typename ResourceT>
void CoarsenGrid<BuildT, ResourceT>::coarsenInternalNodes()
{
// Computes the masks of upper and (densified) lower internal nodes, as a result of the coarsening operation
// Masks of lower internal nodes are densified in the sense that a serialized array of them is allocated,
@@ -210,26 +216,26 @@ void CoarsenGrid<BuildT>::coarsenInternalNodes()
if (auto srcLeafCount = mSrcTreeData.mNodeCount[0]) { // Unless it's an empty grid
util::cuda::lambdaKernel<<<numBlocks(srcLeafCount), mNumThreads, 0, mStream>>>(
srcLeafCount, util::morphology::cuda::CoarsenInternalNodesFunctor<BuildT>(),
mDeviceSrcGrid, mBuilder.deviceProcessedRoot(), mBuilder.mUpperMasks.deviceData(), mBuilder.mLowerMasks.deviceData() );
mDeviceSrcGrid, mBuilder.deviceProcessedRoot(), mBuilder.mUpperMasks.data(), mBuilder.mLowerMasks.data() );
}
}// CoarsenGrid<BuildT>::coarsenInternalNodes
}// CoarsenGrid<BuildT, ResourceT>::coarsenInternalNodes
//-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
template <typename BuildT>
void CoarsenGrid<BuildT>::processGridTreeRoot()
template <typename BuildT, typename ResourceT>
void CoarsenGrid<BuildT, ResourceT>::processGridTreeRoot()
{
// Copy GridData from source grid
// By convention: this will duplicate grid name and map. Others will be reset later
cudaCheck(cudaMemcpyAsync(&mBuilder.data()->getGrid(), mDeviceSrcGrid->data(), GridT::memUsage(), cudaMemcpyDeviceToDevice, mStream));
util::cuda::lambdaKernel<<<1, 1, 0, mStream>>>(1, topology::detail::BuildGridTreeRootFunctor<BuildT>(), mBuilder.deviceData());
cudaCheckError();
}// CoarsenGrid<BuildT>::processGridTreeRoot
}// CoarsenGrid<BuildT, ResourceT>::processGridTreeRoot
//-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
template<typename BuildT>
void CoarsenGrid<BuildT>::coarsenLeafNodes()
template<typename BuildT, typename ResourceT>
void CoarsenGrid<BuildT, ResourceT>::coarsenLeafNodes()
{
// Coarsens the active masks of the source grid (as indicated at the leaf level), into a new grid that
// has been already topologically coarsened to include all necessary leaf nodes.
@@ -241,7 +247,7 @@ void CoarsenGrid<BuildT>::coarsenLeafNodes()
// Update leaf offsets and prefix sums
mBuilder.processLeafOffsets(mStream);
}// CoarsenGrid<BuildT>::coarsenLeafNodes
}// CoarsenGrid<BuildT, ResourceT>::coarsenLeafNodes
//-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+27 -21
View File
@@ -30,9 +30,12 @@ namespace nanovdb {
namespace tools::cuda {
template <typename BuildT>
template <typename BuildT, typename ResourceT = nanovdb::cuda::DeviceResource>
class DilateGrid
{
static_assert(nanovdb::cuda::is_async_resource<ResourceT>::value,
"DilateGrid allocates stream-ordered scratch and requires an AsyncResource");
using GridT = NanoGrid<BuildT>;
using TreeT = NanoTree<BuildT>;
using RootT = NanoRoot<BuildT>;
@@ -43,8 +46,11 @@ public:
/// @brief Constructor
/// @param deviceGrid source device grid to be dilated
/// @param stream optional CUDA stream (defaults to CUDA stream 0)
DilateGrid(const GridT* d_srcGrid, cudaStream_t stream = 0)
: mBuilder(stream), mStream(stream), mTimer(stream), mDeviceSrcGrid(d_srcGrid) {}
/// @param resource resource instance all device scratch is allocated from;
/// must outlive this operator (defaults to the per-type default resource)
DilateGrid(const GridT* d_srcGrid, cudaStream_t stream = 0,
ResourceT& resource = nanovdb::cuda::default_resource<ResourceT>())
: mBuilder(stream, resource), mStream(stream), mTimer(stream), mDeviceSrcGrid(d_srcGrid) {}
/// @brief Toggle on and off verbose mode
/// @param level Verbose level: 0=quiet, 1=timing, 2=benchmarking
@@ -78,21 +84,21 @@ private:
static constexpr unsigned int mNumThreads = 128;// for kernels spawned via lambdaKernel (others may specialize)
static unsigned int numBlocks(unsigned int n) {return (n + mNumThreads - 1) / mNumThreads;}
TopologyBuilder<BuildT> mBuilder;
TopologyBuilder<BuildT, ResourceT> mBuilder;
cudaStream_t mStream{0};
util::cuda::Timer mTimer;
int mVerbose{0};
const GridT *mDeviceSrcGrid;
morphology::NearestNeighbors mOp{morphology::NN_FACE_EDGE_VERTEX};
TreeData mSrcTreeData;
};// tools::cuda::DilateGrid<BuildT>
};// tools::cuda::DilateGrid<BuildT, ResourceT>
//-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
template<typename BuildT>
template<typename BuildT, typename ResourceT>
template<typename BufferT>
GridHandle<BufferT>
DilateGrid<BuildT>::getHandle(const BufferT &pool)
DilateGrid<BuildT, ResourceT>::getHandle(const BufferT &pool)
{
// Copy TreeData from GPU -> CPU
cudaStreamSynchronize(mStream);
@@ -152,12 +158,12 @@ DilateGrid<BuildT>::getHandle(const BufferT &pool)
cudaStreamSynchronize(mStream);
return GridHandle<BufferT>(std::move(buffer));
}// DilateGrid<BuildT>::getHandle
}// DilateGrid<BuildT, ResourceT>::getHandle
//-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
template<typename BuildT>
void DilateGrid<BuildT>::dilateRoot()
template<typename BuildT, typename ResourceT>
void DilateGrid<BuildT, ResourceT>::dilateRoot()
{
// This method conservatively and speculatively dilates the root tiles, to accommodate
// any new root nodes that might be introduced by the dilation operation.
@@ -220,12 +226,12 @@ void DilateGrid<BuildT>::dilateRoot()
for (const auto& [key, tile] : dilatedTiles)
*dilatedRootPtr->tile(t++) = tile;
mBuilder.mProcessedRoot.deviceUpload(device, mStream, false);
}// DilateGrid<BuildT>::dilateRoot
}// DilateGrid<BuildT, ResourceT>::dilateRoot
//-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
template<typename BuildT>
void DilateGrid<BuildT>::dilateInternalNodes()
template<typename BuildT, typename ResourceT>
void DilateGrid<BuildT, ResourceT>::dilateInternalNodes()
{
// Computes the masks of upper and (densified) lower internal nodes, as a result of the dilation operation
// Masks of lower internal nodes are densified in the sense that a serialized array of them is allocated,
@@ -247,24 +253,24 @@ void DilateGrid<BuildT>::dilateInternalNodes()
<<<dim3(mSrcTreeData.mNodeCount[1],Op::SlicesPerLowerNode,1), Op::MaxThreadsPerBlock, 0, mStream>>>
(mDeviceSrcGrid, mBuilder.deviceProcessedRoot(), mBuilder.deviceUpperMasks(), mBuilder.deviceLowerMasks()); }
}
}// DilateGrid<BuildT>::dilateInternalNodes
}// DilateGrid<BuildT, ResourceT>::dilateInternalNodes
//-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
template <typename BuildT>
void DilateGrid<BuildT>::processGridTreeRoot()
template <typename BuildT, typename ResourceT>
void DilateGrid<BuildT, ResourceT>::processGridTreeRoot()
{
// Copy GridData from source grid
// By convention: this will duplicate grid name and map. Others will be reset later
cudaCheck(cudaMemcpyAsync(&mBuilder.data()->getGrid(), mDeviceSrcGrid->data(), GridT::memUsage(), cudaMemcpyDeviceToDevice, mStream));
util::cuda::lambdaKernel<<<1, 1, 0, mStream>>>(1, topology::detail::BuildGridTreeRootFunctor<BuildT>(), mBuilder.deviceData());
cudaCheckError();
}// DilateGrid<BuildT>::processGridTreeRoot
}// DilateGrid<BuildT, ResourceT>::processGridTreeRoot
//-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
template<typename BuildT>
void DilateGrid<BuildT>::dilateLeafNodes()
template<typename BuildT, typename ResourceT>
void DilateGrid<BuildT, ResourceT>::dilateLeafNodes()
{
// Dilates the active masks of the source grid (as indicated at the leaf level), into a new grid that
// has been already topologically dilated to include all necessary leaf nodes.
@@ -279,13 +285,13 @@ void DilateGrid<BuildT>::dilateLeafNodes()
else if (mOp == morphology::NN_FACE_EDGE_VERTEX) {
using Op = util::morphology::cuda::DilateLeafNodesFunctor<BuildT, morphology::NN_FACE_EDGE_VERTEX>;
util::cuda::operatorKernel<Op>
<<<dim3(mBuilder.data()->nodeCount[1],Op::SlicesPerLowerNode,1), Op::MaxThreadsPerBlock>>>
<<<dim3(mBuilder.data()->nodeCount[1],Op::SlicesPerLowerNode,1), Op::MaxThreadsPerBlock, 0, mStream>>>
(mDeviceSrcGrid, static_cast<GridT*>(mBuilder.data()->d_bufferPtr)); }
}
// Update leaf offsets and prefix sums
mBuilder.processLeafOffsets(mStream);
}// DilateGrid<BuildT>::dilateLeafNodes
}// DilateGrid<BuildT, ResourceT>::dilateLeafNodes
//-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
@@ -20,6 +20,9 @@
#include <nanovdb/cuda/UnifiedBuffer.h>
#include <nanovdb/tools/cuda/PointsToGrid.cuh>
#include <nanovdb/util/cuda/Util.h>
#include <algorithm>
#include <cuda/cmath>
namespace nanovdb {
@@ -37,112 +40,84 @@ private:
uint32_t* mNodeCounts;
};
/// @brief Indicator functor that returns 1 if the input value matches the member value, 0 otherwise
template <typename T, typename std::enable_if<std::is_integral<T>::value>::type* = nullptr>
struct EqualityIndicator
{
EqualityIndicator(const T* value) : mValue(value) {}
__hostdev__
T operator()(const T& x) const
{
return x == (*mValue);
}
private:
const T* mValue;
};
/// @brief Implements the merge path binary search algorithm in order to find the median across two sorted input key arrays
/// @brief Find a partition at an arbitrary diagonal in the conceptual merge of two sorted input arrays.
template<typename KeyIteratorIn>
__device__
void mergePath(KeyIteratorIn keys1, size_t keys1Count, KeyIteratorIn keys2, size_t keys2Count, ptrdiff_t* key1Intervals, ptrdiff_t* key2Intervals, int intervalIndex)
void mergePath(KeyIteratorIn keys1, size_t keys1Count, KeyIteratorIn keys2, size_t keys2Count, ptrdiff_t* key1Intervals, ptrdiff_t* key2Intervals, size_t combinedIndex)
{
using key_type = typename ::cuda::std::iterator_traits<KeyIteratorIn>::value_type;
const size_t combinedIndex = intervalIndex * (keys1Count + keys2Count) / 2;
size_t leftTop = combinedIndex > keys1Count ? keys1Count : combinedIndex;
size_t rightTop = combinedIndex > keys1Count ? combinedIndex - keys1Count : 0;
size_t leftBottom = rightTop;
key_type leftKey;
key_type rightKey;
while(true)
{
ptrdiff_t offset = (leftTop - leftBottom) / 2;
ptrdiff_t leftMid = leftTop - offset;
ptrdiff_t rightMid = rightTop + offset;
if (leftMid > keys1Count - 1 || rightMid < 1) {
leftKey = 1;
rightKey = 0;
}
else {
leftKey = *(keys1 + leftMid);
rightKey = *(keys2 + rightMid - 1);
}
if (leftKey > rightKey) {
if (rightMid > keys2Count - 1 || leftMid < 1) {
leftKey = 0;
rightKey = 1;
}
else {
leftKey = *(keys1 + leftMid - 1);
rightKey = *(keys2 + rightMid);
}
if (leftKey <= rightKey) {
*key1Intervals = leftMid;
*key2Intervals = rightMid;
break;
}
else {
leftTop = leftMid - 1;
rightTop = rightMid + 1;
}
}
else {
leftBottom = leftMid + 1;
size_t begin = combinedIndex > keys2Count ? combinedIndex - keys2Count : 0;
size_t end = combinedIndex < keys1Count ? combinedIndex : keys1Count;
while (begin < end) {
const size_t key1Index = (begin + end) / 2;
const size_t key2Index = combinedIndex - 1 - key1Index;
if (!(keys2[key2Index] < keys1[key1Index])) {
begin = key1Index + 1;
} else {
end = key1Index;
}
}
*key1Intervals = begin;
*key2Intervals = combinedIndex - begin;
}
namespace kernels {
/// @brief Kernel wrapper for the merge path algorithm
/// @brief Kernel wrapper for the merge path algorithm.
template<typename KeyIteratorIn>
__global__
void mergePathKernel(KeyIteratorIn keys1, size_t keys1Count, KeyIteratorIn keys2, size_t keys2Count, ptrdiff_t* key1Intervals, ptrdiff_t* key2Intervals, size_t intervalOffset)
void mergePathKernel(KeyIteratorIn keys1, size_t keys1Count, KeyIteratorIn keys2, size_t keys2Count, ptrdiff_t* key1Intervals, ptrdiff_t* key2Intervals, size_t combinedIndex)
{
const unsigned int intervalIndex = threadIdx.x + blockIdx.x * blockDim.x + intervalOffset;
mergePath(keys1, keys1Count, keys2, keys2Count, key1Intervals, key2Intervals, intervalIndex);
mergePath(keys1, keys1Count, keys2, keys2Count, key1Intervals, key2Intervals, combinedIndex);
}
/// @brief Extends or shortens the left end of an array interval
template<typename DistanceIteratorIn, typename CountIteratorOut, typename OffsetIteratorOut>
/// @brief Snap each device boundary in stripeOffsets to the nearest edge of the run of equal
/// keys containing it, so that no run (i.e. upper-node tile) straddles two devices. The keys
/// must be globally sorted. The boundaries are adjusted monotonically from left to right, so a
/// run spanning several stripes is consolidated onto one device and the fully-interior devices
/// are left with empty stripes. Runs sequentially on a single thread: the boundary chain is a
/// sequential dependence over deviceCount entries and each run extent is found by binary search.
template<typename KeyT>
__global__
void leftRebalanceKernel(DistanceIteratorIn leftDistance, DistanceIteratorIn rightDistance, CountIteratorOut leftCount, OffsetIteratorOut leftOffset)
void snapBoundariesToRunsKernel(const KeyT* keys, ptrdiff_t keyCount, int deviceCount, ptrdiff_t* stripeOffsets, size_t* stripeCounts)
{
if (*leftDistance < *rightDistance) {
*leftCount -= *leftDistance;
ptrdiff_t previousBoundary = stripeOffsets[0]; // device 0 always starts at 0
for (int deviceId = 1; deviceId < deviceCount; ++deviceId) {
ptrdiff_t boundary = stripeOffsets[deviceId];
if (boundary >= keyCount) {
boundary = keyCount;
} else if (boundary > previousBoundary && keys[boundary] == keys[boundary - 1]) {
// The even-split boundary falls inside a run; binary search for the run's extent
// and snap to whichever end keeps the boundary closest to the even split without
// crossing the previous boundary.
const KeyT key = keys[boundary];
ptrdiff_t lo = previousBoundary, hi = boundary;
while (lo < hi) {
const ptrdiff_t mid = lo + (hi - lo) / 2;
if (keys[mid] < key) lo = mid + 1; else hi = mid;
}
const ptrdiff_t runStart = lo;
lo = boundary; hi = keyCount;
while (lo < hi) {
const ptrdiff_t mid = lo + (hi - lo) / 2;
if (keys[mid] <= key) lo = mid + 1; else hi = mid;
}
const ptrdiff_t runEnd = lo;
if (runStart <= previousBoundary) {
boundary = runEnd; // the run reaches the previous device, give the whole tile away
} else {
boundary = (boundary - runStart <= runEnd - boundary) ? runStart : runEnd;
}
}
if (boundary < previousBoundary) boundary = previousBoundary;
stripeOffsets[deviceId] = boundary;
previousBoundary = boundary;
}
else {
*leftCount += *rightDistance;
}
}
/// @brief Extends or shortens the right end of an array interval
template<typename DistanceIteratorIn, typename CountIteratorOut, typename OffsetIteratorOut>
__global__
void rightRebalanceKernel(DistanceIteratorIn leftDistance, DistanceIteratorIn rightDistance, CountIteratorOut rightCount, OffsetIteratorOut rightOffset)
{
if (*leftDistance < *rightDistance) {
*rightCount += *leftDistance;
*rightOffset -= *leftDistance;
}
else {
*rightCount -= *rightDistance;
*rightOffset += *rightDistance;
// Recompute the per-device counts from the adjusted, monotonic offsets.
for (int deviceId = 0; deviceId < deviceCount; ++deviceId) {
const ptrdiff_t nextOffset = (deviceId + 1 < deviceCount) ? stripeOffsets[deviceId + 1] : keyCount;
stripeCounts[deviceId] = static_cast<size_t>(nextOffset - stripeOffsets[deviceId]);
}
}
@@ -163,135 +138,183 @@ void rightRebalanceKernel(DistanceIteratorIn leftDistance, DistanceIteratorIn ri
#endif// ifdef _WIN32
#endif// ifndef CUB_LAUNCH
/// @brief Make every device stream wait for work submitted to all device streams.
inline void mergeStreams(const nanovdb::cuda::DeviceMesh& deviceMesh, cudaEvent_t* events)
{
static constexpr int mergeDeviceId = 0;
// Record an event for each device in its respective stream.
for (const auto& [deviceId, stream] : deviceMesh) {
cudaCheck(cudaSetDevice(deviceId));
cudaCheck(cudaEventRecord(events[deviceId], stream));
}
// Fan in the per-device events on the merge stream.
cudaCheck(cudaSetDevice(mergeDeviceId));
const auto mergeStream = deviceMesh[mergeDeviceId].stream;
for (const auto& deviceNode : deviceMesh) {
cudaCheck(cudaStreamWaitEvent(mergeStream, events[deviceNode.id]));
}
cudaCheck(cudaEventRecord(events[mergeDeviceId], mergeStream));
// Fan the merged dependency back out to every device stream.
for (const auto& [deviceId, stream] : deviceMesh) {
cudaCheck(cudaSetDevice(deviceId));
cudaCheck(cudaStreamWaitEvent(stream, events[mergeDeviceId]));
}
}
template<typename KeyT, typename ValueT, typename NumItemsT, typename OffsetT, typename CountT>
void radixSortAsync(const nanovdb::cuda::DeviceMesh& deviceMesh, nanovdb::cuda::TempDevicePool* pools, KeyT* keysIn, KeyT* keysOut, ValueT* valuesIn, ValueT* valuesOut, NumItemsT numItems, OffsetT* mergeIntervals, const OffsetT* offsets, const CountT* counts, cudaEvent_t* preEvents, cudaEvent_t* postEvents)
{
// Radix sort the subset of keys assigned to each device in parallel
if (!numItems) return;
const int deviceCount = static_cast<int>(deviceMesh.deviceCount());
OffsetT* leftIntervals = mergeIntervals;
OffsetT* rightIntervals = mergeIntervals + deviceCount + 1;
const auto offset = [&](int deviceIndex) {
return deviceIndex == deviceCount ? static_cast<OffsetT>(numItems) : offsets[deviceIndex];
};
// Radix sort the subset of key/value pairs assigned to each GPU.
for (const auto& [deviceId, stream] : deviceMesh) {
cudaCheck(cudaSetDevice(deviceId));
cudaCheck(cudaEventSynchronize(preEvents[deviceId]));
if (!counts[deviceId]) {
cudaCheck(cudaEventRecord(postEvents[deviceId], stream));
continue;
}
const KeyT* deviceKeysIn = keysIn + offsets[deviceId];
const ValueT* deviceValuesIn = valuesIn + offsets[deviceId];
KeyT* deviceKeysOut = keysOut + offsets[deviceId];
ValueT* deviceValuesOut = valuesOut + offsets[deviceId];
cudaCheck(util::cuda::memPrefetchAsync(deviceKeysIn, counts[deviceId] * sizeof(KeyT), deviceId, stream));
cudaCheck(util::cuda::memPrefetchAsync(deviceValuesIn, counts[deviceId] * sizeof(ValueT), deviceId, stream));
cudaCheck(util::cuda::memPrefetchAsync(deviceKeysOut, counts[deviceId] * sizeof(KeyT), deviceId, stream));
cudaCheck(util::cuda::memPrefetchAsync(deviceValuesOut, counts[deviceId] * sizeof(ValueT), deviceId, stream));
// TODO: Add begin and end bit support
CUB_LAUNCH(DeviceRadixSort::SortPairs, pools[deviceId], stream, deviceKeysIn, deviceKeysOut, deviceValuesIn, deviceValuesOut, counts[deviceId], 0, sizeof(KeyT) * 8);
cudaCheck(cudaEventRecord(postEvents[deviceId], stream));
}
mergeStreams(deviceMesh, postEvents);
// TODO: Generalize to numbers of GPUs that aren't powers of two
// For each pair of devices, merge the local sorts by first computing the median across the two devices followed by merging
// the elements less than and greater than/equal to the median onto the first and second device of the pair respectively.
// This avoids the allocating memory for and gathering the values from both devices onto a single device.
const int log2DeviceCount = log2(deviceMesh.deviceCount());
OffsetT* leftIntervals = mergeIntervals;
OffsetT* rightIntervals = mergeIntervals + deviceMesh.deviceCount();
for (int deviceExponent = 0; deviceExponent < log2DeviceCount; ++deviceExponent) {
std::swap(keysIn, keysOut);
std::swap(valuesIn, valuesOut);
const int deviceIncrement = 1 << deviceExponent;
KeyT* currentKeys = keysOut;
KeyT* nextKeys = keysIn;
ValueT* currentValues = valuesOut;
ValueT* nextValues = valuesIn;
std::vector<std::thread> threads;
for (int leftDeviceId = 0; leftDeviceId < static_cast<int>(deviceMesh.deviceCount()); leftDeviceId += 2 * deviceIncrement) {
threads.emplace_back([&, leftDeviceId]() {
const int rightDeviceId = leftDeviceId + deviceIncrement;
// At each level, split every pair of sorted runs at all of the group's output-shard boundaries.
// Every GPU in the group merges one balanced output shard, including at the upper merge-tree levels.
for (int runDeviceCount = 1; runDeviceCount < deviceCount; runDeviceCount *= 2) {
for (int groupStart = 0; groupStart < deviceCount; groupStart += 2 * runDeviceCount) {
const int groupMiddle = std::min(groupStart + runDeviceCount, deviceCount);
const int groupEnd = std::min(groupStart + 2 * runDeviceCount, deviceCount);
if (groupMiddle == groupEnd) continue;
CountT leftDeviceItemCount = 0;
for (int deviceId = leftDeviceId; deviceId < rightDeviceId; ++deviceId)
leftDeviceItemCount += counts[deviceId];
const CountT leftCount = offset(groupMiddle) - offset(groupStart);
const CountT rightCount = offset(groupEnd) - offset(groupMiddle);
const KeyT* leftKeys = currentKeys + offset(groupStart);
const KeyT* rightKeys = currentKeys + offset(groupMiddle);
CountT rightDeviceItemCount = 0;
for (int deviceId = rightDeviceId; deviceId < rightDeviceId + deviceIncrement; ++deviceId)
rightDeviceItemCount += counts[deviceId];
const KeyT* leftDeviceKeysIn = keysIn + offsets[leftDeviceId];
const ValueT* leftDeviceValuesIn = valuesIn + offsets[leftDeviceId];
const KeyT* rightDeviceKeysIn = keysIn + offsets[leftDeviceId] + leftDeviceItemCount;
const ValueT* rightDeviceValuesIn = valuesIn + offsets[leftDeviceId] + leftDeviceItemCount;
// Wait on the prior sort to finish on both devices before computing the median across both devices
auto mergePathSubfunc = [&](int deviceId, int otherDeviceId, int intervalIndex) {
cudaCheck(cudaSetDevice(deviceId));
cudaCheck(cudaStreamWaitEvent(deviceMesh[deviceId].stream, postEvents[otherDeviceId]));
kernels::mergePathKernel<<<1, 1, 0, deviceMesh[deviceId].stream>>>(leftDeviceKeysIn, leftDeviceItemCount, rightDeviceKeysIn, rightDeviceItemCount, leftIntervals + deviceId, rightIntervals + deviceId, intervalIndex);
cudaCheck(cudaEventRecord(postEvents[deviceId], deviceMesh[deviceId].stream));
};
mergePathSubfunc(leftDeviceId, rightDeviceId, 0);
mergePathSubfunc(rightDeviceId, leftDeviceId, 1);
cudaCheck(cudaEventSynchronize(postEvents[leftDeviceId]));
cudaCheck(cudaEventSynchronize(postEvents[rightDeviceId]));
// Merge the pairs less than the median to the left device
{
cudaCheck(cudaSetDevice(leftDeviceId));
const KeyT* leftKeysIn = leftDeviceKeysIn + leftIntervals[leftDeviceId];
const ValueT* leftValuesIn = leftDeviceValuesIn + leftIntervals[leftDeviceId];
CountT leftCount = leftIntervals[rightDeviceId] - leftIntervals[leftDeviceId];
const KeyT* rightKeysIn = rightDeviceKeysIn + rightIntervals[leftDeviceId];
const ValueT* rightValuesIn = rightDeviceValuesIn + rightIntervals[leftDeviceId];
CountT rightCount = rightIntervals[rightDeviceId] - rightIntervals[leftDeviceId];
OffsetT outputOffset = offsets[leftDeviceId] + leftIntervals[leftDeviceId] + rightIntervals[leftDeviceId];
CUB_LAUNCH(DeviceMerge::MergePairs, pools[leftDeviceId], deviceMesh[leftDeviceId].stream, leftKeysIn, leftValuesIn, leftCount, rightKeysIn, rightValuesIn, rightCount, keysOut + outputOffset, valuesOut + outputOffset, {});
cudaCheck(cudaEventRecord(postEvents[leftDeviceId], deviceMesh[leftDeviceId].stream));
};
// Merge the pairs greater than/equal to the median to the right device
{
cudaCheck(cudaSetDevice(rightDeviceId));
const KeyT* leftKeysIn = leftDeviceKeysIn + leftIntervals[rightDeviceId];
const ValueT* leftValuesIn = leftDeviceValuesIn + leftIntervals[rightDeviceId];
CountT leftCount = leftDeviceItemCount - leftIntervals[rightDeviceId];
const KeyT* rightKeysIn = rightDeviceKeysIn + rightIntervals[rightDeviceId];
const ValueT* rightValuesIn = rightDeviceValuesIn + rightIntervals[rightDeviceId];
CountT rightCount = rightDeviceItemCount - rightIntervals[rightDeviceId];
OffsetT outputOffset = offsets[leftDeviceId] + leftIntervals[rightDeviceId] + rightIntervals[rightDeviceId];
CUB_LAUNCH(DeviceMerge::MergePairs, pools[rightDeviceId], deviceMesh[rightDeviceId].stream, leftKeysIn, leftValuesIn, leftCount, rightKeysIn, rightValuesIn, rightCount, keysOut + outputOffset, valuesOut + outputOffset, {});
cudaCheck(cudaEventRecord(postEvents[rightDeviceId], deviceMesh[rightDeviceId].stream));
};
cudaCheck(cudaEventSynchronize(postEvents[leftDeviceId]));
cudaCheck(cudaEventSynchronize(postEvents[rightDeviceId]));
});
for (int boundaryDevice = groupStart + 1; boundaryDevice < groupEnd; ++boundaryDevice) {
cudaCheck(cudaSetDevice(boundaryDevice));
const auto stream = deviceMesh[boundaryDevice].stream;
const CountT diagonal = offset(boundaryDevice) - offset(groupStart);
kernels::mergePathKernel<<<1, 1, 0, stream>>>(leftKeys, leftCount, rightKeys, rightCount, leftIntervals + boundaryDevice, rightIntervals + boundaryDevice, diagonal);
cudaCheckError();
cudaCheck(cudaEventRecord(postEvents[boundaryDevice], stream));
}
}
std::for_each(threads.begin(), threads.end(), [](std::thread& t) { t.join(); });
// CUB needs the partition sizes on the host. Synchronize only the tiny merge-path kernels.
for (int groupStart = 0; groupStart < deviceCount; groupStart += 2 * runDeviceCount) {
const int groupMiddle = std::min(groupStart + runDeviceCount, deviceCount);
const int groupEnd = std::min(groupStart + 2 * runDeviceCount, deviceCount);
if (groupMiddle == groupEnd) continue;
for (int boundaryDevice = groupStart + 1; boundaryDevice < groupEnd; ++boundaryDevice) {
cudaCheck(cudaEventSynchronize(postEvents[boundaryDevice]));
}
}
for (int groupStart = 0; groupStart < deviceCount; groupStart += 2 * runDeviceCount) {
const int groupMiddle = std::min(groupStart + runDeviceCount, deviceCount);
const int groupEnd = std::min(groupStart + 2 * runDeviceCount, deviceCount);
// A trailing run without a merge partner must still be copied because the
// current and next buffers are swapped after every merge-tree level.
if (groupMiddle == groupEnd) {
for (int outputDevice = groupStart; outputDevice < groupEnd; ++outputDevice) {
cudaCheck(cudaSetDevice(outputDevice));
const auto stream = deviceMesh[outputDevice].stream;
if (counts[outputDevice]) {
cudaCheck(cudaMemcpyAsync(nextKeys + offset(outputDevice), currentKeys + offset(outputDevice), counts[outputDevice] * sizeof(KeyT), cudaMemcpyDefault, stream));
cudaCheck(cudaMemcpyAsync(nextValues + offset(outputDevice), currentValues + offset(outputDevice), counts[outputDevice] * sizeof(ValueT), cudaMemcpyDefault, stream));
}
cudaCheck(cudaEventRecord(postEvents[outputDevice], stream));
}
continue;
}
const CountT leftCount = offset(groupMiddle) - offset(groupStart);
const CountT rightCount = offset(groupEnd) - offset(groupMiddle);
const KeyT* leftKeys = currentKeys + offset(groupStart);
const KeyT* rightKeys = currentKeys + offset(groupMiddle);
const ValueT* leftValues = currentValues + offset(groupStart);
const ValueT* rightValues = currentValues + offset(groupMiddle);
for (int outputDevice = groupStart; outputDevice < groupEnd; ++outputDevice) {
cudaCheck(cudaSetDevice(outputDevice));
const auto stream = deviceMesh[outputDevice].stream;
const CountT leftBegin = outputDevice == groupStart ? 0 : leftIntervals[outputDevice];
const CountT leftEnd = outputDevice + 1 == groupEnd ? leftCount : leftIntervals[outputDevice + 1];
const CountT rightBegin = outputDevice == groupStart ? 0 : rightIntervals[outputDevice];
const CountT rightEnd = outputDevice + 1 == groupEnd ? rightCount : rightIntervals[outputDevice + 1];
const CountT leftSegmentCount = leftEnd - leftBegin;
const CountT rightSegmentCount = rightEnd - rightBegin;
const CountT outputCount = leftSegmentCount + rightSegmentCount;
NANOVDB_ASSERT(outputCount == counts[outputDevice]);
if (outputCount) {
CUB_LAUNCH(DeviceMerge::MergePairs, pools[outputDevice], stream,
leftKeys + leftBegin, leftValues + leftBegin, leftSegmentCount,
rightKeys + rightBegin, rightValues + rightBegin, rightSegmentCount,
nextKeys + offset(outputDevice), nextValues + offset(outputDevice), {});
}
cudaCheck(cudaEventRecord(postEvents[outputDevice], stream));
}
}
mergeStreams(deviceMesh, postEvents);
std::swap(currentKeys, nextKeys);
std::swap(currentValues, nextValues);
}
// There is no merging required for a single device so we simply copy the sorted result to the destination array (where the sort would have been merged to).
if (log2DeviceCount % 2) {
std::swap(keysIn, keysOut);
std::swap(valuesIn, valuesOut);
// Keep the existing contract: callers always receive the final data in the output buffers.
if (currentKeys != keysOut) {
for (const auto& [deviceId, stream] : deviceMesh) {
cudaCheck(cudaSetDevice(deviceId));
cudaMemcpyAsync(keysOut + offsets[deviceId], keysIn + offsets[deviceId], counts[deviceId] * sizeof(KeyT), cudaMemcpyDefault, stream);
cudaMemcpyAsync(valuesOut + offsets[deviceId], valuesIn + offsets[deviceId], counts[deviceId] * sizeof(ValueT), cudaMemcpyDefault, stream);
cudaEventRecord(postEvents[deviceId], stream);
if (counts[deviceId]) {
cudaCheck(cudaMemcpyAsync(keysOut + offsets[deviceId], currentKeys + offsets[deviceId], counts[deviceId] * sizeof(KeyT), cudaMemcpyDefault, stream));
cudaCheck(cudaMemcpyAsync(valuesOut + offsets[deviceId], currentValues + offsets[deviceId], counts[deviceId] * sizeof(ValueT), cudaMemcpyDefault, stream));
}
cudaCheck(cudaEventRecord(postEvents[deviceId], stream));
}
}
mergeStreams(deviceMesh, postEvents);
}
template<typename KeyT, typename ValueT, typename NumItemsT, typename OffsetT, typename CountT>
void radixSortAsync(const nanovdb::cuda::DeviceMesh& deviceMesh, nanovdb::cuda::TempDevicePool* pools, KeyT* keysIn, KeyT* keysOut, ValueT* valuesIn, ValueT* valuesOut, NumItemsT numItems, const OffsetT* offsets, const CountT* counts, cudaEvent_t* preEvents, cudaEvent_t* postEvents)
{
ptrdiff_t* mergeIntervals = nullptr;
cudaCheck(cudaMallocManaged(&mergeIntervals, 2 * deviceMesh.deviceCount() * sizeof(ptrdiff_t)));
cudaCheck(cudaMallocHost(&mergeIntervals, 2 * (deviceMesh.deviceCount() + 1) * sizeof(ptrdiff_t)));
radixSortAsync(deviceMesh, pools, keysIn, keysOut, valuesIn, valuesOut, numItems, mergeIntervals, offsets, counts, preEvents, postEvents);
cudaCheck(cudaFree(mergeIntervals));
cudaCheck(cudaFreeHost(mergeIntervals));
}
/// @brief Launches an async exclusive sum operation across multiple devices. The operator waits on the per-device preEvents[deviceId] before summing over that device's contributions and records postEvents[deviceId] when the device's contribution is summed.
@@ -306,10 +329,12 @@ void exclusiveSumAsync(const nanovdb::cuda::DeviceMesh& deviceMesh, nanovdb::cud
// Required for the host to pass the correct value of counts[deviceId]
cudaCheck(cudaEventSynchronize(preEvents[deviceId]));
uint32_t deviceNumItems = counts[deviceId];
CUB_LAUNCH(DeviceScan::ExclusiveSum, pools[deviceId], stream, deviceIn, deviceOut, deviceNumItems);
if (deviceNumItems) {
CUB_LAUNCH(DeviceScan::ExclusiveSum, pools[deviceId], stream, deviceIn, deviceOut, deviceNumItems);
}
cudaCheck(cudaEventRecord(preEvents[deviceId], stream));
deviceIn += counts[deviceId];
deviceOut += counts[deviceId];
deviceIn += deviceNumItems;
deviceOut += deviceNumItems;
}
deviceIn = in;
@@ -323,7 +348,7 @@ void exclusiveSumAsync(const nanovdb::cuda::DeviceMesh& deviceMesh, nanovdb::cud
if (counts[deviceId]) {
auto segmentExclusiveSum = deviceOut[counts[deviceId] - 1] + deviceIn[counts[deviceId] - 1];
unsigned int numBlocks = (counts[deviceId] + NumThreads - 1) / NumThreads;
unsigned int numBlocks = ::cuda::ceil_div<int>(counts[deviceId], NumThreads);
util::cuda::lambdaKernel<<<numBlocks, NumThreads, 0, stream>>>(counts[deviceId], [=] __device__ (size_t tid) { deviceOut[tid] += partialExclusiveSum; });
cudaCheckError();
@@ -347,10 +372,12 @@ void inclusiveSumAsync(const nanovdb::cuda::DeviceMesh& deviceMesh, nanovdb::cud
// Required for the host to pass the correct value of counts[deviceId]
cudaCheck(cudaEventSynchronize(preEvents[deviceId]));
uint32_t deviceNumItems = counts[deviceId];
CUB_LAUNCH(DeviceScan::InclusiveSum, pools[deviceId], stream, deviceIn, deviceOut, deviceNumItems);
if (deviceNumItems) {
CUB_LAUNCH(DeviceScan::InclusiveSum, pools[deviceId], stream, deviceIn, deviceOut, deviceNumItems);
}
cudaCheck(cudaEventRecord(preEvents[deviceId], stream));
deviceIn += counts[deviceId];
deviceOut += counts[deviceId];
deviceIn += deviceNumItems;
deviceOut += deviceNumItems;
}
deviceIn = in;
@@ -364,7 +391,7 @@ void inclusiveSumAsync(const nanovdb::cuda::DeviceMesh& deviceMesh, nanovdb::cud
if (counts[deviceId]) {
auto segmentInclusiveSum = deviceOut[counts[deviceId] - 1];
unsigned int numBlocks = (counts[deviceId] + NumThreads - 1) / NumThreads;
unsigned int numBlocks = ::cuda::ceil_div<int>(counts[deviceId], NumThreads);
util::cuda::lambdaKernel<<<numBlocks, NumThreads, 0, stream>>>(counts[deviceId], [=] __device__ (size_t tid) { deviceOut[tid] += partialInclusiveSum; });
cudaCheckError();
@@ -425,7 +452,7 @@ public:
private:
static constexpr unsigned int mNumThreads = 128;
static unsigned int numBlocks(unsigned int n) {return (n + mNumThreads - 1) / mNumThreads;}
static unsigned int numBlocks(unsigned int n) {return ::cuda::ceil_div(n, mNumThreads);}
uint32_t* deviceNodeCount(int deviceId) const { return mNodeCounts + 3 * deviceId; }
@@ -476,7 +503,7 @@ DistributedPointsToGrid<BuildT>::DistributedPointsToGrid(const nanovdb::cuda::De
mVoxelOffsets = nullptr;
cudaCheck(cudaMallocManaged(&mVoxelOffsets, mDeviceMesh.deviceCount() * sizeof(uint32_t)));
mIntervals = nullptr;
cudaCheck(cudaMallocManaged(&mIntervals, 2 * mDeviceMesh.deviceCount() * sizeof(ptrdiff_t)));
cudaCheck(cudaMallocHost(&mIntervals, 2 * (mDeviceMesh.deviceCount() + 1) * sizeof(ptrdiff_t)));
}
template <typename BuildT>
@@ -488,7 +515,7 @@ DistributedPointsToGrid<BuildT>::DistributedPointsToGrid(const nanovdb::cuda::De
template <typename BuildT>
DistributedPointsToGrid<BuildT>::~DistributedPointsToGrid()
{
cudaCheck(cudaFree(mIntervals));
cudaCheck(cudaFreeHost(mIntervals));
cudaCheck(cudaFree(mVoxelOffsets));
cudaCheck(cudaFree(mVoxelCounts));
cudaCheck(cudaFree(mNodeOffsets));
@@ -560,8 +587,6 @@ void DistributedPointsToGrid<BuildT>::countNodes(const PtrT coords, size_t coord
// to reduce overhead.
std::vector<cudaEvent_t> sortEvents(mDeviceMesh.deviceCount());
std::vector<cudaEvent_t> runLengthEncodeEvents(mDeviceMesh.deviceCount());
std::vector<cudaEvent_t> transformReduceEvents(mDeviceMesh.deviceCount());
std::vector<cudaEvent_t> rebalanceEvents(mDeviceMesh.deviceCount());
std::vector<cudaEvent_t> tilePrefixSumEvents(mDeviceMesh.deviceCount());
std::vector<cudaEvent_t> voxelCountEvents(mDeviceMesh.deviceCount());
std::vector<cudaEvent_t> leafCountEvents(mDeviceMesh.deviceCount());
@@ -572,8 +597,6 @@ void DistributedPointsToGrid<BuildT>::countNodes(const PtrT coords, size_t coord
cudaCheck(cudaSetDevice(deviceId));
cudaEventCreateWithFlags(&sortEvents[deviceId], cudaEventDisableTiming);
cudaEventCreateWithFlags(&runLengthEncodeEvents[deviceId], cudaEventDisableTiming);
cudaEventCreateWithFlags(&transformReduceEvents[deviceId], cudaEventDisableTiming);
cudaEventCreateWithFlags(&rebalanceEvents[deviceId], cudaEventDisableTiming);
cudaEventCreateWithFlags(&tilePrefixSumEvents[deviceId], cudaEventDisableTiming);
cudaEventCreateWithFlags(&voxelCountEvents[deviceId], cudaEventDisableTiming);
cudaEventCreateWithFlags(&leafCountEvents[deviceId], cudaEventDisableTiming);
@@ -582,33 +605,32 @@ void DistributedPointsToGrid<BuildT>::countNodes(const PtrT coords, size_t coord
cudaEventCreateWithFlags(&leafPrefixSumEvents[deviceId], cudaEventDisableTiming);
}
// Advise per-coord quantities to be split even across devices
// Advise per-coord quantities to be split evenly across devices. Clamp each stripe to
// the input range so that inputs smaller than the device count produce valid trailing empty stripes.
const size_t deviceStripeSize = ::cuda::ceil_div(coordCount, mDeviceMesh.deviceCount());
for (const auto& [deviceId, stream] : mDeviceMesh) {
cudaCheck(cudaSetDevice(deviceId));
size_t deviceStripeCount = (coordCount + mDeviceMesh.deviceCount() - 1) / mDeviceMesh.deviceCount();
const ptrdiff_t deviceStripeOffset = deviceStripeCount * deviceId;
deviceStripeCount = std::min(deviceStripeCount, coordCount - deviceStripeOffset);
const ptrdiff_t deviceStripeOffset = std::min(deviceStripeSize * deviceId, coordCount);
const size_t deviceStripeCount = std::min(deviceStripeSize, coordCount - deviceStripeOffset);
mStripeCounts[deviceId] = deviceStripeCount;
mStripeOffsets[deviceId] = deviceStripeOffset;
nanovdb::Coord* deviceCoords = coords + deviceStripeOffset;
uint64_t* deviceInputKeys = mKeys + deviceStripeOffset;
uint32_t* deviceInputIndices = mIndices + deviceStripeOffset;
uint64_t* deviceOutputKeys = mData->d_keys + deviceStripeOffset;
uint32_t* deviceOutputIndices = mData->d_indx + deviceStripeOffset;
if (deviceStripeCount) {
uint64_t* deviceInputKeys = mKeys + deviceStripeOffset;
uint32_t* deviceInputIndices = mIndices + deviceStripeOffset;
uint64_t* deviceOutputKeys = mData->d_keys + deviceStripeOffset;
uint32_t* deviceOutputIndices = mData->d_indx + deviceStripeOffset;
util::cuda::memAdvise(deviceCoords, deviceStripeCount * sizeof(nanovdb::Coord), cudaMemAdviseSetPreferredLocation, deviceId);
util::cuda::memAdvise(deviceCoords, deviceStripeCount * sizeof(nanovdb::Coord), cudaMemAdviseSetReadMostly, deviceId);
util::cuda::memAdvise(deviceInputKeys, deviceStripeCount * sizeof(uint64_t), cudaMemAdviseSetPreferredLocation, deviceId);
util::cuda::memAdvise(deviceInputIndices, deviceStripeCount * sizeof(uint32_t), cudaMemAdviseSetPreferredLocation, deviceId);
util::cuda::memAdvise(deviceOutputKeys, deviceStripeCount * sizeof(uint64_t), cudaMemAdviseSetPreferredLocation, deviceId);
util::cuda::memAdvise(deviceOutputIndices, deviceStripeCount * sizeof(uint32_t), cudaMemAdviseSetPreferredLocation, deviceId);
util::cuda::memAdvise(deviceInputKeys, deviceStripeCount * sizeof(uint64_t), cudaMemAdviseSetPreferredLocation, deviceId);
util::cuda::memAdvise(deviceInputIndices, deviceStripeCount * sizeof(uint32_t), cudaMemAdviseSetPreferredLocation, deviceId);
util::cuda::memAdvise(deviceOutputKeys, deviceStripeCount * sizeof(uint64_t), cudaMemAdviseSetPreferredLocation, deviceId);
util::cuda::memAdvise(deviceOutputIndices, deviceStripeCount * sizeof(uint32_t), cudaMemAdviseSetPreferredLocation, deviceId);
uint32_t* devicePointsPerTile = mPointsPerTile + deviceStripeOffset;
util::cuda::memAdvise(devicePointsPerTile, deviceStripeCount * sizeof(uint32_t), cudaMemAdviseSetPreferredLocation, deviceId);
uint32_t* devicePointsPerTile = mPointsPerTile + deviceStripeOffset;
util::cuda::memAdvise(devicePointsPerTile, deviceStripeCount * sizeof(uint32_t), cudaMemAdviseSetPreferredLocation, deviceId);
}
util::cuda::memAdvise(deviceNodeCount(deviceId), 3 * sizeof(uint32_t), cudaMemAdviseSetPreferredLocation, deviceId);
}
@@ -619,67 +641,51 @@ void DistributedPointsToGrid<BuildT>::countNodes(const PtrT coords, size_t coord
auto deviceStripeCount = mStripeCounts[deviceId];
auto deviceStripeOffset = mStripeOffsets[deviceId];
util::cuda::memPrefetchAsync(coords, coordCount * sizeof(nanovdb::Coord), deviceId, stream);
util::cuda::offsetLambdaKernel<<<numBlocks(deviceStripeCount), mNumThreads, 0, stream>>>(deviceStripeCount, deviceStripeOffset, TileKeyFunctor<BuildT, PtrT>(), mData, coords, mData->d_keys, mData->d_indx);
if (deviceStripeCount) {
util::cuda::memPrefetchAsync(coords, coordCount * sizeof(nanovdb::Coord), deviceId, stream);
nanovdb::util::cuda::offsetLambdaKernel<<<numBlocks(deviceStripeCount), mNumThreads, 0, stream>>>(deviceStripeCount, deviceStripeOffset, TileKeyFunctor<BuildT, PtrT>(), mData, coords, mData->d_keys, mData->d_indx);
cudaCheckError();
}
}
radixSortAsync(mDeviceMesh, mTempDevicePools, mData->d_keys, mKeys, mData->d_indx, mIndices, coordCount, mIntervals, mStripeOffsets, mStripeCounts, sortEvents.data(), sortEvents.data());
// For each segment of sorted keys on each device, we count how many of the leftmost key occur past the left boundary of the segment. The same is done for the rightmost key with the right boundary of the segment.
auto leftIntervals = mIntervals;
auto rightIntervals = mIntervals + mDeviceMesh.deviceCount();
for (const auto& [deviceId, stream] : mDeviceMesh) {
cudaCheck(cudaSetDevice(deviceId));
auto deviceStripeCount = mStripeCounts[deviceId];
auto deviceStripeOffset = mStripeOffsets[deviceId];
uint64_t* deviceInputKeys = mKeys + deviceStripeOffset;
if (deviceId > 0) {
cudaStreamWaitEvent(stream, sortEvents[deviceId - 1]);
EqualityIndicator<uint64_t> indicator(deviceInputKeys - 1);
CUB_LAUNCH(DeviceReduce::TransformReduce, mTempDevicePools[deviceId], stream, deviceInputKeys, rightIntervals + deviceId, deviceStripeCount, ::cuda::std::plus(), indicator, 0);
}
else {
rightIntervals[deviceId] = 0;
}
if (deviceId < static_cast<int>(mDeviceMesh.deviceCount() - 1)) {
cudaStreamWaitEvent(stream, sortEvents[deviceId + 1]);
EqualityIndicator<uint64_t> indicator(deviceInputKeys + deviceStripeCount);
CUB_LAUNCH(DeviceReduce::TransformReduce, mTempDevicePools[deviceId], stream, deviceInputKeys, leftIntervals + deviceId, deviceStripeCount, ::cuda::std::plus(), indicator, 0);
}
else {
leftIntervals[deviceId] = 0;
}
cudaEventRecord(transformReduceEvents[deviceId], stream);
// Rebalance the device segments so that a device boundary always coincides
// with a change in key value. Because TileKeyFunctor assigns identical keys
// to every point that falls in the same upper-node "tile", this aligns the
// device ownership boundaries with tile boundaries. Downstream construction
// assumes each tile (and therefore each lower node, leaf node, and voxel) is
// owned by exactly one device; if a tile straddled a boundary, multiple
// devices would concurrently build the same leaf and race on its value mask.
//
// A single tile can span three or more devices (e.g. one dense leaf whose
// points are split evenly across the mesh). Adjusting only adjacent pairs of
// boundaries cannot consolidate such a tile because a fully-interior device
// lies entirely within it, so we compute the boundaries globally and
// monotonically. mKeys is globally sorted at this point, so a tile boundary
// is simply a position where mKeys changes. Snapping runs in a single-thread
// kernel on one device (every device stream is already ordered after the
// whole sort by radixSortAsync's final stream merge) with binary searches for
// the run extents; the host only waits on its completion event before reading
// back the small offset/count arrays, mirroring how the sort's merge-path
// partitions are synchronized. Fully-interior devices are left with empty
// stripes, which the rest of the pipeline already handles.
{
static constexpr int snapDeviceId = 0;
cudaCheck(cudaSetDevice(snapDeviceId));
const auto snapStream = mDeviceMesh[snapDeviceId].stream;
kernels::snapBoundariesToRunsKernel<<<1, 1, 0, snapStream>>>(mKeys, static_cast<ptrdiff_t>(coordCount), static_cast<int>(mDeviceMesh.deviceCount()), mStripeOffsets, mStripeCounts);
cudaCheckError();
cudaCheck(cudaEventRecord(sortEvents[snapDeviceId], snapStream));
// CUB needs the rebalanced partition sizes on the host. Synchronize only the tiny snap kernel.
cudaCheck(cudaEventSynchronize(sortEvents[snapDeviceId]));
}
// Rebalance the segments so that a device segment boundary also corresponds to a change in key value. Effectively, this aligns upper node boundaries with device ownership boundaries.
// Parallel RLE in order to obtain tiles. The device boundaries were finalized
// above before the host read them back, so no per-device rebalance event is needed.
for (const auto& [deviceId, stream] : mDeviceMesh) {
cudaCheck(cudaSetDevice(deviceId));
if (deviceId > 0)
{
cudaStreamWaitEvent(stream, transformReduceEvents[deviceId - 1]);
kernels::rightRebalanceKernel<<<1, 1, 0, stream>>>(leftIntervals + deviceId - 1, rightIntervals + deviceId, mStripeCounts + deviceId, mStripeOffsets + deviceId);
}
if (deviceId < static_cast<int>(mDeviceMesh.deviceCount() - 1))
{
cudaStreamWaitEvent(stream, transformReduceEvents[deviceId + 1]);
kernels::leftRebalanceKernel<<<1, 1, 0, stream>>>(leftIntervals + deviceId, rightIntervals + deviceId + 1, mStripeCounts + deviceId, mStripeOffsets + deviceId);
}
cudaEventRecord(rebalanceEvents[deviceId], stream);
}
// Parallel RLE in order to obtain tiles
for (const auto& [deviceId, stream] : mDeviceMesh) {
cudaCheck(cudaSetDevice(deviceId));
cudaCheck(cudaEventSynchronize(rebalanceEvents[deviceId]));
auto deviceStripeCount = mStripeCounts[deviceId];
auto deviceStripeOffset = mStripeOffsets[deviceId];
@@ -689,7 +695,11 @@ void DistributedPointsToGrid<BuildT>::countNodes(const PtrT coords, size_t coord
// util::cuda::memPrefetchAsync(deviceInputKeys, deviceStripeCount * sizeof(uint64_t), deviceId, stream);
CUB_LAUNCH(DeviceRunLengthEncode::Encode, mTempDevicePools[deviceId], stream, deviceInputKeys, deviceOutputKeys, devicePointsPerTile, deviceNodeCount(deviceId) + 2, deviceStripeCount);
if (deviceStripeCount) {
CUB_LAUNCH(DeviceRunLengthEncode::Encode, mTempDevicePools[deviceId], stream, deviceInputKeys, deviceOutputKeys, devicePointsPerTile, deviceNodeCount(deviceId) + 2, deviceStripeCount);
} else {
cudaCheck(cudaMemsetAsync(deviceNodeCount(deviceId) + 2, 0, sizeof(uint32_t), stream));
}
cudaCheck(cudaEventRecord(runLengthEncodeEvents[deviceId], stream));
}
@@ -699,7 +709,9 @@ void DistributedPointsToGrid<BuildT>::countNodes(const PtrT coords, size_t coord
cudaCheck(cudaEventSynchronize(runLengthEncodeEvents[deviceId]));
auto deviceStripeOffset = mStripeOffsets[deviceId];
uint64_t* deviceKeys = mData->d_keys + deviceStripeOffset;
cudaCheck(cudaMemcpyAsync(mData->d_tile_keys + upperOffset, deviceKeys, sizeof(uint64_t) * deviceNodeCount(deviceId)[2], cudaMemcpyDefault, stream));
if (deviceNodeCount(deviceId)[2]) {
cudaCheck(cudaMemcpyAsync(mData->d_tile_keys + upperOffset, deviceKeys, sizeof(uint64_t) * deviceNodeCount(deviceId)[2], cudaMemcpyDefault, stream));
}
deviceNodeOffset(deviceId)[2] = upperOffset;
upperOffset += deviceNodeCount(deviceId)[2];
}
@@ -766,23 +778,25 @@ void DistributedPointsToGrid<BuildT>::countNodes(const PtrT coords, size_t coord
uint64_t* deviceInputKeys = mKeys + mStripeOffsets[deviceId];
const uint64_t* deviceOutputKeys = mData->d_keys + mStripeOffsets[deviceId];
if (deviceId == 0) {
if (deviceId > 0) {
cudaCheck(cudaEventSynchronize(voxelCountEvents[deviceId - 1]));
devicePointsPerVoxel += mVoxelCounts[deviceId - 1];
cudaCheck(cudaEventSynchronize(leafCountEvents[deviceId - 1]));
devicePointsPerLeaf += deviceNodeCount(deviceId - 1)[0];
}
if (mStripeCounts[deviceId]) {
CUB_LAUNCH(DeviceRunLengthEncode::Encode, mTempDevicePools[deviceId], stream, deviceOutputKeys, deviceInputKeys, devicePointsPerVoxel, mVoxelCounts + deviceId, mStripeCounts[deviceId]);
cudaCheck(cudaEventRecord(voxelCountEvents[deviceId], stream));
CUB_LAUNCH(DeviceRunLengthEncode::Encode, mTempDevicePools[deviceId], stream, thrust::make_transform_iterator(deviceOutputKeys, ShiftRight<9>()), deviceInputKeys, devicePointsPerLeaf, deviceNodeCount(deviceId), mStripeCounts[deviceId]);
cudaCheck(cudaEventRecord(leafCountEvents[deviceId], stream));
}
else
{
cudaCheck(cudaEventSynchronize(voxelCountEvents[deviceId - 1]));
devicePointsPerVoxel += mVoxelCounts[deviceId - 1];
CUB_LAUNCH(DeviceRunLengthEncode::Encode, mTempDevicePools[deviceId], stream, deviceOutputKeys, deviceInputKeys, devicePointsPerVoxel, mVoxelCounts + deviceId, mStripeCounts[deviceId]);
} else {
cudaCheck(cudaMemsetAsync(mVoxelCounts + deviceId, 0, sizeof(uint32_t), stream));
cudaCheck(cudaEventRecord(voxelCountEvents[deviceId], stream));
cudaCheck(cudaEventSynchronize(leafCountEvents[deviceId - 1]));
devicePointsPerLeaf += deviceNodeCount(deviceId - 1)[0];
CUB_LAUNCH(DeviceRunLengthEncode::Encode, mTempDevicePools[deviceId], stream, thrust::make_transform_iterator(deviceOutputKeys, ShiftRight<9>()), deviceInputKeys, devicePointsPerLeaf, deviceNodeCount(deviceId), mStripeCounts[deviceId]);
cudaCheck(cudaMemsetAsync(deviceNodeCount(deviceId), 0, sizeof(uint32_t), stream));
cudaCheck(cudaEventRecord(leafCountEvents[deviceId], stream));
}
}
@@ -796,7 +810,9 @@ void DistributedPointsToGrid<BuildT>::countNodes(const PtrT coords, size_t coord
for (const auto& [deviceId, stream] : mDeviceMesh) {
cudaCheck(cudaSetDevice(deviceId));
uint64_t* deviceKeys = mKeys + mStripeOffsets[deviceId];
cudaCheck(cudaMemcpyAsync(mData->d_leaf_keys + leafOffset, deviceKeys, sizeof(uint64_t) * deviceNodeCount(deviceId)[0], cudaMemcpyDefault, stream));
if (deviceNodeCount(deviceId)[0]) {
cudaCheck(cudaMemcpyAsync(mData->d_leaf_keys + leafOffset, deviceKeys, sizeof(uint64_t) * deviceNodeCount(deviceId)[0], cudaMemcpyDefault, stream));
}
deviceNodeOffset(deviceId)[0] = leafOffset;
leafOffset += deviceNodeCount(deviceId)[0];
}
@@ -808,8 +824,12 @@ void DistributedPointsToGrid<BuildT>::countNodes(const PtrT coords, size_t coord
uint64_t* deviceInputKeys = mKeys + mStripeOffsets[deviceId];
uint64_t* deviceOutputKeys = mData->d_keys + mStripeOffsets[deviceId];
CUB_LAUNCH(DeviceSelect::Unique, mTempDevicePools[deviceId], stream, thrust::make_transform_iterator(deviceOutputKeys, ShiftRight<21>()), deviceInputKeys, deviceNodeCount(deviceId) + 1, mStripeCounts[deviceId]);
cudaEventRecord(lowerCountEvents[deviceId], stream);
if (mStripeCounts[deviceId]) {
CUB_LAUNCH(DeviceSelect::Unique, mTempDevicePools[deviceId], stream, thrust::make_transform_iterator(deviceOutputKeys, ShiftRight<21>()), deviceInputKeys, deviceNodeCount(deviceId) + 1, mStripeCounts[deviceId]);
} else {
cudaCheck(cudaMemsetAsync(deviceNodeCount(deviceId) + 1, 0, sizeof(uint32_t), stream));
}
cudaCheck(cudaEventRecord(lowerCountEvents[deviceId], stream));
}
uint32_t lowerOffset = 0;
@@ -817,7 +837,9 @@ void DistributedPointsToGrid<BuildT>::countNodes(const PtrT coords, size_t coord
cudaCheck(cudaSetDevice(deviceId));
cudaCheck(cudaEventSynchronize(lowerCountEvents[deviceId]));
uint64_t* deviceKeys = mKeys + mStripeOffsets[deviceId];
cudaCheck(cudaMemcpyAsync(mData->d_lower_keys + lowerOffset, deviceKeys, sizeof(uint64_t) * deviceNodeCount(deviceId)[1], cudaMemcpyDefault, stream));
if (deviceNodeCount(deviceId)[1]) {
cudaCheck(cudaMemcpyAsync(mData->d_lower_keys + lowerOffset, deviceKeys, sizeof(uint64_t) * deviceNodeCount(deviceId)[1], cudaMemcpyDefault, stream));
}
deviceNodeOffset(deviceId)[1] = lowerOffset;
lowerOffset += deviceNodeCount(deviceId)[1];
}
@@ -837,8 +859,6 @@ void DistributedPointsToGrid<BuildT>::countNodes(const PtrT coords, size_t coord
cudaCheck(cudaSetDevice(deviceId));
cudaEventDestroy(sortEvents[deviceId]);
cudaEventDestroy(runLengthEncodeEvents[deviceId]);
cudaEventDestroy(transformReduceEvents[deviceId]);
cudaEventDestroy(rebalanceEvents[deviceId]);
cudaEventDestroy(tilePrefixSumEvents[deviceId]);
cudaEventDestroy(voxelCountEvents[deviceId]);
cudaEventDestroy(leafCountEvents[deviceId]);
@@ -909,13 +929,16 @@ inline void DistributedPointsToGrid<BuildT>::processGridTreeRoot(const PtrT poin
util::cuda::lambdaKernel<<<1, 1, 0, stream>>>(1, BuildGridTreeRootFunctor<BuildT, PtrT>(), mData, mPointType, pointCount);// lambdaKernel
cudaCheckError();
// Zero the name field, then copy only the actual string (if any).
char *dst = mData->getGrid().mGridName;
if (const char *src = mGridName.data()) {
cudaCheck(cudaMemcpyAsync(dst, src, GridData::MaxNameSize, cudaMemcpyHostToDevice, stream));
} else {
cudaCheck(cudaMemsetAsync(dst, 0, GridData::MaxNameSize, stream));
cudaCheck(cudaMemsetAsync(dst, 0, GridData::MaxNameSize, stream));
if (!mGridName.empty()) {
// Copy at most MaxNameSize-1 bytes so the memset's trailing '\0' always
// survives; a name >= MaxNameSize is truncated, never left unterminated.
const size_t nameSize = std::min<size_t>(mGridName.size(), GridData::MaxNameSize - 1);
cudaCheck(cudaMemcpyAsync(dst, mGridName.c_str(), nameSize, cudaMemcpyHostToDevice, stream));
}
cudaEventRecord(processGridTreeRootEvent);
cudaCheck(cudaEventRecord(processGridTreeRootEvent, stream));
for (const auto& [otherDeviceId, otherStream] : mDeviceMesh) {
cudaSetDevice(otherDeviceId);
@@ -976,12 +999,14 @@ inline void DistributedPointsToGrid<BuildT>::processNodes()
std::vector<cudaEvent_t> leafCountEvents(mDeviceMesh.deviceCount());
std::vector<cudaEvent_t> valueIndexPrefixSumEvents(mDeviceMesh.deviceCount());
auto lastDeviceId = cudaInvalidDeviceId;
for (const auto& [deviceId, stream] : mDeviceMesh) {
cudaSetDevice(deviceId);
cudaEventCreateWithFlags(&leafCountEvents[deviceId], cudaEventDisableTiming);
cudaEventCreateWithFlags(&valueIndexPrefixSumEvents[deviceId], cudaEventDisableTiming);
if (deviceNodeCount(deviceId)[0]) {
lastDeviceId = deviceId;
kernels::fillValueIndexKernel<BuildT><<<numBlocks(deviceNodeCount(deviceId)[0]), mNumThreads, 0, stream>>>(deviceNodeCount(deviceId)[0], deviceNodeOffset(deviceId)[0], mValueIndex, mData);
cudaCheckError();
}
@@ -990,10 +1015,18 @@ inline void DistributedPointsToGrid<BuildT>::processNodes()
LeafCountIterator leafCountIterator(mNodeCounts);
inclusiveSumAsync(mDeviceMesh, mTempDevicePools, mValueIndex, mValueIndexPrefix, leafCountIterator, leafCountEvents.data(), valueIndexPrefixSumEvents.data());
// The first leaf on each device reads the last prefix value produced by the
// previous device, while the first leaf globally reads the final prefix value.
// Wait for the corresponding producer before launching each leaf-processing kernel.
auto previousDeviceId = cudaInvalidDeviceId;
for (const auto& [deviceId, stream] : mDeviceMesh) {
cudaSetDevice(deviceId);
cudaStreamWaitEvent(stream, valueIndexPrefixSumEvents.back());
if (deviceNodeCount(deviceId)[0]) {
cudaStreamWaitEvent(stream, valueIndexPrefixSumEvents[lastDeviceId]);
if (previousDeviceId != cudaInvalidDeviceId) {
cudaStreamWaitEvent(stream, valueIndexPrefixSumEvents[previousDeviceId]);
}
previousDeviceId = deviceId;
kernels::leafPrefixSumKernel<BuildT><<<numBlocks(deviceNodeCount(deviceId)[0]), mNumThreads, 0, stream>>>(deviceNodeCount(deviceId)[0], deviceNodeOffset(deviceId)[0], mValueIndexPrefix, mData);
cudaCheckError();
}
+120 -12
View File
@@ -109,6 +109,89 @@ inline unique_ptr<uint32_t> createCrc32Lut(size_t extra = 0, cudaStream_t stream
return lut;
}
/// @brief Cuda kernel computing per-block CRC32 checksums via slicing-by-4.
/// The 256-entry base LUT is staged into shared memory and three derived
/// slice tables are built in place, so the four divergent lookups per 4-byte
/// step hit shared memory and the dependent update chain advances four bytes
/// per step instead of one. Bit-identical to the byte-serial crc32().
/// The final block absorbs any remainder of @c totalSize.
static __global__ void crc32SlicedKernel(const void *d_data, uint32_t* d_blockCRC, uint64_t blockCount, uint32_t log2BlockSize, uint64_t totalSize, const uint32_t *d_lut)
{
__shared__ uint32_t sLut[4][256];
for (uint32_t i = threadIdx.x; i < 256; i += blockDim.x) sLut[0][i] = d_lut[i];
__syncthreads();
for (int k = 1; k < 4; ++k) {
for (uint32_t i = threadIdx.x; i < 256; i += blockDim.x) {
const uint32_t c = sLut[k-1][i];
sLut[k][i] = (c >> 8) ^ sLut[0][c & 0xffu];
}
__syncthreads();
}
const uint64_t tid = blockIdx.x * (uint64_t)blockDim.x + threadIdx.x;
if (tid >= blockCount) return;
const uint8_t *p = (const uint8_t*)d_data + (tid << log2BlockSize);
uint64_t n = uint64_t(1) << log2BlockSize;
if (tid + 1 == blockCount) n += totalSize - (blockCount << log2BlockSize);
uint32_t crc = ~0u;
// blocks start at power-of-two offsets of a 32B-aligned buffer -> 4B aligned
const uint32_t *w = (const uint32_t*)p;
for (uint64_t i = 0, nw = n >> 2; i < nw; ++i) {
const uint32_t x = crc ^ w[i];
crc = sLut[3][x & 0xffu] ^ sLut[2][(x >> 8) & 0xffu] ^ sLut[1][(x >> 16) & 0xffu] ^ sLut[0][x >> 24];
}
for (uint64_t i = n & ~3ull; i < n; ++i) {
crc ^= p[i];
for (int j = 0; j < 8; ++j) crc = (crc >> 1) ^ (0xEDB88320u & (-(crc & 1u)));
}
d_blockCRC[tid] = ~crc;
}
/// @brief y = M x over GF(2), M given as 32 column words
__host__ __device__ inline uint32_t crc32Gf2MatTimes(const uint32_t *mat, uint32_t vec)
{
uint32_t sum = 0;
while (vec) {
if (vec & 1u) sum ^= *mat;
vec >>= 1;
++mat;
}
return sum;
}
/// @brief Build into @c dst[32] the GF(2) operator that advances a CRC past
/// @c bits zero bits, by binary exponentiation of the single-bit operator
/// (zlib crc32_combine construction). O(log bits x 32) - a few microseconds on
/// the host, which is where it is called so the device combine stays a cheap fold.
__host__ __device__ inline void crc32BuildShiftOp(uint32_t *dst, uint64_t bits)
{
uint32_t oddBuf[32], evenBuf[32];
uint32_t *cur = oddBuf, *nxt = evenBuf;
cur[0] = 0xEDB88320u;// operator for a single zero bit
for (int n = 1; n < 32; ++n) cur[n] = 1u << (n - 1);
for (int n = 0; n < 32; ++n) dst[n] = 1u << n;// identity
while (bits) {
if (bits & 1ull) for (int n = 0; n < 32; ++n) dst[n] = crc32Gf2MatTimes(cur, dst[n]);
bits >>= 1;
if (bits) { for (int n = 0; n < 32; ++n) nxt[n] = crc32Gf2MatTimes(cur, cur[n]); uint32_t *t = cur; cur = nxt; nxt = t; }
}
}
/// @brief Single-thread kernel folding per-chunk CRCs into the CRC of the whole
/// stream, using crc(A||B) = shift(crc(A), len(B)) ^ crc(B). The two GF(2) shift
/// operators - @c d_acc for a full chunk and @c d_accLast for the final,
/// possibly shorter, chunk - are precomputed on the host, so this is just an
/// O(chunkCount) fold. Bit-identical to a serial crc32 over the concatenated
/// stream. (@c d_accLast equals @c d_acc when the last chunk is full.)
static __global__ void crc32CombineKernel(const uint32_t *d_chunkCRC, uint64_t chunkCount, const uint32_t *d_acc, const uint32_t *d_accLast, uint32_t *d_crc)
{
uint32_t crc = d_chunkCRC[0];
for (uint64_t i = 1; i < chunkCount; ++i) {
const uint32_t *op = (i + 1 == chunkCount) ? d_accLast : d_acc;
crc = crc32Gf2MatTimes(op, crc) ^ d_chunkCRC[i];
}
*d_crc = crc;
}
/// @brief Compute CRC32 checksum of 4K block
/// @param d_data device pointer to start of data
/// @param size number of bytes
@@ -121,14 +204,39 @@ inline void blockedCRC32(const void *d_data, size_t size, const uint32_t *d_lut,
const uint64_t checksumCount = size >> NANOVDB_CRC32_LOG2_BLOCK_SIZE;// 4 KB (4096 byte)
unique_ptr<uint32_t> buffer(checksumCount, stream);// for checksums of 4 KB blocks
uint32_t *d_checksums = buffer.get();
lambdaKernel<<<blocksPerGrid(checksumCount, threadsPerBlock), threadsPerBlock, 0, stream>>>(checksumCount, [=] __device__(size_t tid) {
uint32_t blockSize = 1 << NANOVDB_CRC32_LOG2_BLOCK_SIZE;
if (tid+1 == checksumCount) blockSize += size - (checksumCount<<NANOVDB_CRC32_LOG2_BLOCK_SIZE);
d_checksums[tid] = crc32((const uint8_t*)d_data + (tid<<NANOVDB_CRC32_LOG2_BLOCK_SIZE), blockSize, d_lut);
}); cudaCheckError();
lambdaKernel<<<1, 1, 0, stream>>>(1, [=] __device__(size_t) {// Compute CRC32 of all the 4K blocks
*d_crc = crc32((const uint8_t*)d_checksums, checksumCount*sizeof(uint32_t), d_lut);
}); cudaCheckError();
crc32SlicedKernel<<<blocksPerGrid(checksumCount, threadsPerBlock), threadsPerBlock, 0, stream>>>(
d_data, d_checksums, checksumCount, NANOVDB_CRC32_LOG2_BLOCK_SIZE, size, d_lut);
cudaCheckError();
// CRC of the block-checksum array itself. The former single-thread pass
// over checksumCount*4 bytes (megabytes for multi-GB grids) is replaced
// by parallel per-chunk CRCs plus a GF(2) combine - bit-identical result.
const uint64_t checksumBytes = checksumCount*sizeof(uint32_t);
constexpr uint64_t log2ChunkSize = 12, chunkSize = uint64_t(1) << log2ChunkSize;
if (checksumBytes <= 2*chunkSize) {// small: single-thread CRC is fine
lambdaKernel<<<1, 1, 0, stream>>>(1, [=] __device__(size_t) {
*d_crc = crc32((const uint8_t*)d_checksums, checksumBytes, d_lut);
}); cudaCheckError();
} else {
const uint64_t chunkCount = checksumBytes >> log2ChunkSize;// final chunk absorbs the remainder
const uint64_t lastChunkBytes = chunkSize + (checksumBytes - (chunkCount << log2ChunkSize));
unique_ptr<uint32_t> chunkBuffer(chunkCount, stream);
uint32_t *d_chunkCRC = chunkBuffer.get();
crc32SlicedKernel<<<blocksPerGrid(chunkCount, threadsPerBlock), threadsPerBlock, 0, stream>>>(
d_checksums, d_chunkCRC, chunkCount, log2ChunkSize, checksumBytes, d_lut);
cudaCheckError();
// Precompute the two GF(2) shift operators on the host (data-independent,
// ~microseconds) and upload them, instead of rebuilding them on a single
// device thread inside the combine - that build dominated the checksum
// for small and medium grids.
uint32_t hOps[64];
crc32BuildShiftOp(hOps, chunkSize * 8ull);// operator for a full chunk
crc32BuildShiftOp(hOps + 32, lastChunkBytes * 8ull);// operator for the last chunk
unique_ptr<uint32_t> opBuffer(64, stream);
uint32_t *d_ops = opBuffer.get();
cudaCheck(cudaMemcpyAsync(d_ops, hOps, 64*sizeof(uint32_t), cudaMemcpyHostToDevice, stream));
crc32CombineKernel<<<1, 1, 0, stream>>>(d_chunkCRC, chunkCount, d_ops, d_ops + 32, d_crc);
cudaCheckError();
}
}// void cudaBlockedCRC32(const void *d_data, size_t size, const uint32_t *d_lut, uint32_t *d_crc, cudaStream_t stream)
/// @brief Compute CRC32 checksum of 4K block
@@ -234,7 +342,7 @@ inline Checksum evalChecksum(const GridData *d_gridData, CheckMode mode, cudaStr
if (mode != CheckMode::Empty) {
auto d_lut = util::cuda::createCrc32Lut(1, stream);
crc32Head(d_gridData, d_lut.get(), d_lut.get() + 256, stream);
cudaCheck(cudaMemcpyAsync(&(cs.head()), d_lut.get() + 256, headSize, cudaMemcpyDeviceToHost, stream));
cudaCheck(cudaMemcpyAsync(&(cs.head()), d_lut.get() + 256, sizeof(uint32_t), cudaMemcpyDeviceToHost, stream));
if (mode == CheckMode::Full) {
std::unique_ptr<char[]> buffer(new char[headSize]);
auto *gridData = (GridData*)(buffer.get());
@@ -244,7 +352,7 @@ inline Checksum evalChecksum(const GridData *d_gridData, CheckMode mode, cudaStr
} else {
callNanoGrid<Crc32TailOld>(d_gridData, gridData, d_lut.get(), d_lut.get() + 256, stream);
}
cudaCheck(cudaMemcpyAsync(&(cs.tail()), d_lut.get() + 256, headSize, cudaMemcpyDeviceToHost, stream));
cudaCheck(cudaMemcpyAsync(&(cs.tail()), d_lut.get() + 256, sizeof(uint32_t), cudaMemcpyDeviceToHost, stream));
}
}
return cs;
@@ -265,7 +373,7 @@ Checksum evalChecksum(const NanoGrid<BuildT> *d_grid, CheckMode mode, cudaStream
if (mode != CheckMode::Empty) {
auto d_lut = util::cuda::createCrc32Lut(1, stream);
crc32Head(d_grid, d_lut.get(), d_lut.get() + 256, stream);
cudaCheck(cudaMemcpyAsync(&(cs.head()), d_lut.get() + 256, headSize, cudaMemcpyDeviceToHost, stream));
cudaCheck(cudaMemcpyAsync(&(cs.head()), d_lut.get() + 256, sizeof(uint32_t), cudaMemcpyDeviceToHost, stream));
if (mode == CheckMode::Full) {
std::unique_ptr<char[]> buffer(new char[headSize]);
auto *gridData = (GridData*)(buffer.get());
@@ -275,7 +383,7 @@ Checksum evalChecksum(const NanoGrid<BuildT> *d_grid, CheckMode mode, cudaStream
} else {
crc32TailOld(d_grid, gridData, d_lut.get(), d_lut.get() + 256, stream);
}
cudaCheck(cudaMemcpyAsync(&(cs.tail()), d_lut.get() + 256, headSize, cudaMemcpyDeviceToHost, stream));
cudaCheck(cudaMemcpyAsync(&(cs.tail()), d_lut.get() + 256, sizeof(uint32_t), cudaMemcpyDeviceToHost, stream));
}
}
return cs;

Some files were not shown because too many files have changed in this diff Show More