Compare commits

...
Author SHA1 Message Date
Jonathan Swartz 2563cd3118 Attempting to fix secretes manager access error
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-25 10:35:02 +12:00
Jonathan Swartz 0510e49968 Improving buildspec logging
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-25 10:23:53 +12:00
Jonathan Swartz 1363171185 Build fixes and printing URLs for monitoring
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-25 10:18:03 +12:00
Jonathan Swartz 5cfdc69063 Trying to retrieve codebuild log info
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-25 09:52:32 +12:00
Jonathan Swartz 9e0d18504b Better testing
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-25 09:31:01 +12:00
Jonathan Swartz 3f1a6c851c Fix condition
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-25 09:28:46 +12:00
Jonathan Swartz 16ce308e76 Print if credentialas are configured
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-25 09:27:07 +12:00
Jonathan Swartz de4c6c89f7 Attempting to set permissions
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-25 09:23:33 +12:00
Jonathan Swartz 2ad5811297 Switch back to v4 because credentials action requires later version
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-19 10:34:47 +12:00
Jonathan Swartz 705e6d79c5 Merge remote-tracking branch 'origin/feature/fvdb' into fvdb_codebuild
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-19 10:32:56 +12:00
Jonathan Swartz af0fc1e7ec Merge remote-tracking branch 'origin/feature/fvdb' into fvdb_codebuild
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-19 10:30:08 +12:00
Jonathan Swartz 46d8f3c26c Change configure-aws-credentials action version to test
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-19 10:26:29 +12:00
Jonathan Swartz 3fad26296f Merge pull request #2052 from swahtz/feature/fvdb
Update feature/fvdb
2025-06-17 11:34:07 +12:00
Jonathan Swartz a41e9319aa Merge remote-tracking branch 'downstream/feature/fvdb' into feature/fvdb
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-17 11:32:37 +12:00
Jonathan SwartzandMark Harris 1563730c0a GaussianRasterizeForward shared memory calculation fix (#539)
* Fix for shared memory calculation to account for Gaussian2D alignment

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Mark Harris <mharris@nvidia.com>
2025-06-17 11:27:07 +12:00
Andrew 6fd95083d4 Unify editor shaders. 2025-06-16 10:10:18 -05:00
Petra Hapalova 3cc26291ca NanoVDB editor - track shader includes changes (#540)
* track shader includes changes
* clear shader cache button
2025-06-16 14:13:24 +00:00
Petra HapalovaandAndrew Reidmeyer 9f08f33fe6 Merge NanoVDB editor into fVDB (#534)
* adding NanoVDB Editor
---------

Co-authored-by: Andrew Reidmeyer <areidmeyer@nvidia.com>
2025-06-13 22:32:43 +02:00
Jonathan Swartz 0741614991 Merge pull request #2050 from swahtz/feature/fvdb
feature/fvdb update
2025-06-13 18:01:58 +12:00
Jonathan Swartz 0de6bf111a Merge remote-tracking branch 'downstream/feature/fvdb' into feature/fvdb
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-13 17:57:56 +12:00
Jonathan Swartz 216ef435fc Improve launch_bounds coverage (#530)
* Improve launch_bounds coverage everywhere there's an easily inferred number of threads
* Add a global DEFAULT_NUM_THREADS_PER_BLOCK and made sure all kernels use GET_BLOCKS

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-13 17:54:36 +12:00
Jonathan Swartz 07a099136c Fix an issue where the PCA performed by the ColmapDataset in 3dgs was non-deterministic when using numpy built with OpenBLAS. Substitute for MKL blas. (#531)
Fix README error

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-12 16:55:30 +12:00
Jonathan Swartz 504fa0fcdf Added up_axis controls to 3dgs Viser viewer (#532)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-12 16:55:21 +12:00
Matthew Cong 3a260e0374 Remove compile_commands.json symlink and add to .gitignore (#520)
* Remove compile_commands.json
* Add compile_commands.json symlink to .gitignore

Signed-off-by: Mark Harris mharris@nvidia.com
2025-06-11 10:40:59 +10:00
Mark Harris 4963380b23 Store image dims in input data sets, use in gtests (#518)
Adds image dims to the test data generation for forward gsplat rasterization. uses this in gtests. Adds a new script for inspecting generated datasets. Updates test data commit hash in cmake.

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-06-11 10:36:33 +10:00
Jonathan Swartz 8ad3e9ba75 Add camera images to gsplat viser viewer (#514)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-10 19:15:19 +12:00
Mark Harris 8ce1fca72e Add debug build option to build.sh (#515)
* build.sh: add debug build option

* Auto update compile_commands.json symlink
2025-06-10 19:03:59 +12:00
Mark Harris d871dc46d3 Move implementation of tensor loading utilities to .cpp file, inline other utilities in headers. (#516)
Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-06-10 19:01:08 +12:00
Mark Harris 9e3b691e2a Seaube clang-format extension requries env.VAR rather than env:VAR (#517)
Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-06-10 18:58:00 +12:00
Matthew Cong 74581cdc6b Use PIMPL idiom for GridBatch/GridBatchImpl (#513)
* WIP: Use PIMPL idiom for GridBatch/GridBatchImpl

* Expose remaining sparse conv functions

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Remove variants of GridBatch::to(...)

* Remove address()

* Refactor Python bindings

* Restore address

* Remove implementations from header files

* Clean up API more

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-06-09 16:09:19 -07:00
Matthew Cong 9c537c776e Remove unused RAIIDeviceBuffer includes (#511) 2025-06-05 08:45:28 -07:00
Matthew Cong 938e02b5d6 Remove RAIIDeviceGuard (#510) 2025-06-05 08:45:19 -07:00
Matthew Cong d51a25714f Remove NanoVDBFileGridIdentifier union type (#505)
* Remove NanoVDBFileGridIdentifier from Python API

* Format

* Remove NanoVDBFileGridIdentifier from C++ API

* Expose parseDeviceString

* Centralize device string overloads to Python bindings
2025-06-05 07:01:16 -04:00
Mark Harris 24aa6d706b Use unique_ptr in RAIIDeviceGuard (#509)
Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-06-05 06:55:17 -04:00
Jonathan Swartz 010cbd9b5c Merge pull request #2049 from swahtz/feature/fvdb
feature/fvdb update
2025-06-05 21:44:57 +12:00
Mark Harris d6b2f243d3 Use Seaube.clangFormat vscode extension for clang-format (#508)
We have been getting intermittent crashes of clangd, and since we have been using the clang-format that is built into the clangd server, when clangd crashes we lose the format-on-save capability of vscode for C++ and CUDA C++ files.

This PR adds settings and recommendation for the vscode extension Seaube.clangFormat which allows specifying a clang-format executable to be used for formatting C++ and CUDA C++ files. This is a fork of the more widely used xaver.clang-format vscode extension because the latter has not released an update in years, in spite of having merged NVIDIA pull requests to support the cuda-cpp language.

After merging, you will need to make sure to install the vscode extension Seaube.clangFormat. VSCode will prompt to install it.

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-06-05 18:13:48 +12:00
Matthew Cong 32aac9e6e4 Remove JaggedTensorIndex union type (#503)
* Remove JaggedTensorIndex union type

* Fix error type

* Fix error type again

* Restore accidentally commented import
2025-06-04 21:58:27 -07:00
Mark Harris 7e0e66645d Add verbose option to build.sh, use it in CI workflow, add build.sh help (#507)
Add `verbose` option to build.sh and use it in CI.  Add -h/--help
---------
Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-06-05 12:42:11 +10:00
Francis Williams 6221f03a83 Move voxel size and bounding box functions into GridBatchImpl (#500)
* more gridbatch slimming

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more slimming

Signed-off-by: Francis Williams <francis@fwilliams.info>

* even more slimming

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more deviceguard cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more deviceguard

Signed-off-by: Francis Williams <francis@fwilliams.info>

* naming convention

Signed-off-by: Francis Williams <francis@fwilliams.info>

* address more comments

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more thinning

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more lean, and commetns

Signed-off-by: Francis Williams <francis@fwilliams.info>

---------

Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-06-04 21:10:44 -04:00
Jonathan Swartz 64ceb23374 Merge remote-tracking branch 'downstream/feature/fvdb' into feature/fvdb
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-06-05 11:18:44 +12:00
Mark Harris ca9ff4a4ee Add CUDA attribute formatting and kernel launch bounds improvements (#498)
* Add AttributeMacros to .clang-format for CUDA attributes

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Reformat source for CUDA attributes, add launch bounds to some 3DGS kernels

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Update script for running clang-format

Signed-off-by: Mark Harris <mharris@nvidia.com>

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-06-04 13:06:48 -04:00
Matthew Cong 278a3ae8c9 Remove unused definitions which have moved into GridBatchImpl (#502) 2025-06-04 13:06:17 -04:00
Matthew Cong c95e422243 Remove Tensor -> JaggedTensor typecaster in favor of implicit conversion (#501) 2025-06-04 13:06:07 -04:00
Eftychios Sifakis 3d36ec7e99 Added grid merging functionality to fVDB (#494)
* Initial steps for grid merging routine

* cleanup and completion of root node merge

* completed enumeration of nodes on merge operation

* finished MergeGrids operation; cleanup

* Exposed MergeGrids in fVDB

* formatting fixes

* formatting fixes

* formatting fixes
2025-06-04 10:13:38 -04:00
Mark Harris 64c8d65333 Fix spherical harmonic tensor shapes in test data generation script (#497)
Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-06-04 15:08:00 +12:00
Mark Harris bb8dda8987 Update VSCode settings for CUDA support and clean up C++ file associations (#496)
Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-06-04 13:57:58 +12:00
Matthew Cong bdc38eb961 Cleanup unneeded includes (#495) 2025-06-03 11:06:45 -07:00
Christopher Horvath 6ab2ed501a fvdb source location restructure (#490)
Refactor: move of fvdb library sources to src/fvdb, update includes and .clang-format rules.

---------

Signed-off-by: Christopher Horvath <blackencino+git@gmail.com>
2025-06-03 15:11:51 +10:00
Francis Williams 624b227afa Move more grid building functions from GridBatch into detail (#477)
* build grid

Signed-off-by: Francis Williams <francis@fwilliams.info>

* unskip test

Signed-off-by: Francis Williams <francis@fwilliams.info>

* check device for optionals

Signed-off-by: Francis Williams <francis@fwilliams.info>

* pull out grid building code into seperate module

Signed-off-by: Francis Williams <francis@fwilliams.info>

* cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more migration

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* another one

Signed-off-by: Francis Williams <francis@fwilliams.info>

* include guards

Signed-off-by: Francis Williams <francis@fwilliams.info>

* include guards

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more include guards

Signed-off-by: Francis Williams <francis@fwilliams.info>

* include guards

Signed-off-by: Francis Williams <francis@fwilliams.info>

* move to GridBatchImpl

Signed-off-by: Francis Williams <francis@fwilliams.info>

* clang fmt

Signed-off-by: Francis Williams <francis@fwilliams.info>

* rm write accessor

Signed-off-by: Francis Williams <francis@fwilliams.info>

* need writeaccessor

Signed-off-by: Francis Williams <francis@fwilliams.info>

* fix

Signed-off-by: Francis Williams <francis@fwilliams.info>

* update

Signed-off-by: Francis Williams <francis@fwilliams.info>

* cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* move to static

Signed-off-by: Francis Williams <francis@fwilliams.info>

* typo

Signed-off-by: Francis Williams <francis@fwilliams.info>

* typo 2

Signed-off-by: Francis Williams <francis@fwilliams.info>

* rename

Signed-off-by: Francis Williams <francis@fwilliams.info>

* rename

Signed-off-by: Francis Williams <francis@fwilliams.info>

* rename

Signed-off-by: Francis Williams <francis@fwilliams.info>

---------

Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-06-02 19:18:06 +00:00
Matthew Cong 533cf4c20e Restore mGPU support for createGridFromPoints/IJK (#489) 2025-06-02 12:07:26 -07:00
Matthew Cong 22de0d9aca Fix build error (#488) 2025-05-30 21:06:34 -04:00
zlalena 384198f3ee Gaussian Splatting Camera Pose Optimization with multiGPU (#422) 2025-05-30 20:18:18 -04:00
Matthew Cong 4213aa354a Re-enable warning as errors (#484)
* Fix class-memaccess warning in CreateNanoGrid.h

* Revert "comment out Werr (#464)"

This reverts commit c760c1ccbe.
2025-05-30 17:57:58 -04:00
Francis Williams f4ef228672 re-enable test (#486)
Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-05-30 21:19:16 +00:00
Francis Williams 5b011d23c4 Move grid building logic and error checking out of GridBatch into detail (#476)
* build grid

Signed-off-by: Francis Williams <francis@fwilliams.info>

* unskip test

Signed-off-by: Francis Williams <francis@fwilliams.info>

* check device for optionals

Signed-off-by: Francis Williams <francis@fwilliams.info>

* pull out grid building code into seperate module

Signed-off-by: Francis Williams <francis@fwilliams.info>

* cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* include guards

Signed-off-by: Francis Williams <francis@fwilliams.info>

* include guards

Signed-off-by: Francis Williams <francis@fwilliams.info>

* move to GridBatchImpl

Signed-off-by: Francis Williams <francis@fwilliams.info>

* clang fmt

Signed-off-by: Francis Williams <francis@fwilliams.info>

* rm write accessor

Signed-off-by: Francis Williams <francis@fwilliams.info>

* need writeaccessor

Signed-off-by: Francis Williams <francis@fwilliams.info>

* fix

Signed-off-by: Francis Williams <francis@fwilliams.info>

* skip test again

Signed-off-by: Francis Williams <francis@fwilliams.info>

---------

Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-05-30 21:12:48 +00:00
Matthew Cong 05b2bf6f11 Fix one more warning (#481) 2025-05-29 21:51:40 -07:00
Matthew Cong cde4160d42 Fix newly introduced warnings in fVDB (#480) 2025-05-29 16:50:19 -07:00
Matthew Cong 1b29548661 Add initial support for Unified Memory backend (#460)
* Add initial support for Unified Memory

* Add MGPU support for transform points

* Cleanup

* Add guard for CUDA version

* Add cudamanaged support for JOffsetsFromJIdx and PointsInGrid

* Fixes for signed/unsigned warnings

* Rename for consistency

* Remove hardcoded launch bounds

* Internalize numThreads to forEach kernels

* Rename macro to explicitly call out backends

* Clean up signed/unsigned indexing and reduce default block size

* Clean up macro

* Format code

* Remove unnecessary includes

* Address some review comments

* Fix helper function and formatting

* Encapsulate allocation logic
2025-05-29 15:11:43 -07:00
Francis Williams a414cef4c5 Don't use verbosity 1 in dilation and restore dilation in ray marching tests (#475)
* small clangd fix

Signed-off-by: Francis Williams <francis@fwilliams.info>

* Don't use verbosity 1 in dilation and restore previously crashing dilation in test_ray_marching.py

Signed-off-by: Francis Williams <francis@fwilliams.info>

---------

Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-05-29 14:17:52 +00:00
Francis Williams 55d80ce33d small clangd fix (#474)
Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-05-29 14:11:17 +00:00
Eftychios Sifakis 87ce3b6c71 Adding support for dilation operations that generate new root nodes (#473)
* Bugfix: consistent passing of stream parameter & synchronization

* using cub::WarpReduce instead of sm80 intrinsic

* replacing sm80+ intrinsic with cub::WarpReduce

* Supporting dilation scenarios that create new root nodes
2025-05-29 09:32:33 -04:00
Francis WilliamsandMatthew Cong c875c46a20 Remove mutable grids from internal API (#462)
* grid dilation in fVDB

Signed-off-by: Francis Williams <francis@fwilliams.info>

* SPDX

Signed-off-by: Francis Williams <francis@fwilliams.info>

* start removing pad args

Signed-off-by: Francis Williams <francis@fwilliams.info>

* rm mutable grids from public API

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more nuking

Signed-off-by: Francis Williams <francis@fwilliams.info>

* fix comments

Signed-off-by: Francis Williams <francis@fwilliams.info>

* Fix CI failures due to dilation (#454)

* Remove unneeded atomic include (#451)

* Add minor cleanup (#452)

* move dilate to ops

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more mutable reference removal

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* fine from coarse

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* mas cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* rm all of build directory

Signed-off-by: Francis Williams <francis@fwilliams.info>

* clean up build kernels

Signed-off-by: Francis Williams <francis@fwilliams.info>

* include guards

* checkpoint

* rm prints

Signed-off-by: Francis Williams <francis@fwilliams.info>

* rm macros

Signed-off-by: Francis Williams <francis@fwilliams.info>

* rm custom accessors

Signed-off-by: Francis Williams <francis@fwilliams.info>

* comment out Werr

* clang format

Signed-off-by: Francis Williams <francis@fwilliams.info>

* clang format after downgrade to 18

Signed-off-by: Francis Williams <francis@fwilliams.info>

* small fix

Signed-off-by: Francis Williams <francis@fwilliams.info>

* use OnIndexGrid instead of NanoGrid<...>

Signed-off-by: Francis Williams <francis@fwilliams.info>

---------

Signed-off-by: Francis Williams <francis@fwilliams.info>
Co-authored-by: Matthew Cong <1372750+matthewdcong@users.noreply.github.com>
2025-05-29 09:29:45 -04:00
Francis Williams 88b69a968b Remove allocate/free functions from GridBatchImpl header (#471)
* clean up header

Signed-off-by: Francis Williams <francis@fwilliams.info>

* rm if

Signed-off-by: Francis Williams <francis@fwilliams.info>

* herp derp

Signed-off-by: Francis Williams <francis@fwilliams.info>

---------

Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-05-28 20:15:56 -07:00
Matthew Cong 782c0c403e Merge pull request #470 from matthewdcong/develop_fvdb_merge
Merge latest develop into feature/fvdb
2025-05-28 09:52:06 -07:00
Francis WilliamsandMatthew Cong 8162fb42fe Remove mutable grids from public API (#461)
* grid dilation in fVDB

Signed-off-by: Francis Williams <francis@fwilliams.info>

* SPDX

Signed-off-by: Francis Williams <francis@fwilliams.info>

* start removing pad args

Signed-off-by: Francis Williams <francis@fwilliams.info>

* rm mutable grids from public API

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more nuking

Signed-off-by: Francis Williams <francis@fwilliams.info>

* fix comments

Signed-off-by: Francis Williams <francis@fwilliams.info>

* Fix CI failures due to dilation (#454)

* Remove unneeded atomic include (#451)

* Add minor cleanup (#452)

* move dilate to ops

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more mutable reference removal

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* fine from coarse

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* mas cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* more cleanup

Signed-off-by: Francis Williams <francis@fwilliams.info>

* rm all of build directory

Signed-off-by: Francis Williams <francis@fwilliams.info>

* clean up build kernels

Signed-off-by: Francis Williams <francis@fwilliams.info>

* include guards

* comment out Werr

* fix clangd crash

Signed-off-by: Francis Williams <francis@fwilliams.info>

* fix tests for now

Signed-off-by: Francis Williams <francis@fwilliams.info>

* fix

Signed-off-by: Francis Williams <francis@fwilliams.info>

---------

Signed-off-by: Francis Williams <francis@fwilliams.info>
Co-authored-by: Matthew Cong <1372750+matthewdcong@users.noreply.github.com>
2025-05-28 12:40:05 -04:00
Matthew Cong d1b970b688 Merge branch 'develop' into develop_fvdb_merge 2025-05-28 08:18:17 -07:00
Francis Williams 38c078a773 Fix clangd crash in GridBatchImpl (#469)
* fix clangd crash

Signed-off-by: Francis Williams <francis@fwilliams.info>

* fix

Signed-off-by: Francis Williams <francis@fwilliams.info>

---------

Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-05-28 14:44:28 +00:00
Matthew Cong d02dc930f0 Fixes for signed/unsigned warnings (#465) 2025-05-27 16:05:03 -07:00
Francis Williams c760c1ccbe comment out Werr (#464) 2025-05-27 12:40:04 -07:00
Mark Harris 07993354a7 Warnings are Errors (#437)
This PR fixes all current warnings in fVDB. The goal is that by default, the build should only print messages when there is something wrong. It is quiet by default and there are no warnings.

 -  Fixes existing warnings that occur without -Wall
 - Fixes additional (many more) warnings that occur with -Wall. Mostly signed/unsigned comparisons and unused variables.
 - Enables -Wall -Werror for CXX options
 - Enables -Werror=all-warnings -XCompiler=-Wall,-Werror for CUDA options.
 - Disables a bunch of #pragma message output by default
 - Makes the build quiet by default rather than verbose.

Once this is merged, compilation will fail if there are any warnings.

To do a verbose build now, add -v for verbose pip output, and -C build.verbose=true for verbose CMake output (print every compilation command line).

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-05-27 13:08:53 +10:00
Eftychios Sifakis f5b68239b1 Bug fixes in DilateGrid (#458)
* Bugfix: consistent passing of stream parameter & synchronization

* using cub::WarpReduce instead of sm80 intrinsic

* replacing sm80+ intrinsic with cub::WarpReduce
2025-05-23 17:18:44 -04:00
Matthew Cong 712839bed5 Move tensorBytePointer into GaussianSplatting TU (#459) 2025-05-23 11:05:41 -07:00
Matthew Cong 8fb305aa28 Add minor cleanup (#452) 2025-05-23 09:47:22 -04:00
Matthew Cong 49ca5c2a7a Remove unneeded atomic include (#451) 2025-05-23 09:46:55 -04:00
Matthew Cong 7b0f8bf685 Fix CI failures due to dilation (#454) 2025-05-22 22:47:52 -07:00
Francis Williams 2009f7da08 Integrate new dilation into fvdb GridBatch (#447)
* grid dilation in fVDB

Signed-off-by: Francis Williams <francis@fwilliams.info>

* SPDX

Signed-off-by: Francis Williams <francis@fwilliams.info>

---------

Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-05-22 14:03:15 -04:00
Mark Harris d6a68d1ef3 Clang-format: disable bin packing and other improvements. (#434)
* Improve clang-format settings and .clang-format organization

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-05-22 14:07:22 +10:00
Eftychios Sifakis 063b4b9bee Morphological dilation in NanoVDB (#420)
This new feature provides a device-accelerated process for computing a morphological dilation of a NanoVDB IndexGrid.
3 modes of dilation are supported (see NearestNeighbors in util/MorphologyHelpers.h):
NN_FACE (6-connected); NN_FACE_EDGE (18-connected); NN_FACE_EDGE_VERTEX (26-connected).

Known limitation: As of right now (easily fixable when we have a real-world test case), the operation will proceed correctly only if the dilation operation does not create any new ROOT (4096^3) node.

An example of use can be found in examples/ex_dilate_nanovdb_cuda, which dilates an OpenVDB .vdb file, and compares the result against the NanoVDB/CUDA new implementation.
2025-05-21 14:00:26 -05:00
Mark Harris 0987d220ba Rename AccessorHelpers.h to AccessorHelpers.cuh (#436)
The file AccessorHelpers.h added in #432 is only included from .cu source files, and clangd reports problems because some of the types it uses are only defined when compiling CUDA code. So it's better to make this a CUDA header (.cuh). This PR renames it and updates all includes.

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-05-21 15:35:14 +10:00
Matthew Cong 005cd8fc48 Modularize functionality in Utils.cuh (#432)
* Separate countEnabledPerLeaf

* Separate forEachCUDA functions from Utils.cuh

* Separate out forEachCPU calls

* Separate out Accessor helper functions

* Remove PrivateUse1

* Rename
2025-05-20 08:41:26 -07:00
Matthew Cong ecbc9ff68f Replace Torch type <-> string functions with PyTorch built-ins (#429)
* Replace Torch type <-> string functions with PyTorch built-ins

* Revert "Replace Torch type <-> string functions with PyTorch built-ins"

This reverts commit 2e9e43515af924459cf48875ebd75698efecf0b9.

* Use PyTorch built-ins in later versions
2025-05-19 16:01:28 -04:00
Matthew Cong d79383c194 Separate RAII device buffer from forEach macros (#431)
* Separate RAII device buffer from forEach macros

* Fix header guard
2025-05-19 12:57:41 -07:00
Matthew Cong 66013466c3 Replace c10::optional with std::optional (#430) 2025-05-19 14:50:31 -04:00
Francis Williams c70b3dbac4 Jagged Tensor squeeze (#423) 2025-05-16 19:03:22 -04:00
Matthew Cong 6545a4b620 Cleanup accessor utility functions using std::conditional (#427) 2025-05-16 15:49:00 -07:00
Matthew Cong e8ea08db78 Fix test_gsplat formatting which is breaking CI (#428) 2025-05-16 14:47:15 -07:00
j-nordling 5a741ae46f bugfix: save nan splat (#426) 2025-05-16 17:25:01 -04:00
Francis Williams 8dba107547 Add optional stream and shared memory arguments to forEachVoxelCUDA/etc. (#416)
* streams and shared memory in forEachXXX

Signed-off-by: Francis Williams <francis@fwilliams.info>

* overload

Signed-off-by: Francis Williams <francis@fwilliams.info>

* option to call callback everytime

Signed-off-by: Francis Williams <francis@fwilliams.info>

---------

Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-05-16 11:23:26 -04:00
Jonathan Swartz 9e531b0a16 Merge pull request #2045 from swahtz/fvdb_update 2025-05-15 16:28:37 +12:00
Matthew Cong c2eb7e6ce3 Switch to modern AT_DISPATCH_V2 macro (#402) 2025-05-14 15:24:53 -07:00
Matthew Cong cba1da5961 Merge pull request #406 from matthewdcong/develop_fvdb_merge
Redo develop to feature/fvdb merge to account for squashing changes
2025-05-14 10:22:05 -07:00
Matthew Cong b0b26884a1 Merge remote-tracking branch 'upstream/develop' into feature/fvdb 2025-05-14 10:19:00 -07:00
e5d9c13b5d Merge latest develop into feature/fvdb (#405)
* Merge latest master into develop (#394)

* Avoid unnecessary and expensive getLastWriteTime calls for memory mapped files on windows

Signed-off-by: jjtParadox <jturner@4nsi.com>

* Add [[nodiscard]] to confusing math functions.

Signed-off-by: Jeff Lait <jlait@andorra.sidefx.com>

* Fix missing OpenVDB dependency for NanoVDB-only build

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* improved change notes and updated version number

* improved change notes and updated version number

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* improved documentation

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Separate PointsToGrid kernel lambdas into functors

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add missing iomanip include for latest GTest

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* improved nanovdb_convert

* improved nanovdb_convert

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* fixed typo

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* added unit tests

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* review feedback

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix format strings and Windows conversion warning

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix ODR violation due to anonymous namespace in SignedFloodFill

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Minor fix to unit-test of NanoVDB

* Minor fix to unit-test of NanoVDB

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* minor change

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add DeviceMesh and (optional) NCCL CMake Support

* Add device mesh

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Restore current device in constructor and destructor

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Enable range-based for loops

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Use structured bindings inside for loop

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Remove uneeded accessors and modularize functions

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Statically initialize entry point

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Encapsulate

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Switch to lightweight wrapper class for DeviceNodes vector

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Cleanup

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Minor changes needed for DistributedPointsToGrid

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add RAII DeviceGuard

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* More cleanup

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Clean up NCCL dependency

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Remove parallel_for to modularize

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Implement and test move constructor/assignment

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add docs

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Expand test and documentation

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Move non-reinit comment to function description

Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>

* Switch to using aliases and modern type names

Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>

* Use size_type instead of int for return

Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>

* Separate out DeviceGuard and clean up API

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Switch to DeviceMesh for MGPU sparse convolution example

* Convert MGPU convolution example to use DeviceMesh

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix typos and clarify

Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>

* Check for errors and switch to for_each

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add missing const variant of deviceData(device)

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add missing inline for DeviceMesh functions

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add missing NCCL ifdef

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix invalid free in UnifiedBuffer move constructor

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Change UnifiedBuffer prefetch and advise to be const

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix missing relocatable device code flag

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix namespace for AbsDiff and RelDiff ostream overloads

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix out-of-bounds device id

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix TBB template deduction in C++20

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fixed issue in signedfloodfill

* Fixed issue in signedfloodfill

Signed-off-by: Ken <ken.museth@gmail.com>

* partially addressed review comments

Signed-off-by: Ken <ken.museth@gmail.com>

* added RootData::TileIterator

Signed-off-by: Ken <ken.museth@gmail.com>

* cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* fixed assert bug

Signed-off-by: Ken <ken.museth@gmail.com>

* snapshot

Signed-off-by: Ken <ken.museth@gmail.com>

* fixed bug and improved unit-test

Signed-off-by: Ken <ken.museth@gmail.com>

* improved unit test and documentation

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix DeviceMesh.h header to be self-contained

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add missing constructor for C++20 used in AddBlindData.cuh

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* removed dead code and updated the minor version number

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix class-memaccess warning due to non-trivial copy-assignment in GridBlindMetaData

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* major improvements to GridBlindMetaData

* major improvements to GridBlindMetaData

Signed-off-by: Ken <ken.museth@gmail.com>

* minor changes to fix clang issue

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* minor simplification in CreateNanoGrid

* minor simplification in CreateNanoGrid

Signed-off-by: Ken <ken.museth@gmail.com>

* fixed typo

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* introducing new magic numbers for grids and files

* introducing new magic numbers for grids and files

Signed-off-by: Ken <ken.museth@gmail.com>

* removed two redundant magic numbers

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* fixed typos and improved documentation in NanoVDB

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Consolidate DeviceGuard into DeviceMesh header

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add distributed implementation of PointsToGrid

* Add distributed implementation of PointsToGrid

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix race condition in merge and single GPU case

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix kernel/cudaFree race conditions

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add DistributedPointsToGrid unittest

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Clean up example

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Use range-based for loops

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Use structured bindings inside for loop

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix Windows build and some warnings

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Update for refactored DeviceMesh

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix copyright/include

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add parallelForEach helper

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Templatize kernels to align with CUB better

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Shorten TemporaryDevicePool to TempDevicePool

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Refactor to use TempDevicePool

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Switch to class

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Parallelize kernel dispatch in build

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix race condition wrt to root node processing

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Speed up synchronization

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add comments and fix deprecated TransformInputIterator

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Address review comments

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Address more review comments

* Add more cudaCheck calls and restrict EqualityIndicator type

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix license identifiers and remove unused code

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add missing include

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: = <=>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* revert deprecation warning

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* minor cleanup

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Silence CMake warning about FindBoost module deprecation

* Silence CMake warning about FindBoost module deprecation

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix typo

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Adding /bigobj to TestNanoVDB.cu compilation options to fix Windows build failures

* Adding /bigobj to TestNanoVDB.cu compilation options to fix Windows build failure
---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Suppress stringop-overflow warning due to GCC 11 regression

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Generalize MGPU grid construction to any power-of-two number of GPUs

* Start support for more than two GPUs in key merge

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Flip flop storage

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Generalize concurrent leftIntervals/rightIntervals usage

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix median search

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix rebalancing

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix recursive MGPU merge

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Update TODO

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add guards for zero-tile GPUs

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix race condition across different merge levels (and memory leak in corresponding tests)

* Fix memory leaks in PointsToGrid tests

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix leapfrogging across recursion levels

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Parallelize kernel dispatch for different levels

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Feature/fix get set methods

* minor cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* fixed issue in get/set random access methods

Signed-off-by: Ken <ken.museth@gmail.com>

* cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* deleted white space

Signed-off-by: Ken <ken.museth@gmail.com>

* cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* added unit tests

Signed-off-by: Ken <ken.museth@gmail.com>

* improved unit tests

Signed-off-by: Ken <ken.museth@gmail.com>

* improved unit tests

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add optimizations to MGPU PointsToGrid

* Add timer for MGPU PointsToGrid

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Avoid blocking kernel launches due to cudaFree

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix overlapping of copies

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Pipeline pointsPerVoxelPrefix sum

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Cleanup and optimization

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Further improve pipelining

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Remove unnecessary sync point

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Revert "Remove unnecessary sync point"

This reverts commit f01b36ad852f32ec65517b3baa08ea267d5edc2d.

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Switch to GPU sync

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix event destruction race condition and reduce host thread latency

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Revert timer add

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* More fine-grained pipelining

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix event wait/destruction race condition

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Improve points to grid

* minor cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* minor improvements to nanovdb::tools::cuda::PointsToGrid

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix Windows warnings

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Force set _WIN32 flag

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Move away from deprecated 20.04 runners

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: jjtParadox <jturner@4nsi.com>
Signed-off-by: Jeff Lait <jlait@andorra.sidefx.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>
Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: jjtParadox <jturner@4nsi.com>
Co-authored-by: Jeff Lait <jlait@andorra.sidefx.com>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>
Co-authored-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Nick Avramoussis <4256455+Idclip@users.noreply.github.com>

* Templatize buffer type for distributed points to grid (#403)

* Templatize buffer type for distributed points to grid

* Add missing template parameter

* Add pool

---------

Signed-off-by: jjtParadox <jturner@4nsi.com>
Signed-off-by: Jeff Lait <jlait@andorra.sidefx.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>
Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: jjtParadox <jturner@4nsi.com>
Co-authored-by: Jeff Lait <jlait@andorra.sidefx.com>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>
Co-authored-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Nick Avramoussis <4256455+Idclip@users.noreply.github.com>
2025-05-14 13:03:06 -04:00
Matthew Cong 566f6c6d6c Templatize buffer type for distributed points to grid (#403)
* Templatize buffer type for distributed points to grid

* Add missing template parameter

* Add pool
2025-05-14 08:43:45 -07:00
Mark HarrisandJonathan Swartz ff9f4a5532 Fix get_torch.cmake for pytorch packages without libtorch_python.so symlink (#404)
* Make get_torch.cmake robust to missing conda symlink
* Fix setting `nvtx3_dir` to help torch find nvtx3 installed in fvdb build

Signed-off-by: Mark Harris <mharris@nvidia.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Jonathan Swartz <jonathan@jswartz.info>
2025-05-14 20:37:48 +10:00
Mark HarrisandJonathan Swartz a06dc77fc5 Add clangd configuration for vscode (#382)
Add clangd configuration, docs and vscode settings to use it.

Signed-off-by: Mark Harris <mharris@nvidia.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Jonathan Swartz <jonathan@jswartz.info>
2025-05-14 11:29:39 +10:00
Francis Williams d196ae04ea Permute dimensions of SH coefficients to have gaussians first (#317)
* split sh into two files

move spherical harmonics autograd into its own file

small refactor

factor out jagged spherical harmonics

split jagged spherical harmonics in autograd

cleanup

cleanup

even more cleanup

remove camera dim from sh coeffs

split out shN and sh0

* more cleanup

* no more separate jagged op

* rm old jagged kernel

* don't hardcode float32

* tests

* more tests

* typo fix

* whitespace

* permute SH axes in kernel

* n first

* fix naninfmask tests

* fix merge issues

Signed-off-by: Francis Williams <francis@fwilliams.info>

* clang format

Signed-off-by: Francis Williams <francis@fwilliams.info>

* fix gsplat tests

Signed-off-by: Francis Williams <francis@fwilliams.info>

* comments

Signed-off-by: Francis Williams <francis@fwilliams.info>

* fix PR comments

Signed-off-by: Francis Williams <francis@fwilliams.info>

---------

Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-05-12 13:16:56 -04:00
bbartlett-nv cd29610ade Multi-GPU Splat Training (#380)
* wip split script

* working split, but grows beyond target region

* first working split training

* working multi-gpu script

* refactor into sandbox

* adding dockerfile for 3dgs

* adding script to downsize colmap images

* updates after multi-gpu testing

* updates after testing

* refactor back to project and remove sandbox

* merge with upstream feature branch

* added docs more testing cleanups

* formatting / license updates

* updates to ortho script to address comments
2025-05-09 12:36:05 -04:00
380a8d698c Merge latest master into develop (#394)
* Avoid unnecessary and expensive getLastWriteTime calls for memory mapped files on windows

Signed-off-by: jjtParadox <jturner@4nsi.com>

* Add [[nodiscard]] to confusing math functions.

Signed-off-by: Jeff Lait <jlait@andorra.sidefx.com>

* Fix missing OpenVDB dependency for NanoVDB-only build

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* improved change notes and updated version number

* improved change notes and updated version number

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* improved documentation

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Separate PointsToGrid kernel lambdas into functors

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add missing iomanip include for latest GTest

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* improved nanovdb_convert

* improved nanovdb_convert

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* fixed typo

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* added unit tests

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* review feedback

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix format strings and Windows conversion warning

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix ODR violation due to anonymous namespace in SignedFloodFill

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Minor fix to unit-test of NanoVDB

* Minor fix to unit-test of NanoVDB

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* minor change

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add DeviceMesh and (optional) NCCL CMake Support

* Add device mesh

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Restore current device in constructor and destructor

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Enable range-based for loops

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Use structured bindings inside for loop

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Remove uneeded accessors and modularize functions

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Statically initialize entry point

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Encapsulate

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Switch to lightweight wrapper class for DeviceNodes vector

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Cleanup

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Minor changes needed for DistributedPointsToGrid

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add RAII DeviceGuard

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* More cleanup

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Clean up NCCL dependency

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Remove parallel_for to modularize

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Implement and test move constructor/assignment

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add docs

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Expand test and documentation

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Move non-reinit comment to function description

Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>

* Switch to using aliases and modern type names

Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>

* Use size_type instead of int for return

Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>

* Separate out DeviceGuard and clean up API

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Switch to DeviceMesh for MGPU sparse convolution example

* Convert MGPU convolution example to use DeviceMesh

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix typos and clarify

Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>

* Check for errors and switch to for_each

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add missing const variant of deviceData(device)

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add missing inline for DeviceMesh functions

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add missing NCCL ifdef

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix invalid free in UnifiedBuffer move constructor

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Change UnifiedBuffer prefetch and advise to be const

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix missing relocatable device code flag

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix namespace for AbsDiff and RelDiff ostream overloads

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix out-of-bounds device id

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix TBB template deduction in C++20

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fixed issue in signedfloodfill

* Fixed issue in signedfloodfill

Signed-off-by: Ken <ken.museth@gmail.com>

* partially addressed review comments

Signed-off-by: Ken <ken.museth@gmail.com>

* added RootData::TileIterator

Signed-off-by: Ken <ken.museth@gmail.com>

* cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* fixed assert bug

Signed-off-by: Ken <ken.museth@gmail.com>

* snapshot

Signed-off-by: Ken <ken.museth@gmail.com>

* fixed bug and improved unit-test

Signed-off-by: Ken <ken.museth@gmail.com>

* improved unit test and documentation

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix DeviceMesh.h header to be self-contained

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add missing constructor for C++20 used in AddBlindData.cuh

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* removed dead code and updated the minor version number

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix class-memaccess warning due to non-trivial copy-assignment in GridBlindMetaData

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* major improvements to GridBlindMetaData

* major improvements to GridBlindMetaData

Signed-off-by: Ken <ken.museth@gmail.com>

* minor changes to fix clang issue

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* minor simplification in CreateNanoGrid

* minor simplification in CreateNanoGrid

Signed-off-by: Ken <ken.museth@gmail.com>

* fixed typo

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* introducing new magic numbers for grids and files

* introducing new magic numbers for grids and files

Signed-off-by: Ken <ken.museth@gmail.com>

* removed two redundant magic numbers

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* fixed typos and improved documentation in NanoVDB

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Consolidate DeviceGuard into DeviceMesh header

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add distributed implementation of PointsToGrid

* Add distributed implementation of PointsToGrid

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix race condition in merge and single GPU case

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix kernel/cudaFree race conditions

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add DistributedPointsToGrid unittest

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Clean up example

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Use range-based for loops

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Use structured bindings inside for loop

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix Windows build and some warnings

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Update for refactored DeviceMesh

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix copyright/include

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add parallelForEach helper

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Templatize kernels to align with CUB better

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Shorten TemporaryDevicePool to TempDevicePool

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Refactor to use TempDevicePool

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Switch to class

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Parallelize kernel dispatch in build

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix race condition wrt to root node processing

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Speed up synchronization

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add comments and fix deprecated TransformInputIterator

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Address review comments

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Address more review comments

* Add more cudaCheck calls and restrict EqualityIndicator type

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix license identifiers and remove unused code

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add missing include

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: = <=>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* revert deprecation warning

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* minor cleanup

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Silence CMake warning about FindBoost module deprecation

* Silence CMake warning about FindBoost module deprecation

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix typo

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Adding /bigobj to TestNanoVDB.cu compilation options to fix Windows build failures

* Adding /bigobj to TestNanoVDB.cu compilation options to fix Windows build failure
---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Suppress stringop-overflow warning due to GCC 11 regression

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Generalize MGPU grid construction to any power-of-two number of GPUs

* Start support for more than two GPUs in key merge

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Flip flop storage

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Generalize concurrent leftIntervals/rightIntervals usage

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix median search

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix rebalancing

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix recursive MGPU merge

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Update TODO

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add guards for zero-tile GPUs

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix race condition across different merge levels (and memory leak in corresponding tests)

* Fix memory leaks in PointsToGrid tests

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix leapfrogging across recursion levels

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Parallelize kernel dispatch for different levels

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Feature/fix get set methods

* minor cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* fixed issue in get/set random access methods

Signed-off-by: Ken <ken.museth@gmail.com>

* cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* deleted white space

Signed-off-by: Ken <ken.museth@gmail.com>

* cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* added unit tests

Signed-off-by: Ken <ken.museth@gmail.com>

* improved unit tests

Signed-off-by: Ken <ken.museth@gmail.com>

* improved unit tests

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add optimizations to MGPU PointsToGrid

* Add timer for MGPU PointsToGrid

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Avoid blocking kernel launches due to cudaFree

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix overlapping of copies

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Pipeline pointsPerVoxelPrefix sum

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Cleanup and optimization

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Further improve pipelining

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Remove unnecessary sync point

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Revert "Remove unnecessary sync point"

This reverts commit f01b36ad852f32ec65517b3baa08ea267d5edc2d.

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Switch to GPU sync

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix event destruction race condition and reduce host thread latency

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Revert timer add

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* More fine-grained pipelining

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix event wait/destruction race condition

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Improve points to grid

* minor cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* minor improvements to nanovdb::tools::cuda::PointsToGrid

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix Windows warnings

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Force set _WIN32 flag

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Move away from deprecated 20.04 runners

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: jjtParadox <jturner@4nsi.com>
Signed-off-by: Jeff Lait <jlait@andorra.sidefx.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>
Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Ken <ken.museth@gmail.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: jjtParadox <jturner@4nsi.com>
Co-authored-by: Jeff Lait <jlait@andorra.sidefx.com>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>
Co-authored-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Nick Avramoussis <4256455+Idclip@users.noreply.github.com>
2025-05-07 16:36:22 -07:00
Matthew Cong 2ba8712023 Clean up API by switching to CoordBBox (#397) 2025-05-07 16:36:05 -07:00
Mark Harris 9cec4f1934 Update the test data commit hash to the latest (#396)
* Update test data commit hash
---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-05-07 16:10:22 +12:00
Mark Harris 652b7bff97 Add multicamera rasterization test and writePNG utility (#393)
* Add new image saving test utility and create fvdb_test_utils static lib
* Add `gtests` and `benchmarks` args to build.sh
* Add multicamera rasterization test
---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-05-07 01:15:41 +00:00
Matthew Cong 34996d0f0b Revert "Replace binary search for jidx computation with nested kernels (#375)" (#392)
This reverts commit 378b08b84d.
2025-05-06 15:50:37 -07:00
Jonathan Swartz 39d2c17c89 Merge pull request #2040 from swahtz/fvdb_update
fvdb update
2025-05-01 16:16:31 +12:00
Matthew Cong dd8862cca3 Add venv docs and update build requirements file (#373) 2025-05-01 14:57:28 +12:00
Francis Williams 0d79d8d35c Rewrite colmap dataset (#388)
* rewrite colmap dataset

Signed-off-by: Francis Williams <francis@fwilliams.info>

* named args

Signed-off-by: Francis Williams <francis@fwilliams.info>

---------

Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-04-30 17:25:33 +00:00
Francis Williams e97d916284 remove stray cerr (#389) 2025-04-30 13:05:06 -04:00
Matthew Cong 378b08b84d Replace binary search for jidx computation with nested kernels (#375)
* Replace binary search for computing jidx with nested kernels

* Cleanup for CPU implementation
2025-04-30 09:21:35 -04:00
Jonathan Swartz c31b75ce69 Fix bug where the maximum radius was not being calculated across all cameras in computeGradientState (#385)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-04-30 09:12:56 -04:00
Jonathan Swartz 76ad1c97d4 3dgs: Replace ProjectBackwards GLM math use with NanoVDB Math (#344)
This remove the last vestiges of glm usage from our code by refactoring any Gaussian Projection Backwards code to use nanovdb::math classes and to cleanup any style/naming conventions on code that is refactored. To that end, several themes of work in this PR include:

Removing GLM from our build infrastucture
Transitioning the code in GaussianProjectedBackward.cu & GaussianProjectedJaggedBackward.cu to using nanovdb::math in place of glm
Adding functionality to nanovdb::math to support a few needed missing operators
Removing GaussianTypes.cuh which was used to define glm types for Projection
Consolidated any remaining, needed warp-prefix functions from GaussianTypes.cuh and GaussianVectorTypes.cuh into GaussianWarpUtils.cuh
Refactored functions in GaussianUtils.cuh to return their calculated results instead of directly modifying output parameters where possible. The return values of these functions is then used by the calling code for modifying variables in the calling code. For instance, instead of add_blur_vjp adding blur to a parameter passed by reference, generateBlurVectorJacobianProduct now builds a nanovdb::Math::Mat2 which is used by the calling code.
Add docstrings to GaussianUtils.cuh
Moving snake_case variables/functions to camelCase and being more descriptive in their naming
Making variable types and names consistent between related functions (forward/backwards)
Fixing worldToCamMatrices being mistakenly called camToWorldMatrices

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-04-30 00:17:00 +00:00
Jonathan Swartz 2878a819d8 3dgs Backward Projection Test Fix (#372)
* Debugging

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Clarifying comments

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Fix logic in computeGradientState
Fixed appropriate sizing of the tensors it writes to

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Change the computeGradientState loops over C*N to save on some multiplications

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* backward test checkpoint

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Revert BackwardTest rework

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* update test data tag

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* cleanup

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-04-29 20:41:09 +12:00
Christopher Horvath 27a59587a5 Added a fvdb devcontainer (#381)
* Added a fvdb devcontainer so that VSCode and Github Codespaces can run inside the docker environment automatically

Signed-off-by: Christopher Horvath <blackencino+git@gmail.com>

* Improved the language in the comments

Signed-off-by: Christopher Horvath <blackencino+git@gmail.com>

* Made suggested configuration changes from VSCode copilot

Signed-off-by: Christopher Horvath <blackencino+git@gmail.com>

---------

Signed-off-by: Christopher Horvath <blackencino+git@gmail.com>
2025-04-28 11:38:41 -04:00
Matthew Cong 5dc696e88c Use PyTorch's pybind11 instead of source checkout (if possible) (#359) 2025-04-23 19:27:18 -07:00
Jonathan Swartz 4665c441a3 Merge pull request #2033 from swahtz/fvdb_update 2025-04-24 12:27:12 +12:00
Jonathan Swartz 5592b5fca8 Merge branch 'feature/fvdb' into fvdb_update
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-04-24 12:22:48 +12:00
Jonathan Swartz e5c380d127 Merge pull request #371 from matthewdcong/develop_fvdb_merge
Merge latest develop into feature/fvdb
2025-04-24 12:00:36 +12:00
Matthew Cong 360513e139 Merge remote-tracking branch 'upstream/develop' into feature/fvdb 2025-04-23 14:51:58 -07:00
Mark Harris 923be735a0 Refactor GaussianRasterizeForward (#308)
Similar to #231 this PR refactors the forward pass of gaussian rasterization.

    Adds a new Gaussian structure for use in both passes
    Adds a DeviceArgs class that simplifies configuring and passing arguments for the kernel, and contains the method volumeRenderTileForward which operates on the class data directly.
    Updates code for our coding style standard (e.g. camel case names)
    Renames "colors" to "features" everywhere to better match high-dimensional rendering capabilities.
    Removes redundant "pixel" and "perPixel" prefix/suffixes from some variables.

Signed-off-by: Mark Harris mharris@nvidia.com

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-04-24 07:48:51 +10:00
Matthew Cong 68160beb58 Merge pull request #370 from matthewdcong/master_develop_merge_part2
Merge latest master into develop
2025-04-23 14:47:29 -07:00
Matthew Cong 3ff8a4bb17 Squashed commit of the following:
Merge latest master into develop (squashed to avoid duplicates)
2025-04-23 14:41:05 -07:00
Matthew Cong 050ac875b5 Merge pull request #369 from matthewdcong/master_develop_merge_part1
Merge upstream commits from ASWF master into develop
2025-04-23 14:36:41 -07:00
Matthew Cong 09b77cb8ba Merge branch 'master' into develop 2025-04-23 14:33:17 -07:00
Mark Harris eadf8e38a1 Add a CMakeLists for standalone caching of fvdb_test_data (#362)
This adds a CMakeLists.txt that simply uses CPM to download / cache the FVDB test data if it is not already in the CPM_SOURCE_CACHE.

This is run by build.sh when running ctest (build.sh ctest).

Note that the CPM_SOURCE_CACHE needs to be set to the same value as set when building the gtests (or unset on both).


Signed-off-by: Mark Harris <mharris@nvidia.com>

* Add CPM_SOURCE_CACHE to env in fvdb-gtests GHA job

Signed-off-by: Mark Harris <mharris@nvidia.com>

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-04-23 15:49:49 +10:00
Francis Williams 356efd6d00 Refactor projection forward pass (#334)
Refactors forward pass of projection to use tensor accessors and wrap kernel call in a class.
Also load cameras into shared memory which should be more efficient than having each thread load the same camera.

Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-04-23 15:28:11 +10:00
Jonathan Swartz c2d6ea99ed 3dgs projection tests (#356)
Adding gtests for the 3D Gaussian Splatting Projection code using the garden scene.

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-04-22 22:34:11 +00:00
Christopher Horvath 91846684b4 Moved docker troubleshooting into a separate markdown file (#360)
* Moved docker troubleshooting into a separate markdown file

Signed-off-by: Christopher Horvath <blackencino+git@gmail.com>

* Contents of docker_readme updated.

Signed-off-by: Christopher Horvath <blackencino+git@gmail.com>

---------

Signed-off-by: Christopher Horvath <blackencino+git@gmail.com>
2025-04-22 13:05:23 -07:00
Matthew Cong bc07dba8c5 Merge pull request #354 from matthewdcong/develop_fvdb_merge
Merge latest develop into feature/fvdb
2025-04-21 22:18:21 -07:00
Mark Harris c678006f6e Use object libraries to reduce recompilation of bindings and tests. (#349)
Use intermediate object libraries to reduce recompilation of gtests when modifying libfvdb sources.

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-04-22 14:40:49 +10:00
Christopher Horvath 4fba75b70d Added WSL2/Ubuntu docker troubleshooting to the README (#355)
Signed-off-by: Christopher Horvath <blackencino+git@gmail.com>
2025-04-21 20:39:45 -07:00
Matthew Cong 8607b78e4d Fix calling host function from hostdev function warning (#351) 2025-04-21 19:14:39 -07:00
Matthew Cong 9d3efb8a21 More updates 2025-04-18 11:51:37 -07:00
Matthew Cong 695729a0c0 Update FVDB accessors 2025-04-18 09:43:48 -07:00
Matthew Cong 79f732ad1a Merge branch 'develop' into feature/fvdb 2025-04-18 08:46:47 -07:00
Francis Williams 48d68e9f60 unfuck git (#312)
Signed-off-by: Francis Williams <francis@fwilliams.info>
2025-04-17 17:56:08 -04:00
Matthew Cong 97472f6ef1 Minor changes to GridBatchImpl in preparation for Unified Memory support (#341)
* Switch mHostMetadata to raw pointer in preparation for unified memory

* Set GPU metadata via kernels

* Add missing device guard

* Insert newline

* Use kernels for setFineTransform and setCoarseTransform

* Fix GridBatchImpl for empty grid

* More cleanup

* Move indexInternal to .cu file

* Add missed file
2025-04-17 11:24:09 -07:00
Mark Harris afd31e5509 Add fvdb-dev as CODEOWNERS for the fvdb directory (#346)
Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-04-17 14:35:14 +12:00
Matthew Cong e8a28fe1fb Merge pull request #343 from matthewdcong/upstream_aswf_merge
Merge latest master into develop
2025-04-16 16:34:45 -07:00
Matthew Cong 753d98fb7d Merge branch 'master' into upstream_aswf_merge 2025-04-16 14:06:19 -07:00
Jonathan SwartzandMark Harris 3b6880d96d CI: gtest build and run setup (#336)
This PR sets up running our gtest build targets as part of our automated fVDB CI actions.

First, all the gtests are built as part of the library & wheel building and the whole build directory is uploaded as an artifact separately of the wheel.
In a separate task on a CI machine with a GPU, the tar'd build directory is downloaded. The build directory is un-tar'd and the gtests are run normally with ./build.sh ctest
Fixes #172
Fixes #173

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Mark Harris <mharris@nvidia.com>
2025-04-15 16:51:06 +10:00
Matthew Cong 535923f2be Re-enable NumPy 2 support (#338)
Previously we pinned NumPy to versions prior to v2.0.0 (i.e. v1.x.x) due to the lack of NumPy 2 ABI support in point-cloud-utills. The latest version of point-cloud-utils now supports the NumPy 2 ABI so we can relax this pinning.

PyTorch v2.6.0 requires NumPy v2.x.x and it's likely that slightly earlier versions also built with NumPy v2.x.x as well.
2025-04-15 15:51:40 +12:00
Matthew Cong 150c901bf8 Remove StringOrListOfStrings in favor of function overloads (#339)
* Remove StringOrListOfStrings in favor of function overloads

* Replace default constructor with nullopt

* Fix formatting

* Another formatting fix
2025-04-14 20:44:43 -07:00
Matthew Cong fbd61b1b4b Fix missing device ordinals in examples (#337) 2025-04-14 20:14:13 -07:00
Matthew CongandJonathan Swartz fa12f4e5ce Remove TorchDeviceOrString in favor of stricter function overloads (#330)
* Remove TorchDeviceOrString in favor of stricter function overloads

* Small fixes to device index

* Fix formatting

* Remove trailing whitespace

* Add missing binding

* Small fixes

* Switch from torch::optional to std::optional

* Fix missing device ordinal

* Update stub file

* Format

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Jonathan Swartz <jonathan@jswartz.info>
2025-04-14 14:26:02 -07:00
Matthew Cong a5380f30c5 Add support for editable PyTorch installs (#333)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-04-14 13:23:43 -07:00
Sérgio AgostinhoandJonathan Swartz 1225b9c798 fVDB: Promote SyncBatchNorm to a first-class layer (#322)
Promotes the SyncBatchNorm layer to a first-class citizen.
Addresses the issue with the original implementation where checkpoints created with SyncBatchNorm were incompatible with vanilla BatchNorm, which is a behavior PyTorch natively supports.

---------

Signed-off-by: Sérgio Agostinho <sagostinho@nvidia.com>
Co-authored-by: Jonathan Swartz <jonathan@jswartz.info>
2025-04-14 05:33:21 +00:00
Jonathan Swartz f290b512ca CI Fix Benchmarks job (#332)
* Fix benchmark task branch checkout failure

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-04-14 16:20:12 +12:00
Jonathan Swartz 9101f1dfdf Renamed gsplat headers that had prefix 'Gsplat' or had no mention of Gaussian to 'Gaussian*' prefix (#326)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-04-11 10:55:02 -04:00
Matthew Cong aa3e269b6d More cleanup for TorchDeviceBuffer (#320)
* More cleanup for TorchDeviceBuffer

* Format

* More cleanup
2025-04-11 00:33:32 -07:00
Jonathan Swartz 9ab8037226 CI: Fix merging reference branch to PR branch before building (#329)
* Added fvdb CI build steps to merge feature/fvdb branch in PR to PR branch before testing


---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-04-11 18:27:36 +12:00
Jonathan Swartz 868ef3a1eb Add global cmake download timeout and CPM_SOURCE_CACHE to fvdb CI(#328)
---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-04-11 17:18:29 +12:00
Jonathan SwartzandMark Harris 24a942d5b6 Replaced use of glm with nanovdb math in projectForwardKernel (#238)
Implemented overloaded fucntions in GsplatUtils, Projection, etc. to not break other pieces of pipeline that still use glm
Implemented various Matrix classes in nanovdb::math that were required.

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Mark Harris <mharris@nvidia.com>
2025-04-11 02:55:15 +00:00
Mark HarrisandJonathan Swartz 40b1e6a8de Build fVDB module using scikit-build-core / CMake (#264)
Switched from using the PyTorch built-in setuptools-based cpp_extension module to a scikit-build-core + CMake build system. Additionally this includes the introduction of a new build script, updates to the Dockerfile, and updates to the documentation.

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Jonathan Swartz <jonathan@jswartz.info>
2025-04-11 12:32:22 +12:00
Ken Museth f18d30ca0d Improve points to grid (#318)
* minor cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* minor improvements to nanovdb::tools::cuda::PointsToGrid

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
2025-04-09 11:15:20 -07:00
Matthew Cong 87d053396f Fix missing declaration in TorchDeviceBuffer (#319)
* Fix missing declaration in TorchDeviceBuffer
2025-04-09 00:33:25 +00:00
Francis Williams 8d876b90c8 fix grid stride loops (#316) 2025-04-08 15:51:57 -04:00
Matthew Cong e9277d5fe8 Refactor TorchDeviceBuffer to support an arbitrary single torch::Device (#302) 2025-04-04 13:47:54 -07:00
e05dd592bb Use new Gaussian Splat C++ API from Python (#290)
Implements a new GaussianSplat3d class in C++ that also tracks appropriate gradient state for optimization.
Clean up the Gaussian splatting code in python. We now only depend on Pytorch!
Clean up the optimizer code a lot by moving gradient state tracking to C++ class.
Move refinement part of optimization code to training script since it's clearer. The whole algorithm can now be read in one file.
Add a bunch of new error checking.

Still outstanding:

    Implement pruning/splitting/duplication in CUDA (Going to be a seperate PR)

-----
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Jonathan Swartz <jonathan@jswartz.info>
2025-04-03 14:22:22 +11:00
Matthew Cong e217cbe6f2 Add patches to external dependencies required for CMake 4 (#305) 2025-04-02 13:12:26 -07:00
Matthew Cong 836d4a14d8 Merge pull request #306 from matthewdcong/upstream_aswf_merge
Merge in latest master into develop
2025-04-02 10:58:45 -07:00
Matthew Cong dcf8724324 Merge branch 'master' into develop 2025-04-02 09:43:35 -07:00
Matthew Cong e281e93a97 Merge pull request #300 from matthewdcong/upstream_aswf_merge
Merge latest master into develop
2025-03-31 14:07:27 -07:00
Matthew Cong 987cf322d6 Merge branch 'master' into develop 2025-03-31 12:16:02 -07:00
Mark Harris f87be92caa Add Doxygen docs for Gaussian Splatting C++ functions (#278)
Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-03-26 14:01:01 +11:00
Jonathan Swartz d977b72a4a fVDB: Add transform metadata when saving non-index grids (#295)
* Add transform metadata to non-index grids we're saving
fixes #32
* Fix data grid saving to be able to save non-uniform voxel scales
Add tests
---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-03-24 11:06:45 +13:00
Mark Harris c877c9e7f0 Unit tests for GaussianForwardRasterization (#289)
New GaussianRasterizeForward unit tests.

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-03-18 05:46:59 +11:00
Jonathan Swartz d32be67783 Update credentials version
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-03-17 20:23:43 +13:00
Jonathan Swartz b94d3e1a96 added fvdb aws codebuild configurations
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-03-17 20:20:47 +13:00
Matthew Cong 74d46e5601 Add optimizations to MGPU PointsToGrid (#283)
* Add timer for MGPU PointsToGrid

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Avoid blocking kernel launches due to cudaFree

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix overlapping of copies

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Pipeline pointsPerVoxelPrefix sum

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Cleanup and optimization

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Further improve pipelining

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Remove unnecessary sync point

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Revert "Remove unnecessary sync point"

This reverts commit f01b36ad852f32ec65517b3baa08ea267d5edc2d.

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Switch to GPU sync

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix event destruction race condition and reduce host thread latency

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Revert timer add

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* More fine-grained pipelining

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix event wait/destruction race condition

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-03-13 15:49:17 -07:00
Ken Museth 9f52c46ad0 Feature/fix get set methods (#288)
* minor cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* fixed issue in get/set random access methods

Signed-off-by: Ken <ken.museth@gmail.com>

* cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* deleted white space

Signed-off-by: Ken <ken.museth@gmail.com>

* cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* added unit tests

Signed-off-by: Ken <ken.museth@gmail.com>

* improved unit tests

Signed-off-by: Ken <ken.museth@gmail.com>

* improved unit tests

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
2025-03-13 13:21:24 -07:00
Francis WilliamsandFrancis Williams 4e376ff930 C++ Class for GaussianSplat3d (#280)
---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Francis Williams <francis@fwilliams.info>
2025-03-12 12:34:29 +13:00
Matthew Cong 8edb137baa Fix race condition across different merge levels (and memory leak in corresponding tests) (#274)
* Fix memory leaks in PointsToGrid tests

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix leapfrogging across recursion levels

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Parallelize kernel dispatch for different levels

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-03-05 12:52:38 -08:00
Francis Williams 82aab4082c Wide channel width in GaussianRasterizeBackwards (#262)
* move atomic adds to helper

* add fixture and basic cat test

* move gradient contribution to its own function

* renaming

* remove color struct to allow different shared memory channels

* channel ranges for atomic add and shared memory color dim arg

* channel range for color fetching into shared memory and split color and gaussian shared memory fetching into two functions

* size_t

* add template parameter for shared channels

* stub for chunked

* checkpoint

* checkpoint

* rm prints

* some constexpr

* more loop unrolling

* more unrolling

* fix

* more tests

* rm old codepath

* start renaming

* more renames

* rename

* rename all members to use m prefix

* rename template args

* more renames

* cpu throws test

* error checking and bring back all dimensions

* small cleanup
2025-03-05 00:34:15 +00:00
Matthew Cong 270d2238b8 Generalize MGPU grid construction to any power-of-two number of GPUs (#270)
* Start support for more than two GPUs in key merge

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Flip flop storage

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Generalize concurrent leftIntervals/rightIntervals usage

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix median search

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix rebalancing

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix recursive MGPU merge

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Update TODO

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add guards for zero-tile GPUs

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-03-04 12:42:17 -08:00
Matthew Cong 0e8e1966b7 Merge pull request #271 from matthewdcong/sync/master_to_develop
Merge latest ASWF master into develop
2025-03-04 12:13:18 -08:00
Matthew Cong 84c258562e Merge branch 'master' into develop 2025-03-04 11:10:29 -08:00
12276a7afc Unit tests for GaussianSplatBackward (#259)
* move gaussian accumulation into a function. Move delta and sigma into methods on Gaussians
* Added a simple test to compare GaussianRasterizeBackward against some saved data.
* Fixed a bug where packed was uninitialized.
* Fixed another latent bug where we were getting a pointer from a tensor accessor. This might fail if that accessor is strided in the last dimension.
* Populate compile time definition of path to test data in C++ unit tests.

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Mark Harris <harrism@users.noreply.github.com>
Co-authored-by: Jonathan Swartz <swahtz@users.noreply.github.com>
2025-02-26 16:33:33 +11:00
Mark Harris e2b295a197 CMake configuration for building libfvdb (#237)
* Add CMakeLists and dependency getters for libfvdb

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Add cudnn to dev conda environment

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Fix compilation errors uncovered when building with cmake

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Add build dir to gitignore

* cudnn_fe --> cudnn_frontend, fix cutlass

* Revert adding cudnn to conda dev_environment

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Build gtests and gbenchmarks from main cmakelists

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Fix tensor dimension corruption by scoping IntArrayRef's vector lifetime

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Add Cutlass patch

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Add cpm source cache.

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Fix CPM source cache and make it overrideable

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Add blosc and missing compilation settings.

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Since we require CUDA 12+, we can drop the cutlass patch.

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Fix blosc build options

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Fix up compiler options and properly scope cmake options, and generate compile_commands.json

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Remove commented out lines.

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Add install rules

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Whitespace

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Make get_torch.cmake a bit more robust (more informative error on import failure)

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Add missing typename keyword (pre-C++20)

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Update to C++20, which for some reason makes us need the cutlass patch again, so enable it.

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-02-26 07:51:31 +11:00
Jonathan Swartz bafe8fe86e add CMAKE_CUDA_ARCHITECTURES to nanovdb weekly tests (#257)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-02-25 08:56:56 -08:00
bbartlett-nvandFrancis Williams e1cf99fbed adding in scripts for ortho, mesh, mask training etc (#247)
* adding in scripts for ortho, mesh, mask training etc

* rm dep on colmapparser in gaussiansplatting

* move colmap dataset out of core fvdb into project

* update imports in gaussian splatting project

* revert dev env

* autoformat and rename T1 -> normalizat_transform

* clean up normalization argument

* print to warning

* rm print

* bring back sorting

---------

Co-authored-by: Francis Williams <francis@fwilliams.info>
2025-02-25 11:36:52 -05:00
Francis Williams 794820311b Refactor Backward Gaussian Rasterization and remove glm (#231)
* remove glm dep and use struct

* rename macros

* small cleanup and const some things

* move volume render backward to helper function

* checkpoint

* rename S -> ScalarType

* pass in warp as variable

* camel case and type cleanup

* S -> ScalarType

* clean up full file

* address PR comments

* rename thread -> blockDim, blocks -> gridDim

* fixed and moved to class

* checkpoint

* checkpoint

* checkpoint

* rename

* cleanup

* checkpoint

* cleanup

* more cleanup

* Fix comment

* handle empty inputs

* fix

* fix error checking
2025-02-22 20:07:44 +00:00
Jonathan Swartz 350febee18 Merge pull request #248 from matthewdcong/fvdb_develop_integ
Merge develop into feature/fvdb
2025-02-21 12:37:39 +13:00
Matthew Cong ebca97a1a2 Merge develop into feature/fvdb
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-02-20 14:09:23 -08:00
Matthew Cong 4f60c64028 Suppress stringop-overflow warning due to GCC 11 regression (#246)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-02-20 13:08:44 -08:00
Mark Harris fb999c49e4 Fix strict aliasing violation in encode_depth, and document helper functions. (#236)
Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-02-19 08:46:26 +13:00
Jonathan Swartz 2a7498f7f7 Merging current develop into feature/fvdb (#234)
---------
Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-02-18 14:22:08 +13:00
Jonathan Swartz c9a06baf88 Adding /bigobj to TestNanoVDB.cu compilation options to fix Windows build failures (#235)
* Adding /bigobj to TestNanoVDB.cu compilation options to fix Windows build failure
---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-02-18 11:33:27 +13:00
Matthew Cong e170b879ae Use cuDNN 9 that's included in PyTorch 2.4 and above (#222)
Add cuDNN include directory and vendor in cuDNN frontend

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-02-18 09:04:43 +13:00
Jonathan SwartzandFrancis Williams 8c6c449898 Change status_checks to use self-hosted runner (#226)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Francis Williams <fwilliams@users.noreply.github.com>
2025-02-15 00:07:35 +00:00
Matthew CongandKen Museth 1cc5aa13ee Silence CMake warning about FindBoost module deprecation (#230)
* Silence CMake warning about FindBoost module deprecation

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix typo

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
2025-02-14 14:44:54 -08:00
Ken Museth 692cdde46c minor cleanup (#229)
Signed-off-by: Ken <ken.museth@gmail.com>
2025-02-14 13:59:38 -08:00
Ken Museth 0d157c199f revert deprecation warning (#228)
Signed-off-by: Ken <ken.museth@gmail.com>
2025-02-14 12:02:33 -08:00
Mark HarrisandFrancis Williams 7b040d3c67 Gaussian Splatting sparse tile intersection (#213)
* Implementation and tests of dispatchGaussianTileIntersectionSparse (not yet passing)

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Some bug fixes, intermediate changes, still failing tests.

* fix

* Use custom kernel for sparse tile_offsets. Fixes current sparse test failures.

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Make non-sparse tile_offsets look like it used to.

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Refactor dispatchGaussianTileIntersection to reduce repetition between dense and sparse.

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Add a random sparsity test.

Signed-off-by: Mark Harris <mharris@nvidia.com>

* docs and move sparse intersection into a different header

* grid stride loops

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Francis Williams <francis@fwilliams.info>
2025-02-14 14:18:10 -05:00
Matthew Congand= 0783b2a7e1 Add distributed implementation of PointsToGrid (#102)
* Add distributed implementation of PointsToGrid

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix race condition in merge and single GPU case

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix kernel/cudaFree race conditions

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add DistributedPointsToGrid unittest

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Clean up example

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Use range-based for loops

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Use structured bindings inside for loop

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix Windows build and some warnings

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Update for refactored DeviceMesh

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix copyright/include

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add parallelForEach helper

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Templatize kernels to align with CUB better

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Shorten TemporaryDevicePool to TempDevicePool

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Refactor to use TempDevicePool

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Switch to class

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Parallelize kernel dispatch in build

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix race condition wrt to root node processing

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Speed up synchronization

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add comments and fix deprecated TransformInputIterator

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Address review comments

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Address more review comments

* Add more cudaCheck calls and restrict EqualityIndicator type

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix license identifiers and remove unused code

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add missing include

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: = <=>
2025-02-14 08:51:18 -08:00
Matthew Cong 20c22d71e5 Add support for venv builds (#219)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-02-14 00:12:25 +00:00
Matthew Congand= 372e48a430 Defer psutil import which is not specified as a Python requirement (#217)
* Defer psutil import which is not specified as a Python requirement

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix formatting

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: = <=>
2025-02-14 00:02:08 +00:00
Matthew Cong ad63e07a03 Consolidate DeviceGuard into DeviceMesh header (#221)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-02-13 12:40:14 -08:00
Ken Museth 1bb3b12482 fixed typos and improved documentation in NanoVDB (#220)
Signed-off-by: Ken <ken.museth@gmail.com>
2025-02-13 11:46:16 -08:00
Matthew Cong 273d63e2ca Switch AX CI jobs to be weekly (#216)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-02-12 19:03:39 -08:00
Ken Museth 7cdf89202f introducing new magic numbers for grids and files (#210)
* introducing new magic numbers for grids and files

Signed-off-by: Ken <ken.museth@gmail.com>

* removed two redundant magic numbers

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
2025-02-12 13:45:38 -08:00
Matthew Cong 69a7808d4d Remove TBB dependency introduced by GCC (#215)
* Remove TBB dependency introduced by GCC

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix formatting

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-02-12 12:16:33 -08:00
Mark Harris 7f22194e09 Return activeTileMask from computeSparseInfo (#211)
Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-02-11 18:43:10 -05:00
7c04627c24 Add C++ unit tests for GaussianTileIntersection (#203)
* Rename computeSparseInfoTest and add new unit tests for GaussianTileIntersection.

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Fix handling of all-zero-radius splats. Fixes failing unit test.

Signed-off-by: Mark Harris <mharris@nvidia.com>

* spdx header

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Fix the failing BasicIntersectionTest

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Add an explanatory comment about packed mode.

Signed-off-by: Mark Harris <mharris@nvidia.com>

* refactoring

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Add failing test, commented out.

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Separate GaussianFullyFusedProjection*.cu into distinct files (#201)

* separate projection files

* rename some functions

* rename jagged

* fix empty gaussian test, add error checking, and add error checking test

* Rename NanInfMaskTest to match other test file naming

Signed-off-by: Mark Harris <mharris@nvidia.com>

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Francis Williams <fwilliams@users.noreply.github.com>
Co-authored-by: Francis Williams <francis@fwilliams.info>
2025-02-11 18:00:24 -05:00
Francis Williams 813ccdcf6b C++ PLY saving (#188)
* save ply for gaussian splats

* fixes

* filter operation in cuda

* checkpoint

* fixes and python bindings

* end to end test in python

* rm unused imports

* helper function for getting bytes from tensor

* rm thread divergent returns and inline on header only function

* use std generate

* use tensorOpts

* use config array

* const filename

* error message fix

* contiguous

* stylefix

* whitespace

* stylefix2

* grid stride loop and launch bounds

* use stream in sparse info computation (#190)

* Add a Vec2 type to nanovdb math (#166)

* orthographic projection in gaussian splatting

add orthographic flag to depth rendering function

derp

add ortho projection to jagged projection

* small refactor

* add vec2 class to nanovdb math

* fix

* Separate GaussianFullyFusedProjection*.cu into distinct files (#201)

* separate projection files

* rename some functions

* rename jagged
2025-02-11 16:12:45 -05:00
Matthew Cong bb0c4d141f Replace deprecated torch::linalg::inv with equivalent at::linalg_inv (#209)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-02-11 14:15:13 -05:00
Francis Williams 0f3578d60a Separate GaussianFullyFusedProjection*.cu into distinct files (#201)
* separate projection files

* rename some functions

* rename jagged
2025-02-11 10:36:48 -05:00
Ken Museth 35d6eea9d2 minor simplification in CreateNanoGrid (#202)
* minor simplification in CreateNanoGrid

Signed-off-by: Ken <ken.museth@gmail.com>

* fixed typo

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
2025-02-10 17:16:09 -08:00
Francis Williams 91d8bcf3c0 Add a Vec2 type to nanovdb math (#166)
* orthographic projection in gaussian splatting

add orthographic flag to depth rendering function

derp

add ortho projection to jagged projection

* small refactor

* add vec2 class to nanovdb math

* fix
2025-02-10 16:23:13 -05:00
Ken Museth 90ac7aa909 major improvements to GridBlindMetaData (#199)
* major improvements to GridBlindMetaData

Signed-off-by: Ken <ken.museth@gmail.com>

* minor changes to fix clang issue

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
2025-02-10 12:56:39 -08:00
Matthew CongandKen Museth a206519467 Fix class-memaccess warning due to non-trivial copy-assignment in GridBlindMetaData (#147)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
2025-02-07 22:12:29 -08:00
Ken Museth b4634c76b8 removed dead code and updated the minor version number (#198)
Signed-off-by: Ken <ken.museth@gmail.com>
2025-02-07 17:11:38 -08:00
Matthew CongandKen Museth 2144e0efd4 Add missing constructor for C++20 used in AddBlindData.cuh (#197)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
2025-02-07 16:26:05 -08:00
Matthew Cong fc2c4b44b6 Fix DeviceMesh.h header to be self-contained (#196)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-02-07 23:34:49 +00:00
Ken Museth 6d54f0e50c Fixed issue in signedfloodfill (#185)
* Fixed issue in signedfloodfill

Signed-off-by: Ken <ken.museth@gmail.com>

* partially addressed review comments

Signed-off-by: Ken <ken.museth@gmail.com>

* added RootData::TileIterator

Signed-off-by: Ken <ken.museth@gmail.com>

* cleanup

Signed-off-by: Ken <ken.museth@gmail.com>

* fixed assert bug

Signed-off-by: Ken <ken.museth@gmail.com>

* snapshot

Signed-off-by: Ken <ken.museth@gmail.com>

* fixed bug and improved unit-test

Signed-off-by: Ken <ken.museth@gmail.com>

* improved unit test and documentation

Signed-off-by: Ken <ken.museth@gmail.com>

---------

Signed-off-by: Ken <ken.museth@gmail.com>
2025-02-07 15:01:11 -08:00
Francis Williams 7f015429ca use stream in sparse info computation (#190) 2025-02-06 11:46:09 -05:00
Matthew Cong 72ffce9949 Fix TBB template deduction in C++20 (#189)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-02-05 16:33:41 -08:00
Matthew Cong 9733b0882e Fix out-of-bounds device id (#180)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-02-05 08:18:02 -08:00
Mark HarrisandFrancis Williams baab90646e Add support for multiple images in sparse gsplat config (#182)
Extends computeSparseInfo() (from #137) to work with a JaggedTensor of pixels to render with outer dimension > 1. In other words, pixels from more than one image.

Adds unit tests to exercise this functionality. Also parameterizes the strided-pixel unit tests to further DRY up the test code.

Collaboration with @swahtz and @fwilliams

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Francis Williams <francis@fwilliams.info>
2025-02-05 19:21:37 +13:00
8c41e2b822 Sparse Gaussian Splatting Configuration (#137)
Closes #60

This pull request adds functionality to compute the configuration tensors for Sparse 3D Gaussian Splatting. These tensors will be used to mask the output of 3DGS when fed a list of sparse pixels to render, and to "regather" the sparse rendered pixels corresponding to the input pixel UV coordinates.

A fairly comprehensive C++ unit test suite is also added.

Build Configuration Updates:
fvdb/.vscode/c_cpp_properties.json: Updated include paths to reflect new dependencies and updated Python version from 3.10 to 3.11.
fvdb/setup.py: Added psutil import and set MAX_JOBS environment variable based on available RAM to control the number of parallel jobs for building. [1] [2]
fvdb/src/cmake/get_fvdb.cmake: Modified to dynamically find the Python version and update the FVDB_BUILD_DIR accordingly.
fvdb/src/tests/CMakeLists.txt: Added compile options for CUDA and C++ in debug mode and configured new unit tests for Gaussian splatting. [1] [2]
Functionality Enhancements:
fvdb/src/detail/ops/gsplat/GaussianSplatSparse.cu: Added new CUDA kernels and functions to implement sparse Gaussian splatting, including detailed comments explaining the algorithm and its components.
fvdb/src/detail/ops/gsplat/GaussianSplatSparse.h: Added header declarations for the new sparse Gaussian splatting functions.

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Matthew Cong <mcong@nvidia.com>
Signed-off-by: Mark Harris <mharris@nvidia.com>
Co-authored-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Francis Williams <francis@fwilliams.info>
2025-02-04 12:07:53 +13:00
Matthew Cong 964c4270d6 Fix namespace for AbsDiff and RelDiff ostream overloads (#177)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-01-30 16:35:17 -08:00
Francis Williams 03cd858bf0 Merge remote-tracking branch 'fwilliams/fw/orthographic_gaussian_splats' into feature/fvdb 2025-01-30 08:51:00 -08:00
Francis Williams ca8fbd1ab4 naming 2025-01-30 07:07:51 -08:00
Francis Williams 3f72e2444a return values out of projection vjp 2025-01-30 07:03:23 -08:00
Jonathan Swartz 1a376f21d0 Moving 3dgs example to projects directory (#175)
* Moved examples/3dgs->projects/3d_gaussian_splatting/training

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Fix up READMEs

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-30 14:53:01 +13:00
Francis Williams 6485bca2db add documentation and move projections to their own header 2025-01-29 16:53:39 -08:00
Francis Williams b47913e604 fix some comments 2025-01-29 16:44:35 -08:00
Matthew Cong 16f190b574 Fix missing relocatable device code flag (#170)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-01-29 13:45:53 -08:00
Francis Williams 9d55e6397a return projection values rather than use output params.
template ortho in jagged projecction
2025-01-29 13:07:15 -08:00
Francis Williams 79a3b3a155 another typo 2025-01-29 08:24:22 -08:00
Francis Williams 1e7ecc2c2a typo fix 2025-01-29 08:20:19 -08:00
Francis Williams b74bad0a3f template ortho for backward 2025-01-29 07:50:55 -08:00
Francis Williams 3ca570de27 fix backward number of return values 2025-01-29 07:50:48 -08:00
Francis Williams 7c5ff85553 fix typo 2025-01-29 07:50:36 -08:00
Jonathan Swartz 470205d833 Removing pull_request trigger from 'Wait for Status Checks' (#163)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-29 17:21:50 +13:00
Jonathan Swartz 7f0b162766 Removing pull_request trigger from 'Wait for Status Checks' (#165)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-29 16:49:03 +13:00
ed220e48d8 Boolean template parameter for orthographic projection.
Co-authored-by: Mark Harris <harrism@users.noreply.github.com>
Co-authored-by: Jonathan Swartz <swahtz@users.noreply.github.com>
2025-01-28 19:17:22 -08:00
Matthew Cong 526a693b43 Change UnifiedBuffer prefetch and advise to be const (#160)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-01-27 23:52:29 +00:00
Matthew Cong 520085852c Fix invalid free in UnifiedBuffer move constructor (#156)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-01-27 22:34:16 +00:00
Francis Williams 500e158f97 small refactor 2025-01-27 14:13:02 -08:00
Jonathan Swartz 6ff3939a05 Fixing build_environment; should be 2.4.1 (#162)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-28 07:54:21 +11:00
Matthew Cong c146d17513 Add missing NCCL ifdef (#164)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-01-28 09:30:39 +13:00
Francis Williams fef302f5ff Revert "orthographic projection in gaussian splatting"
This reverts commit bfe68a0d82.
2025-01-24 13:26:21 -08:00
Francis Williams f38a52d8e6 Revert "add orthographic flag to depth rendering function"
This reverts commit 7803589d9d.
2025-01-24 13:26:08 -08:00
Francis Williams 3673dabd9c Revert "derp"
This reverts commit af38b51cf9.
2025-01-24 13:25:57 -08:00
Francis Williams 00a035affb Revert "add ortho projection to jagged projection"
This reverts commit d800ecfa88.
2025-01-24 13:25:43 -08:00
Francis Williams 589b9c9d16 orthographic projection in gaussian splatting
add orthographic flag to depth rendering function

derp

add ortho projection to jagged projection
2025-01-24 13:24:51 -08:00
Francis Williams d800ecfa88 add ortho projection to jagged projection 2025-01-24 13:21:54 -08:00
Francis Williams af38b51cf9 derp 2025-01-24 12:57:20 -08:00
Francis Williams 7803589d9d add orthographic flag to depth rendering function 2025-01-24 10:19:16 -08:00
Francis Williams bfe68a0d82 orthographic projection in gaussian splatting 2025-01-24 09:42:28 -08:00
Mark Harris 86f2734cd2 Fix conda envs and cmake for unit tests (#153)
* Fix build- and dev- environments and cmake so unit tests will build with conda-forge packages

Signed-off-by: Mark Harris <mharris@nvidia.com>

* Remove no-longer-needed env

Signed-off-by: Mark Harris <mharris@nvidia.com>

---------

Signed-off-by: Mark Harris <mharris@nvidia.com>
2025-01-23 11:38:53 +13:00
Jonathan Swartz 06f003e85a Merge pull request #1997 from swahtz/omni_fvdb_update
fVDB 0.2.1 update
2025-01-23 10:38:01 +13:00
Jonathan Swartz cd5fdeb92a fVDB version bumped to 0.2.1 (#151) (#152)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-22 17:40:25 +13:00
Jonathan Swartz 8145597272 Adding back fix to NanoVDB CI for test failures
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-22 14:49:50 +13:00
Jonathan Swartz 04e3193377 Revert any changes to .github/ made to accomodate CI
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-22 13:21:19 +13:00
Jonathan Swartz 16bfecf4c6 fVDB version bumped to 0.2.1 (#151)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-22 13:15:17 +13:00
Jonathan Swartz eb489a2bdc Remove test env accidentally merged in previous PR (#150)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-22 12:51:20 +13:00
Francis Williams a8e52da53c Fix bug where image cropping wasn't working in Gaussian Splatting 2025-01-22 12:21:48 +13:00
Matthew Cong 7d13055ceb Add missing inline for DeviceMesh functions (#145)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-01-21 09:46:15 -08:00
Francis Williams 2db574080d fix itertools.cycle (#142) 2025-01-21 15:51:32 +13:00
Matthew Cong 2d7bc30af3 Add missing const variant of deviceData(device) (#146)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-01-21 15:48:25 +13:00
Jonathan SwartzandKen Museth a75a507ecb PyTorch, CUDA package sources moved to conda-forge channel (#131)
Moved the recommended build/test/dev environments to use the conda-forge channel as their package source. This has required moving to CUDA 12.0, PyTorch 2.4.1.  Also removed the NKFW tests because they were breaking in the new environment but these tests’ functionality were covered by other unit tests.

closes #121 closes #135
---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
2025-01-21 13:01:25 +11:00
Matthew CongandMark Harris f9a9876ee9 Switch to DeviceMesh for MGPU sparse convolution example (#106)
* Convert MGPU convolution example to use DeviceMesh

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fix typos and clarify

Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>

* Check for errors and switch to for_each

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>
2025-01-16 10:57:19 -08:00
1b7f60fe53 remove tensor conversion on images from dataloader to save on ram useage for large datasets (#129)
Co-authored-by: Francis Williams <fwilliams@users.noreply.github.com>
Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>
2025-01-17 07:50:58 +13:00
Matthew CongandMark Harris 31b5fee9ab Add DeviceMesh and (optional) NCCL CMake Support (#99)
* Add device mesh

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Restore current device in constructor and destructor

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Enable range-based for loops

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Use structured bindings inside for loop

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Remove uneeded accessors and modularize functions

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Statically initialize entry point

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Encapsulate

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Switch to lightweight wrapper class for DeviceNodes vector

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Cleanup

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Minor changes needed for DistributedPointsToGrid

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add RAII DeviceGuard

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* More cleanup

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Clean up NCCL dependency

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Remove parallel_for to modularize

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Implement and test move constructor/assignment

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add docs

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Expand test and documentation

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Move non-reinit comment to function description

Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>

* Switch to using aliases and modern type names

Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>

* Use size_type instead of int for return

Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>

* Separate out DeviceGuard and clean up API

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com>
2025-01-15 13:41:11 -08:00
Mark HarrisandJonathan Swartz d9b35645a3 Add CMake support for unit tests (#133)
* Add basic cmake and an example gbenchmark

* Modify setup.py to exclude benchmarks directory

* gitignore benchmarks build dir

* Make CMake work with conda-forge Torch, and make simple benchmark use Torch.

* Add working C++11 ABI Torch conda environment

* Modularize the cmake.

* Whitespace cleanup

* empty

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Add comment about excluding benchmarks directory

* Modularize python, torch and fvdb dependencies, and add better error checking

* Empty commit to test signing

* Empty commit to test signing

* Add cmake and example for gtest-based C++ unit tests

* remove whitespace change.

* set `files.insertFinalNewLine` to true in workspace settings, fix EOF newlines

* newline at EOF

* Rename ExampleTest

* Add CTest support.

* "benchmarks" --> "tests"

* Fix typo

* Ignore tests directory (oops)

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-15 11:58:10 -05:00
Matthew CongandFrancis Williams da04f647be Create blank issue template (#136)
* Create issue.md

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Update and rename issue.md to task--or--work--item.md

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Update task--or--work--item.md

rm trailing whitespace

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Update and rename task--or--work--item.md to blank--issue

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Rename blank--issue to blank--issue.md

Signed-off-by: Matthew Cong <mcong@nvidia.com>

---------

Signed-off-by: Matthew Cong <mcong@nvidia.com>
Co-authored-by: Francis Williams <fwilliams@users.noreply.github.com>
2025-01-14 14:38:34 -08:00
64f47dec27 Add basic CMake infrastructure for benchmarks (#75)
* Add basic cmake and an example gbenchmark

* Modify setup.py to exclude benchmarks directory

* gitignore benchmarks build dir

* Make CMake work with conda-forge Torch, and make simple benchmark use Torch.

* Add working C++11 ABI Torch conda environment

* Modularize the cmake.

* Whitespace cleanup

* empty

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Add comment about excluding benchmarks directory

* Modularize python, torch and fvdb dependencies, and add better error checking

* Empty commit to test signing

* Empty commit to test signing

* Use `find_library` to find libfvdb.so

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Francis Williams <fwilliams@users.noreply.github.com>
2025-01-14 16:34:27 -05:00
Francis Williams 8f3eaa471a Fw/3dgs lifestyle (#127)
* fix spelling mistake and add log

* quality of life helper
2025-01-14 03:01:13 +00:00
Francis Williams dc887b6509 vscode path settings (#126) 2025-01-13 16:16:57 -05:00
Jonathan Swartz 95166cf5cc Fix docs copyright notice to be OpenVDB contributors (#124)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-10 20:03:53 +00:00
Mark HarrisandJonathan Swartz 1acb6477ed Make setup.py handle case where cudnn directory is created but empty (#50)
Co-authored-by: Jonathan Swartz <2375296+swahtz@users.noreply.github.com>
2025-01-11 08:32:36 +13:00
Jonathan Swartz a4a92bc3fa Wait for status checks also triggered on pushes to pull-request branches (#123)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-11 08:31:13 +13:00
Jonathan SwartzandKen Museth 36f3e0921b CI: Update feature/fvdb from develop (#118)
Updating feature/fvdb branch from current develop branch to pickup the 'enforce-all-checks' action as well as disabled Windows tests setup.

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com>
2025-01-10 17:15:25 +13:00
Jonathan Swartz d2550dba8d Disabling Windows build & nanovdb tests; disabling all tests except for Windows tests in weekly (#120)
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-10 15:43:36 +13:00
Jonathan Swartz 7fdb6283ff black formatting fix for garfield download_example_data (#119)
* black formatting fix for garfield download_example_data
* Removing cv2 import that is only valid for pip opencv package
---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-10 15:09:48 +13:00
Jonathan Swartz d9e5b261e1 Fix status_check directory
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-09 10:49:39 +13:00
Jonathan Swartz e7d5b0c556 Add status check that checks for the pass/fail of only the triggered status checks
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-09 10:49:39 +13:00
Jonathan Swartz 527bf5eae8 Updated README
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-09 04:55:45 +13:00
Jonathan Swartz 7b542ae021 Garfield download_example_data now also downloads colmap data.
dozer_nerfgun_waldo changed to the default dataset

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-09 04:55:45 +13:00
Jonathan Swartz b4ba1cbeab Fix colmap dataset distortion parameters; wrong number of params for simple radial distortion
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-09 04:55:45 +13:00
Jonathan Swartz 99630eb403 Make 3dgs download_example_data executable
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-09 04:55:45 +13:00
Jonathan Swartz 6abaae22dd Add download_example_data script for downloading example colmap data from LERF project used by GARField paper.
Updated README

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-09 04:55:45 +13:00
Jonathan Swartz 8be8734a12 Fix trailing whitespace
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-09 04:55:45 +13:00
Jonathan Swartz d560caea5c Fixing garfield black formatting; adding garfield to SPDX ignore-paths
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-09 04:55:45 +13:00
Jonathan Swartz d2d4e13971 Add a garfield CUDA 12.1 environment
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-09 04:55:45 +13:00
Jonathan Swartz afbd5a9fcc Adding garfield source from research repo
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-09 04:55:45 +13:00
Ken Museth 89979f93f7 Minor fix to unit-test of NanoVDB (#111)
* Minor fix to unit-test of NanoVDB

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* minor change

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
2025-01-07 13:24:51 -08:00
Jonathan Swartz 3f9b956418 Fix BuildDeviceGrid's calls to NanoVDB voxelsToGrid which removed an allocator type as an optional template argument.
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-08 10:04:28 +13:00
Jonathan Swartz 07b3ee731d Attemping to fix fvdb action memory issue
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-07 17:50:43 +13:00
Jonathan Swartz 615a0f1ab9 Fix carriage return typo
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-07 17:50:43 +13:00
Jonathan Swartz c84d4e2848 Uncomment paths-ignore in fvdb action
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-07 17:50:43 +13:00
Jonathan Swartz 182a9e584e Commenting out fvdb paths-ignore
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-07 17:50:43 +13:00
Jonathan Swartz 8d2894cd5d Updating fvdb and whitespace actions for PR bot workflow
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-07 17:50:43 +13:00
Jonathan Swartz f655251b2b Merge branch 'develop' into feature/fvdb
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2025-01-07 09:59:12 +13:00
Matthew Cong 59d2a5c7e3 Fix ODR violation due to anonymous namespace in SignedFloodFill (#97)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-01-06 12:09:18 -08:00
Matthew Cong 5ecb9c3ead Fix format strings and Windows conversion warning (#101)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-01-06 11:15:25 -08:00
Ken Museth 23ded25be2 improved nanovdb_convert (#98)
* improved nanovdb_convert

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* fixed typo

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* added unit tests

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* review feedback

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
2025-01-06 11:09:50 -08:00
Matthew Cong fa1ac5feef Add missing iomanip include for latest GTest
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-01-03 16:59:30 +13:00
Matthew Cong a004afbb05 Separate PointsToGrid kernel lambdas into functors (#94)
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2025-01-02 11:57:39 -06:00
Ken Museth adfd4cd1ef improved change notes and updated version number (#95)
* improved change notes and updated version number

Signed-off-by: Ken Museth <ken.museth@gmail.com>

* improved documentation

Signed-off-by: Ken Museth <ken.museth@gmail.com>

---------

Signed-off-by: Ken Museth <ken.museth@gmail.com>
2024-12-31 12:48:56 -08:00
Matthew Cong 1251b0e016 Fix missing OpenVDB dependency for NanoVDB-only build
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2024-12-31 12:03:58 +13:00
Jonathan Swartz 8c92ec47a6 Pull Request Testing Setup (#86)
* Added copy-pr-bot. Updated triggers to branch pushes per instructions:
https://docs.gha-runners.nvidia.com/apps/copy-pr-bot/

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

* Add NV CPU runners to tasks requiring larger instances
Turning off USE_EXPLICIT_INSTANTIATION, now unblocked by larger runner

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>

---------

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-30 15:04:03 +13:00
Jonathan Swartz 6f10bf3b15 Merge pull request #84 from swahtz/js/voxel-foundation-migration
Migrating last PR changes from private repo over to this repo
2024-12-23 12:02:12 +13:00
Jonathan Swartz 9d691a88d9 Removing GaussianRasterize.cu which was split into two source files
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-22 16:38:38 +13:00
Jonathan Swartz 06957104f9 Merge branch 'feature/fvdb' into js/voxel-foundation-migration 2024-12-22 16:33:38 +13:00
Jonathan Swartz 2d4f3e1bfe Migrating last PR changes from private repo over to this repo including:
Changes to .clang-format for better include sorting
Gsplat rasterization code cleanup and add unused arguments for pixel mask

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-22 16:30:45 +13:00
Jonathan Swartz 548b52e97e Merge pull request #46 from NVIDIA-Omniverse/js/fvdb_codestyle
Add fVDB code style check actions
2024-12-21 13:56:06 +13:00
Matthew Cong 47c5a29be4 Merge pull request #76 from NVIDIA-Omniverse/nanovdb_merge
Merge changes from latest NanoVDB
2024-12-20 15:59:12 -08:00
Matthew Cong faa644ab3d Merge changes from latest NanoVDB
Signed-off-by: Matthew Cong <mcong@nvidia.com>
2024-12-20 15:58:30 -08:00
Jonathan Swartz ce544bd2ff fix clang-format issue with JCat0.cu
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-21 11:26:13 +13:00
Jonathan Swartz 1f4a9d5617 Merge branch 'feature/fvdb' into js/fvdb_codestyle
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-21 11:22:35 +13:00
Jonathan Swartz 263e0e069e Merge pull request #81 from NVIDIA-Omniverse/fw/fix_jcat0
Fix jagged concatenation when concatenating a JaggedTensor with only one element in it
2024-12-20 11:55:11 +13:00
Francis Williams 200cf246ee Enhance JCat0 kernel to handle empty JIdx tensor case and add unit test for concatenation of JaggedTensors 2024-12-19 16:40:46 -05:00
Jonathan Swartz 6ccc92f0c6 newline
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-19 09:41:24 +13:00
Jonathan Swartz 29606d6a74 Update SPDX identifier action with ignore-paths for source files/dirs that have different licenses
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-18 16:48:33 +13:00
Jonathan Swartz 18fc7dc65d Merge pull request #66 from swahtz/disable_docs_for_fvdb
Disable docs Workflow for fVDB changes
2024-12-18 16:43:42 +13:00
Jonathan Swartz 3b5458baad Merge pull request #62 from NVIDIA-Omniverse/master
Updating `feature/fvdb` from `master`
2024-12-18 16:39:25 +13:00
Jonathan Swartz 5ac52485b1 Disabling docs action for fvdb changes
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-18 14:09:10 +13:00
Jonathan Swartz 1d9c7345b6 Addressing review notes
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-17 17:56:40 +13:00
Jonathan Swartz 7053bfd0d2 Fix codestyle source paths
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-17 14:31:07 +13:00
Jonathan Swartz a035c14b90 Add fVDB code style check actions
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-17 09:04:33 +13:00
Jonathan Swartz 70566d7fb2 Merge pull request #45 from NVIDIA-Omniverse/js/fvdb_actions
Add fVDB CI
2024-12-17 08:36:57 +13:00
Jonathan Swartz f70b1a570f Turn off explicit instantiation for debug linux build
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-16 21:56:45 +13:00
Jonathan Swartz 128be93972 fVDB continuous benchmaring results path fix
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-16 13:46:57 +13:00
Jonathan Swartz f088f173d1 Fixing fvdb wheel upload path
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-16 13:07:12 +13:00
Jonathan Swartz 43a5220700 Adding fvdb CI unit tests to github workflows
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-13 12:26:18 +13:00
Jonathan Swartz 38adbab1bc Updating the versions of checkout action; add .gitmodules to ignore list
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-13 11:40:50 +13:00
Jonathan Swartz 093a6382ae fVDB: Added 'projects' directory to include more fully-featured example 'batteries included' projects
Added panoptic segmentation project with MaskPLS implementation
Fixed install issue not including `fvdb.optim`
Re-introduced support for Volta&Turing but added errors for use of certain conv backends
Added interpolation functions to VDBTensor

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-11 15:47:23 +13:00
Jonathan Swartz 11836adebd Merge branch 'master' into feature/fvdb
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-06 12:35:18 +13:00
Jonathan Swartz f7b18d9d7b fVDB: Fix missing fvdb.utils.data modules
Expose various 'ELU non-linearities
Integer JaggedReduce
Updated build TORCH_CUDA_ARCH_LIST for supported CUDA compute architectures, fixed Docker dev and CI images
Updated REAME's env and build instructions

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-12-03 20:11:34 +13:00
Jonathan Swartz f1d9e2bc08 Version 0.2.0 release:
• Significant refactor of gaussian splatting pipeline
• Refactor of examples/3dgs using the new 3dgs pipeline (see README for a starting point to the example)
• License identifiers changed to Apache 2.0 to be in line with OpenVDB
• A `dev_environment.yml` is now included to help construct a reccommended development environment
• A bug was fixed when using `GridBatch.ijk_to_inv_index(cumulative=False)` when the ijk argument has repeated elements
• GLM is being vendored in and is no longer a runtime dependency
• The utility functions needed in the examples and unit tests to download and read the relevant data files has been refactored into `fvdb.utils.tests` and `fvdb.utils.examples` respectively

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-11-12 17:44:24 +13:00
Jonathan Swartz 7496bfa45c Merge pull request #1934 from JamesPerlman/patch-1
fVDB: Fix typo in 01_api_basics.ipynb
2024-10-23 07:35:27 +13:00
James Perlman 3d4a21b185 Update 01_api_basics.ipynb
There are 5 random features per voxel, and 8 grids.  Not 8 random features per voxel.

Signed-off-by: James Perlman <jam.e.perl@gmail.com>
2024-10-11 13:07:59 -07:00
Jonathan Swartz 6faf27c7cb Merge pull request #1915 from swahtz/feature/fvdb
Merge new commits from fVDB repository
2024-09-26 21:01:05 +12:00
Jonathan Swartz b5922af1cb Merge commits from fVDB repository including:
Version v0.1.0 release
Added gaussian splatting support
Significant VDBTensor API rewrite as well as various GridBatch function name changes to make them more consistent

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-09-26 20:47:01 +12:00
Jonathan Swartz f0324e6fca Merge pull request #1907 from AcademySoftwareFoundation/master
Merge master to feature/fvdb
2024-09-18 20:31:54 +12:00
Jonathan Swartz 9accf19836 Merge pull request #1891 from swahtz/feature/fvdb
fVDB improved test setup and PyTorch 2.4 update
2024-09-17 13:32:09 +12:00
Jonathan Swartz 4592f6c887 Updated fVDB requirements and documentation to PyTorch 2.4
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-09-16 21:04:12 +12:00
Jonathan Swartz 5de7ee4da9 Moved all test data to an external repository and updated tests
Updated Github test actions to split apart build and test steps
Added negative operator to JaggedTensor; to operators to SparseConvPackInfo

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-09-06 21:25:33 +12:00
apradhana 70e0264ed1 Merge pull request #1886 from swahtz/feature/fvdb
fVDB Updates: Bugs fixed for multi-GPU support, JaggedTensor slicing/indexing CUDA kernels
2024-09-04 12:09:21 -07:00
Jonathan Swartz 52ba85af93 Bugs fixed for multi-GPU support
JaggedTensor slicing and indexing moved to CUDA kernels
Small updates to basic_concepts docs and README

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-09-02 18:17:30 +12:00
apradhana 975beecdb6 Merge pull request #1876 from swahtz/feature/fvdb
fVDB code format updates
2024-08-20 20:33:09 -07:00
Jonathan Swartz c9aee5964a Formatting changes to match OpenVDB guidelines
Added examples
Updated README and environment/docker updates

Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
2024-08-21 10:47:20 +12:00
apradhana c99b17c85c Merge pull request #1869 from swahtz/feature/fvdb
Updates to fVDB
2024-08-13 17:24:05 -07:00
Jonathan Swartz 9357034f9a fVDB changes including:
•Reintroduced missing src/build/* files that were accidentally ommitted
•JaggedTensor concatentation moved to a cuda kernel
•Non-cumulative indices for ijk mappings is now the default return style (makes indexing into JaggedTensors of data more natural)
•cuDNN disabled for CUDA versions <12
•Updated notebook files to new API and now included
•Removed deprecated usage of `python setup.py test` and running pytest directly

Signed-off-by: Jonathan Swartz <jonathan@swartzfx.com>
2024-08-14 11:54:20 +12:00
apradhana 63f7ab0da4 Merge pull request #1862 from fwilliams/fw/fvdb
Commit fVDB to OpenVDB
2024-07-23 12:01:25 -07:00
Francis Williams 9b178ba16c Commit fVDB to OpenVDB
Signed-off-by: Francis Williams <francis@fwilliams.info>
2024-07-23 14:40:41 -04:00
684 changed files with 144968 additions and 1444 deletions
+1
View File
@@ -0,0 +1 @@
+3 -3
View File
@@ -55,7 +55,7 @@ jobs:
github.event_name != 'workflow_dispatch' ||
github.event.inputs.type == 'all' ||
github.event.inputs.type == 'linux'
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
runs-on: ${{ (github.repository_owner == 'NVIDIA-Omniverse' && 'linux-amd64-cpu32') || (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
name: >
linux-ax:${{ matrix.config.image }}-cxx:${{ matrix.config.cxx }}-${{ matrix.config.build }}
container:
@@ -152,7 +152,7 @@ jobs:
if: |
github.event_name == 'workflow_dispatch' &&
github.event.inputs.type == 'grammar'
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
runs-on: ${{ (github.repository_owner == 'NVIDIA-Omniverse' && 'linux-amd64-cpu32') || (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
container:
image: aswf/ci-openvdb:2023-clang15
steps:
@@ -160,7 +160,7 @@ jobs:
- name: build
run: ./ci/build.sh -v --components=axgr --target=openvdb_ax_grammar --cargs=\"-DOPENVDB_AX_GRAMMAR_NO_LINES=ON\"
- name: upload grammar
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v2
with:
name: ax_grammar
path: ./build/openvdb_ax/openvdb_ax/openvdb_ax/grammar
+1 -1
View File
@@ -62,7 +62,7 @@ jobs:
github.event_name != 'workflow_dispatch' ||
github.event.inputs.type == 'all' ||
github.event.inputs.type == 'linux'
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
runs-on: ${{ (github.repository_owner == 'NVIDIA-Omniverse' && 'linux-amd64-cpu32') || (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
name: >
linux-vfx:${{ matrix.config.image }}-
abi:${{ matrix.config.abi }}-
+118
View File
@@ -0,0 +1,118 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
#
# GitHub Actions workflow file
# https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions
name: fVDB Codebuild
on:
pull_request:
paths-ignore:
- 'CHANGES'
- 'CODEOWNERS'
- 'doc/**'
- 'openvdb/**'
- 'openvdb_cmd/**'
- 'openvdb_ax/**'
- 'openvdb_maya/**'
- 'openvdb_houdini/**'
- 'nanovdb/**'
- 'pendingchanges/**'
- '**.md'
- 'fvdb/debug/**'
- 'fvdb/docs/**'
- 'fvdb/examples/**'
- 'fvdb/notebooks/**'
- 'fvdb/scripts/**'
permissions:
contents: read
id-token: write
jobs:
# ---------------------------------------------------------------------------
# Linux GPU (AWS CodeBuild)
# ---------------------------------------------------------------------------
linux_gpu:
name: 'ci-openvdb 2024'
# Don't run on forks
if: github.repository == 'AcademySoftwareFoundation/openvdb'
# GH-hosted VM. The build runs in CentOS 7 'container' hard-coded in
# AWS CodeBuild project.
# TODO: Add support for dynamic GH Action defined CodeBuild
# container choice.
runs-on: ubuntu-latest
steps:
- name: Verify AWS Credentials Set
run: |
if [ -z "${{ secrets.CODEBUILD_ID }}" ] || [ -z "${{ secrets.CODEBUILD_SECRET }}" ]; then
echo "Error: AWS credentials secrets not properly configured"
echo "Please ensure CODEBUILD_ID and CODEBUILD_SECRET are set in organization secrets"
exit 1
fi
echo "AWS credentials secrets are configured"
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.CODEBUILD_ID }}
aws-secret-access-key: ${{ secrets.CODEBUILD_SECRET }}
aws-region: us-west-2
- name: Verify AWS Authentication
run: |
echo "Testing AWS authentication..."
aws sts get-caller-identity || {
echo "Error: AWS authentication failed"
exit 1
}
echo "AWS authentication successful"
- name: CodeBuild Monitoring URLs
run: |
echo "🚀 Starting CodeBuild job..."
echo "Project: OpenVDB"
echo ""
echo "📊 Monitor the build progress at:"
# Check if public builds are enabled by looking for the public project URL
PUBLIC_URL=$(aws codebuild batch-get-projects --names OpenVDB --query 'projects[0].publicProjectUrl' --output text 2>/dev/null || echo "None")
if [ "$PUBLIC_URL" != "None" ] && [ "$PUBLIC_URL" != "null" ]; then
echo "📖 Public Build Results (no sign-in required):"
echo " $PUBLIC_URL"
echo ""
fi
echo "🔒 AWS Console (requires sign-in):"
echo " Project: https://console.aws.amazon.com/codesuite/codebuild/projects/OpenVDB/history"
echo ""
echo "ℹ️ Individual build URLs will be available once the job starts..."
- name: Run CodeBuild
id: codebuild
uses: aws-actions/aws-codebuild-run-build@v1
with:
project-name: OpenVDB
buildspec-override: ci/buildspec.yml
- name: CodeBuild Results
if: always()
run: |
echo "📋 CodeBuild Results:"
echo "Build ID: ${{ steps.codebuild.outputs.aws-build-id }}"
echo "Build Status: ${{ steps.codebuild.outputs.aws-build-status }}"
echo ""
echo "🔗 Detailed Build Links:"
if [ -n "$CODEBUILD_PUBLIC_BUILD_URL" ]; then
echo "📖 Public Build Details: $CODEBUILD_PUBLIC_BUILD_URL"
fi
echo "🔒 AWS Console Build Details: https://console.aws.amazon.com/codesuite/codebuild/projects/OpenVDB/build/${{ steps.codebuild.outputs.aws-build-id }}/"
echo ""
if [ "${{ steps.codebuild.outputs.aws-build-status }}" != "SUCCEEDED" ]; then
echo "❌ Build failed! Check the build logs at the links above for detailed error information."
echo "Common issues to check:"
echo "- Build environment configuration"
echo "- Source code compilation errors"
echo "- Missing dependencies"
echo "- Buildspec.yml configuration"
else
echo "✅ Build succeeded!"
fi
+80
View File
@@ -0,0 +1,80 @@
name: fVDB Code Style
on:
push:
branches:
- "pull-request/[0-9]+"
paths-ignore:
- 'CHANGES'
- 'CODEOWNERS'
- 'doc/**'
- 'ci/**'
- 'openvdb/**'
- 'openvdb_cmd/**'
- 'openvdb_ax/**'
- 'openvdb_maya/**'
- 'openvdb_houdini/**'
- 'openvdb_wolfram/**'
- 'tsc/**'
- 'nanovdb/**'
- 'pendingchanges/**'
- '**.md'
- 'fvdb/docs/**'
- 'fvdb/env/**'
- 'fvdb/notebooks/**'
# Allow subsequent pushes to the same PR or REF to cancel any previous jobs.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
test-python-black-lint:
name: Check Python code style with black
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: psf/black@stable
with:
options: "--check --verbose --target-version=py311 --line-length=120"
src: "fvdb/"
version: "~= 24.0"
test-cpp-clang-format-lint:
name: Check C++ code style with clang-format
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: DoozyX/clang-format-lint-action@v0.18.2
with:
source: 'fvdb/src/'
extensions: 'h,cpp,cc,cu,cuh'
clangFormatVersion: 18
style: file
include-guards:
name: Check include guards
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: swahtz/include-guards-check-action@master
with:
path: 'fvdb/src/'
check-spdx-identifiers:
name: Check SPDX identifiers
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout@v4
- uses: swahtz/spdx@feature/ignore_paths
with:
licenses: |-
Apache-2.0
ignore-paths: |-
openvdb/openvdb/math/Half.cc
openvdb/openvdb/math/Half.h
openvdb_wolfram/OpenVDBLink
openvdb_ax/openvdb_ax/grammar/generated
fvdb/projects/panoptic_segmentation/garfield/garfield
+1 -1
View File
@@ -65,7 +65,7 @@ jobs:
if: >
${{ needs.checksecret.outputs.HOUDINI_SECRETS == 'true' ||
github.repository_owner == 'AcademySoftwareFoundation' }}
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
runs-on: ${{ (github.repository_owner == 'NVIDIA-Omniverse' && 'linux-amd64-cpu32') || (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
name: hou:${{ matrix.config.hou_hash }}-vfx:${{ matrix.config.image }}-cxx:${{ matrix.config.cxx }}
container:
image: aswf/ci-base:${{ matrix.config.image }}
+2 -2
View File
@@ -51,7 +51,7 @@ jobs:
github.event_name != 'workflow_dispatch' ||
github.event.inputs.type == 'all' ||
github.event.inputs.type == 'linux'
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
runs-on: ${{ (github.repository_owner == 'NVIDIA-Omniverse' && 'linux-amd64-cpu32') || (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
name: >
linux-nanovdb:cxx:${{ matrix.config.cxx }}-${{ matrix.config.build }}
container:
@@ -171,7 +171,7 @@ jobs:
github.event_name != 'workflow_dispatch' ||
github.event.inputs.type == 'all' ||
github.event.inputs.type == 'linux'
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
runs-on: ${{ (github.repository_owner == 'NVIDIA-Omniverse' && 'linux-amd64-cpu32') || (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
container:
image: aswf/ci-openvdb:2024
steps:
+6 -8
View File
@@ -61,7 +61,7 @@ jobs:
(github.event_name != 'workflow_dispatch' ||
github.event.inputs.type == 'all' ||
github.event.inputs.type == 'houdini')
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-20.04-8c-32g-300h') || 'ubuntu-latest' }}
name: linux-houdini:${{ matrix.config.hou_hash }}
env:
CXX: clang++
@@ -146,7 +146,7 @@ jobs:
github.event_name != 'workflow_dispatch' ||
github.event.inputs.type == 'all' ||
github.event.inputs.type == 'extra'
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-20.04-8c-32g-300h') || 'ubuntu-latest' }}
name: linux-extra:${{ matrix.config.name }}
container:
image: aswf/ci-openvdb:2024
@@ -203,7 +203,7 @@ jobs:
- name: install_deps
run: |
if [ "$RUNNER_OS" == "Linux" ]; then
sudo apt-get -q install -y libboost-dev libboost-iostreams-dev libtbb-dev libblosc-dev llvm-15-dev libgtest-dev libgmock-dev libcppunit-dev
sudo apt-get -q install -y libboost-dev libboost-iostreams-dev libtbb-dev libblosc-dev llvm-dev libgtest-dev libcppunit-dev
./ci/install_nanobind.sh 2.0.0
elif [ "$RUNNER_OS" == "macOS" ]; then
./ci/install_macos.sh 15
@@ -284,7 +284,7 @@ jobs:
github.event_name != 'workflow_dispatch' ||
github.event.inputs.type == 'all' ||
github.event.inputs.type == 'ax'
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-20.04-8c-32g-300h') || 'ubuntu-latest' }}
name: >
linux-ax:${{ matrix.config.image }}-cxx:${{ matrix.config.cxx }}-${{ matrix.config.build }}
container:
@@ -313,7 +313,6 @@ jobs:
-DOPENVDB_AX_TEST_CMD_DOWNLOADS=ON
-DUSE_EXPLICIT_INSTANTIATION=OFF
-DOPENVDB_CXX_STRICT=ON
-DOPENVDB_BUILD_VDB_TOOL=OFF
\"
- name: clean
if: matrix.config.components == 'core'
@@ -329,7 +328,6 @@ jobs:
-DOPENVDB_AX_TEST_CMD_DOWNLOADS=ON
-DUSE_EXPLICIT_INSTANTIATION=OFF
-DOPENVDB_CXX_STRICT=ON
-DOPENVDB_BUILD_VDB_TOOL=OFF
\"
- name: test
run: cd build && ctest -V
@@ -496,7 +494,7 @@ jobs:
github.event_name != 'workflow_dispatch' ||
github.event.inputs.type == 'all' ||
github.event.inputs.type == 'blosc'
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-22.04-8c-32g-300h') || 'ubuntu-latest' }}
runs-on: ${{ (github.repository_owner == 'AcademySoftwareFoundation' && 'ubuntu-20.04-8c-32g-300h') || 'ubuntu-latest' }}
name: linux-blosc:${{ matrix.blosc }}
container:
image: aswf/ci-base:2023
@@ -592,7 +590,7 @@ jobs:
-strict
-extended
- name: upload_report
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v3
if: always()
with:
name: abi_report
+2
View File
@@ -1,2 +1,4 @@
/build/*
**/src/benchmarks/build/*
nanovdb/nanovdb/build/*
.DS_Store
+3
View File
@@ -41,3 +41,6 @@ CHANGES @kmuseth @apradhana
/openvdb_houdini/openvdb_houdini/PointUtils.* @danrbailey @idclip
/openvdb_houdini/openvdb_houdini/*_OpenVDB_Points_* @danrbailey @idclip
/openvdb_houdini/openvdb_houdini/GR_PrimVDBPoints.* @danrbailey @idclip
# fVDB
/fvdb/ @NVIDIA-Omniverse/fvdb-dev
+84
View File
@@ -0,0 +1,84 @@
version: 0.2
env:
variables:
PYTHONPATH: ""
# secrets-manager:
# GITHUB_TOKEN: "github-token:token"
phases:
install:
runtime-versions:
python: 3.10
commands:
# Install Conda
- wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
- bash Miniforge3-Linux-x86_64.sh -b -p $HOME/miniforge3
- eval "$($HOME/miniforge3/bin/conda shell.bash hook)"
- conda init
- source ~/.bashrc || true
# Create build environment
- conda env create -f fvdb/env/build_environment.yml -n fvdb_build
- echo "Build environment created successfully"
# Create test environment
- conda env create -f fvdb/env/test_environment.yml -n fvdb_test
- echo "Test environment created successfully"
pre_build:
commands:
- echo "Starting pre_build phase"
- git config --global --add safe.directory "$(pwd)"
# Verify environments exist
- conda env list
- echo "Pre-build phase completed"
build:
commands:
- echo "Starting build phase"
# Build fvdb wheel using conda run (this ensures proper environment activation)
- cd fvdb
- export TORCH_CUDA_ARCH_LIST="7.0;7.5;8.0;8.6+PTX"
- echo "Building wheel with conda run..."
- conda run --no-capture-output -n fvdb_build ./build.sh wheel verbose gtests benchmarks
# Verify wheel was created
- ls -la dist/
- echo "Wheel build completed"
# Move back to root directory
- cd ..
# Install package in test environment using conda run
- echo "Installing wheel in test environment..."
- conda run --no-capture-output -n fvdb_test pip install ./fvdb/dist/*.whl
- echo "Package installation completed"
post_build:
commands:
- echo "Starting post_build phase"
# Run unit tests using conda run
- echo "Running unit tests..."
- cd fvdb/tests
- conda run --no-capture-output -n fvdb_test pytest -v unit
# Run documentation tests using conda run
- echo "Running documentation tests..."
- cd ..
- conda run --no-capture-output -n fvdb_test pytest --markdown-docs docs
- echo "All tests completed successfully"
artifacts:
files:
- fvdb/dist/*.whl
name: fvdb-artifacts
cache:
paths:
- '/root/.cache/pip/**/*'
- '$HOME/miniforge3/**/*'
+24
View File
@@ -0,0 +1,24 @@
# SPDX-License-Identifier: MIT
#
# SPDX-FileCopyrightText: Copyright (c) 2019-2023 Lars Melchior and contributors
set(CPM_DOWNLOAD_VERSION 0.40.2)
set(CPM_HASH_SUM "c8cdc32c03816538ce22781ed72964dc864b2a34a310d3b7104812a5ca2d835d")
if(CPM_SOURCE_CACHE)
set(CPM_DOWNLOAD_LOCATION "${CPM_SOURCE_CACHE}/cpm/CPM_${CPM_DOWNLOAD_VERSION}.cmake")
elseif(DEFINED ENV{CPM_SOURCE_CACHE})
set(CPM_DOWNLOAD_LOCATION "$ENV{CPM_SOURCE_CACHE}/cpm/CPM_${CPM_DOWNLOAD_VERSION}.cmake")
else()
set(CPM_DOWNLOAD_LOCATION "${CMAKE_BINARY_DIR}/cmake/CPM_${CPM_DOWNLOAD_VERSION}.cmake")
endif()
# Expand relative path. This is important if the provided path contains a tilde (~)
get_filename_component(CPM_DOWNLOAD_LOCATION ${CPM_DOWNLOAD_LOCATION} ABSOLUTE)
file(DOWNLOAD
https://github.com/cpm-cmake/CPM.cmake/releases/download/v${CPM_DOWNLOAD_VERSION}/CPM.cmake
${CPM_DOWNLOAD_LOCATION} EXPECTED_HASH SHA256=${CPM_HASH_SUM}
)
include(${CPM_DOWNLOAD_LOCATION})
+44
View File
@@ -0,0 +1,44 @@
include (FindPackageHandleStandardArgs)
if(NCCL_ROOT)
set(_NCCL_ROOT ${NCCL_ROOT})
elseif(DEFINED ENV{NCCL_ROOT})
set(_NCCL_ROOT $ENV{NCCL_ROOT})
endif()
set(_NCCL_INCLUDE_SEARCH_DIRS "")
list(APPEND _NCCL_INCLUDE_SEARCH_DIRS
${_NCCL_ROOT}/include
)
find_path(NCCL_INCLUDE_DIR
NAMES nccl.h
PATHS ${_NCCL_INCLUDE_SEARCH_DIRS}
)
set(_NCCL_LIBRARY_SEARCH_DIRS "")
list(APPEND _NCCL_LIBRARY_SEARCH_DIRS
${_NCCL_ROOT}/bin
)
find_library(NCCL_LIBRARY
NAMES nccl
PATHS ${_NCCL_LIBRARY_SEARCH_DIRS}
)
find_package_handle_standard_args(NCCL REQUIRED_VARS NCCL_INCLUDE_DIR NCCL_LIBRARY)
if (NCCL_FOUND)
mark_as_advanced(NCCL_INCLUDE_DIR)
mark_as_advanced(NCCL_LIBRARY)
endif()
if (NCCL_FOUND AND NOT TARGET NCCL::NCCL)
add_library(NCCL::NCCL SHARED IMPORTED)
if(MSVC)
string(REPLACE ".lib" ".dll" NCCL_DLL ${NCCL_LIBRARY})
set_property(TARGET NCCL::NCCL PROPERTY IMPORTED_LOCATION ${NCCL_DLL})
set_property(TARGET NCCL::NCCL PROPERTY IMPORTED_IMPLIB ${NCCL_LIBRARY})
else()
set_property(TARGET NCCL::NCCL PROPERTY IMPORTED_LOCATION ${NCCL_LIBRARY})
endif()
target_include_directories(NCCL::NCCL INTERFACE ${NCCL_INCLUDE_DIR})
endif()
+139
View File
@@ -0,0 +1,139 @@
---
Language: Cpp
Standard: c++20
# Indentation
TabWidth: 4
IndentWidth: 4
UseTab: Never
IndentPPDirectives: None
IndentWrappedFunctionNames: false
NamespaceIndentation: None
# Empty lines
KeepEmptyLinesAtTheStartOfBlocks: false
MaxEmptyLinesToKeep: 1
# Line length
ColumnLimit: 100
# Line endings
DeriveLineEnding: false
LineEnding: LF
# Breaking and Penalties
BreakBeforeBinaryOperators: None
BreakBeforeBraces: Attach
BreakBeforeTernaryOperators: true
BreakConstructorInitializersBeforeComma: false
BreakInheritanceList: BeforeColon
BreakStringLiterals: false
PenaltyBreakAssignment: 2
PenaltyBreakBeforeFirstCallParameter: 19
PenaltyBreakComment: 300
PenaltyBreakFirstLessLess: 120
PenaltyBreakString: 1000
PenaltyExcessCharacter: 1000000
PenaltyReturnTypeOnItsOwnLine: 60
# Spacing and padding
SpaceAfterCStyleCast: false
SpaceAfterTemplateKeyword: true
SpaceAfterLogicalNot: false
SpaceAfterControlStatementKeyword: true
SpaceBeforeAssignmentOperators: true
SpaceBeforeParens: ControlStatements
SpaceBeforeRangeBasedForLoopColon: false
SpaceInEmptyParentheses: false
SpacesInAngles: false
SpacesInContainerLiterals: false
SpacesInCStyleCastParentheses: false
SpacesInParentheses: false
SpacesBeforeTrailingComments: 1
SpaceBeforeCpp11BracedList: false
SpaceBeforeCtorInitializerColon: true
SpaceBeforeInheritanceColon: true
SpaceBeforeSquareBrackets: false
SpacesInSquareBrackets: false
SpaceBeforeCaseColon: false
# Brace placement
BraceWrapping:
AfterClass: false
AfterControlStatement: false
AfterEnum: false
AfterFunction: false
AfterNamespace: false
AfterStruct: false
AfterUnion: false
BeforeCatch: false
BeforeElse: false
IndentBraces: false
SplitEmptyFunction: false
SplitEmptyRecord: false
SplitEmptyNamespace: false
Cpp11BracedListStyle: true
# Function definitions
# BreakAfterReturnType: AllDefinitions
AlwaysBreakAfterDefinitionReturnType: All
AttributeMacros:
- __host__
- __device__
- __hostdev__
- __global__
- __forceinline__
- __shared__
- __launch_bounds__
# Alignment
AlignConsecutiveAssignments: true
AlignConsecutiveBitFields: true
AlignConsecutiveDeclarations: false
AlignConsecutiveMacros: true
AlignEscapedNewlines: Left
AlignOperands: true
AlignTrailingComments:
Kind: Always
OverEmptyLines: 2
# Single line allowances
BinPackParameters: false
BinPackArguments: false
AllowAllArgumentsOnNextLine: true
AllowAllConstructorInitializersOnNextLine: true
AllowAllParametersOfDeclarationOnNextLine: true
AllowShortBlocksOnASingleLine: true
AllowShortCaseLabelsOnASingleLine: true
AllowShortEnumsOnASingleLine: true
AllowShortFunctionsOnASingleLine: Inline
AllowShortIfStatementsOnASingleLine: false
AllowShortLambdasOnASingleLine: true
AllowShortLoopsOnASingleLine: false
# Sorting
IncludeBlocks: Regroup
IncludeIsMainRegex: "$"
IncludeCategories:
- Regex: '^"' # quoted includes
Priority: 1
- Regex: '^<fvdb/' # fvdb includes
Priority: 2
- Regex: '^<nanovdb/' # nanovdb includes
Priority: 3
- Regex: '^<(torch|c10|THC|ATen)/' # torch includes
Priority: 4
- Regex: '^<(thrust|cub|cuda)/' # CCCL includes
Priority: 5
- Regex: '^<(cooperative_groups|cuco|cuda.h|cuda_|device_types|math_constants|nvtx3|cute)' # CUDA includes
Priority: 6
- Regex: '^<.*\..*' # other system includes (e.g. with a '.')
Priority: 7
- Regex: '^<[^.]+' # STL includes (no '.')
Priority: 8
IncludeIsMainSourceRegex: "$"
SortIncludes: true
SortUsingDeclarations: true
+71
View File
@@ -0,0 +1,71 @@
# https://clangd.llvm.org/config
# Apply a config conditionally to all C files
If:
PathMatch: .*\.(c|h)$
---
# Apply a config conditionally to all C++ files
If:
PathMatch: .*\.(c|h)pp
---
# Apply a config conditionally to all CUDA files
If:
PathMatch: .*\.cuh?
CompileFlags:
Add:
- "-x"
- "cuda"
# No error on unknown CUDA versions
- "-Wno-unknown-cuda-version"
# Allow variadic CUDA functions
- "-Xclang=-fcuda-allow-variadic-functions"
Diagnostics:
Suppress:
- "variadic_device_fn"
- "attributes_not_allowed"
- "unknown_cuda_version"
- "unknown_type_name"
- "implicit_return_from_non_void_function"
- "no_template_args_on_forwarding_function"
---
# Tweak the clangd parse settings for all files
CompileFlags:
Add:
# report all errors
- "-ferror-limit=0"
- "-fmacro-backtrace-limit=0"
- "-ftemplate-backtrace-limit=0"
# Skip the CUDA version check
- "--no-cuda-version-check"
Remove:
# remove gcc's -fcoroutines
- -fcoroutines
# remove nvc++ flags unknown to clang
- "-gpu=*"
- "-stdpar*"
# remove nvcc flags unknown to clang
- "-arch*"
- "-gencode*"
- "--generate-code*"
- "-ccbin*"
- "-t=*"
- "--threads*"
- "-Xptxas*"
- "-Xcudafe*"
- "-Xfatbin*"
- "-Xcompiler*"
- "--diag-suppress*"
- "--diag_suppress*"
- "--compiler-options*"
- "--extended-lambda"
- "--expt-extended-lambda"
- "--expt-relaxed-constexpr"
- "-forward-unknown-to-host-compiler"
- "-Werror=cross-execution-space-call"
- "-Werror=all-warnings"
+41
View File
@@ -0,0 +1,41 @@
// File location: openvdb/fvdb/.devcontainer/devcontainer.json
{
// A friendly name for the container configuration shown in VS Code
"name": "FVDB Development Environment",
// Tells VS Code to use Docker Compose to define and run the environment.
// The path is relative to this devcontainer.json file.
"dockerComposeFile": [
"../docker-compose.yml" // Points to openvdb/fvdb/docker-compose.yml
],
// The name of the specific service within the docker-compose.yml file
// that VS Code should connect to.
"service": "fvdb-dev",
// Specifies the default directory VS Code should open *inside* the container.
// Since the docker-compose.yml mounts the host's `openvdb/` directory to `/openvdb`
// in the container, the fvdb code resides at `/openvdb/fvdb`.
"workspaceFolder": "/openvdb/fvdb",
// Extensions to install inside the container's VS Code server
"customizations": {
"vscode": {
"extensions": [
"ms-python.python", // Python language support
"ms-vscode.cpptools", // C/C++ language support
"ms-vscode.cmake-tools", // CMake integration
"ms-azuretools.vscode-docker" // Docker integration (useful for managing containers)
],
"settings": {
"python.defaultInterpreterPath": "/opt/conda/envs/fvdb/bin/python"
}
}
},
// Optional: What to do when the VS Code window is closed.
// "stopCompose" will stop the container(s) defined in the compose file.
// "none" would leave them running.
"shutdownAction": "stopCompose"
}
+106
View File
@@ -0,0 +1,106 @@
name: Building Release Wheels
on:
release:
types: [published, edited]
workflow_dispatch:
jobs:
wheel:
runs-on:
- self-hosted
strategy:
fail-fast: false
matrix:
# support version based on: https://download.pytorch.org/whl/torch/
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
torch-version: [2.1.0, 2.1.1, 2.1.2, 2.2.0, 2.2.1, 2.2.2, 2.3.0]
cuda-version: ['12.1.1']
exclude:
- torch-version: 2.1.2
python-version: '3.12'
- torch-version: 2.1.1
python-version: '3.12'
- torch-version: 2.1.0
python-version: '3.12'
steps:
- uses: actions/checkout@v2
- name: Generate release_environment.yml
run: |
echo "cp env/release_base_environment.yml release_environment.yml"
cp env/release_base_environment.yml release_environment.yml
echo " - python=${{ matrix.python-version }}" >> release_environment.yml
echo " - pytorch=${{ matrix.torch-version }}" >> release_environment.yml
perl -E '"${{ matrix.cuda-version }}" =~ /^(\d+\.\d+)/; say " - pytorch-cuda=$1"' >> release_environment.yml
perl -E '"${{ matrix.cuda-version }}" =~ /^(\d+)\.(\d+)/; say " - cuda>=$1.$2.0, <$1." . ($2+1) . ".0"' >> release_environment.yml
perl -E '"${{ matrix.cuda-version }}" =~ /^(\d+)\.(\d+)/; say " - cuda-nvcc>=$1.$2.0, <$1." . ($2+1) . ".0"' >> release_environment.yml
- name: Set up fvdb_release Conda env
uses: conda-incubator/setup-miniconda@v3
with:
miniconda-version: "latest"
mamba-version: "*"
channels: nvidia,pytorch,conda-forge,defaults
channel-priority: true
activate-environment: fvdb_release
environment-file: release_environment.yml
- name: Set version
run: |
VERSION=`sed -n "s/^__version__ = '\(.*\)'/\1/p" fvdb/__init__.py`
TORCH_VERSION=`echo "pt${{ matrix.torch-version }}" | sed "s/..$//" | sed "s/\.//g"`
CUDA_VERSION=`perl -E '"${{ matrix.cuda-version }}" =~ /^(\d+)\.(\d+)/; say "cu$1$2"'`
echo "New version name: $VERSION+$TORCH_VERSION$CUDA_VERSION"
sed -i "s/$VERSION/$VERSION+$TORCH_VERSION$CUDA_VERSION/" fvdb/__init__.py
shell:
bash -el {0}
- name: Build wheel
run: |
conda activate fvdb_release
mkdir dist
TORCH_CUDA_ARCH_LIST="6.1;7.0;7.5;8.0;8.6+PTX" python setup.py bdist_wheel --dist-dir=dist
shell:
bash -el {0}
- name: Rename wheel
run: |
python scripts/rename_wheels.py ${{ matrix.torch-version }} `perl -E '"${{ matrix.cuda-version }}" =~ /^(\d+)\.(\d+)/; say "cu$1$2"'`
shell:
bash -el {0}
- name: Release
uses: softprops/action-gh-release@v1
if: startsWith(github.ref, 'refs/tags/')
with:
token: ${{ secrets.RELEASE_ACTION_TOKEN }}
files: |
dist/*.whl
# - name: Upload to artifacts
# uses: actions/upload-artifact@v3
# with:
# name: wheels
# path: |
# dist/*.whl
# - name: Configure AWS
# uses: aws-actions/configure-aws-credentials@v1
# with:
# aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
# aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
# aws-region: ap-northeast-1
# - name: Upload wheel
# run: |
# aws s3 sync dist s3://nksr/whl/torch-${{ matrix.torch-version }}+${{ matrix.cuda-version }} --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers
- name: Clean Conda
run: |
conda clean -pty
shell:
bash -el {0}
+221
View File
@@ -0,0 +1,221 @@
name: Running Tests
on:
pull_request:
branches: [main]
paths-ignore:
- 'debug/**'
- 'docs/**'
- 'examples/**'
- 'notebooks/**'
- 'scripts/**'
# Allow subsequent pushes to the same PR or REF to cancel any previous jobs.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: write
deployments: write
jobs:
fvdb-build:
if: ${{ !startsWith(github.event.pull_request.title, 'Draft:') }}
name: fVDB Build
runs-on:
- self-hosted
container:
image: aswf/ci-openvdb:2024
env:
PYTHONPATH: ""
options: --rm
defaults:
run:
shell: bash -el {0}
steps:
- uses: actions/checkout@v4
- name: Set up fvdb_build Conda env
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-version: latest
conda-remove-defaults: "true"
activate-environment: fvdb_build
environment-file: env/build_environment.yml
- name: Buid fvdb
run: |
TORCH_CUDA_ARCH_LIST="7.0;7.5;8.0;8.6+PTX" MAX_JOBS=$(($(nproc) < $(free -g | awk '/^Mem:/{jobs=int($4/2.5); if(jobs<1) jobs=1; print jobs}') ? $(nproc) : $(free -g | awk '/^Mem:/{jobs=int($4/2.5); if(jobs<1) jobs=1; print jobs}'))) conda run --no-capture-output -n fvdb_build python setup.py bdist_wheel --dist-dir=dist
- name: Upload package
uses: actions/upload-artifact@v4
with:
name: fvdb-test-package
path: dist/*.whl
retention-days: 2
- name: Cleanup
if: always()
run: |
echo "Cleaning up /__w/_temp directory"
sudo rm -rf /__w/_temp/*
echo "Cleanup completed"
fvdb-unit-test:
needs: [fvdb-build]
name: fVDB Unit Tests
runs-on:
- self-hosted
container:
image: aswf/ci-openvdb:2024
env:
PYTHONPATH: ""
options: --rm
defaults:
run:
shell: bash -el {0}
steps:
- uses: actions/checkout@v4
- name: Set up fvdb_test Conda env
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-version: latest
conda-remove-defaults: "true"
activate-environment: fvdb_test
environment-file: env/test_environment.yml
- name: Download package
uses: actions/download-artifact@v4
with:
name: fvdb-test-package
path: ./dist
- name: Install package
run: |
conda activate fvdb_test
pip install ./dist/*.whl
- name: Run tests
run: |
cd tests;
pytest -v unit
- name: Cleanup
if: always()
run: |
echo "Cleaning up /__w/_temp directory"
sudo rm -rf /__w/_temp/*
echo "Cleanup completed"
fvdb-docs-test:
needs: [fvdb-build]
name: fVDB Documentation Tests
runs-on:
- self-hosted
container:
image: aswf/ci-openvdb:2024
env:
PYTHONPATH: ""
options: --rm
defaults:
run:
shell: bash -el {0}
steps:
- uses: actions/checkout@v4
- name: Set up fvdb_test Conda env
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-version: latest
conda-remove-defaults: "true"
activate-environment: fvdb_test
environment-file: env/test_environment.yml
- name: Download package
uses: actions/download-artifact@v4
with:
name: fvdb-test-package
path: ./dist
- name: Install package
run: |
conda activate fvdb_test
pip install ./dist/*.whl
- name: Run tests
run: |
cd .. && pytest --markdown-docs fvdb/docs
- name: Cleanup
if: always()
run: |
echo "Cleaning up /__w/_temp directory"
sudo rm -rf /__w/_temp/*
echo "Cleanup completed"
fvdb-benchmarks:
needs: [fvdb-build]
name: fVDB Continuous Benchmarking
runs-on:
- self-hosted
container:
image: aswf/ci-openvdb:2024
env:
PYTHONPATH: ""
options: --rm
defaults:
run:
shell: bash -el {0}
steps:
- uses: actions/checkout@v4
- name: Set up fvdb_test Conda env
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-version: latest
conda-remove-defaults: "true"
activate-environment: fvdb_test
environment-file: env/test_environment.yml
- name: Download package
uses: actions/download-artifact@v4
with:
name: fvdb-test-package
path: ./dist
- name: Install package
run: |
conda activate fvdb_test
pip install ./dist/*.whl
- name: Disable git ownership verification
run: |
git config --global --add safe.directory "$(pwd)"
- name: Run benchmarks
run: |
cd tests;
pytest benchmark --benchmark-json benchmark/output.json
- name: Store benchmark result
uses: benchmark-action/github-action-benchmark@v1
with:
name: Python Benchmark with pytest-benchmark
tool: 'pytest'
output-file-path: tests/benchmark/output.json
# Use personal access token instead of GITHUB_TOKEN due to https://github.community/t/github-action-not-triggering-gh-pages-upon-push/16096
github-token: ${{ secrets.GITHUB_TOKEN }}
auto-push: true
# Show alert with commit comment on detecting possible performance regression
alert-threshold: '200%'
comment-on-alert: true
fail-on-alert: true
alert-comment-cc-users: '@swahtz'
- name: Cleanup
if: always()
run: |
echo "Cleaning up /__w/_temp directory"
sudo rm -rf /__w/_temp/*
echo "Cleanup completed"
+60
View File
@@ -0,0 +1,60 @@
# Generated version file
fvdb/version.py
# Generated compile_commands.json symlink
compile_commands.json
# Build files
/build/*
/src/benchmarks/build/*
/src/tests/build/*
/src/build/*
*.creator
*.includes
*.files
*.config
*.obj
*.off
__pycache__
*.pyc
*.creator.user
*.egg-info
*.so
dist
.cache
.idea
.eggs
cmake-build-debug
external
.polyscope.ini
imgui.ini
datasets/lib/external
datasets/lib/build
*.png
!docs/imgs/**/*.png
!notebooks/img/**/*.png
*.ply
apex/
*.zip
*.npz
!kernel.npz
*.npy
*.o
*.h5
docs/_build
fvdb/include/
trash
scratch
_build
docs/build
*tmp*.py
*.whl
tests/benchmark/data
releases/
/data/
/data/*
!tests/data
.vscode/launch.json
lightning_logs
tests/data/
+30
View File
@@ -0,0 +1,30 @@
# Gitlab CI Pipeline for building and deploying the documentation
default:
image: mambaorg/micromamba:focal-cuda-12.1.1
tags:
- os/linux
- type/docker
before_script:
# - apt-get update
- export TORCH_CUDA_ARCH_LIST="Turing"
- export FORCE_CUDA=1
# install all required libraries using existing conda environment requirements file:
- micromamba env create -f env/build_environment.yml
- eval "$(micromamba shell hook --shell bash)"
- micromamba activate
- micromamba activate fvdb_build
pages:
stage: deploy
script:
# build and publish automated documentation with Sphinx:
- export MAX_JOBS=$(free -g | awk '/^Mem:/{jobs=int($4/3); if(jobs<1) jobs=1; print jobs}')
- python setup.py build_ext --inplace
- sphinx-build -E -a docs/ public/
artifacts:
paths:
- public
only:
- main # Documentation will only be generated on the main branch
+51
View File
@@ -0,0 +1,51 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${env:CONDA_PREFIX}/envs/fvdb/include",
"${env:CONDA_PREFIX}/envs/fvdb/targets/x86_64-linux/include/",
"${workspaceFolder}/src/tests/build/_deps/googletest-src/googletest/include",
"${workspaceFolder}/src/tests/build/_deps/googletest-src/googlemock/include",
"${workspaceFolder}/src/tests/build/_deps/nvtx3-src/include",
"${workspaceFolder}/src",
"${workspaceFolder}/../nanovdb",
"${workspaceFolder}/external/glm",
"${workspaceFolder}/external/cudnn_fe/include",
"${workspaceFolder}/external/cutlass/include",
"${workspaceFolder}/external/tinyply/source",
"${env:CONDA_PREFIX}/envs/fvdb/include/python3.11",
"${env:CONDA_PREFIX}/envs/fvdb/lib/python3.11/site-packages/torch/include",
"${env:CONDA_PREFIX}/envs/fvdb/lib/python3.11/site-packages/torch/include/torch/csrc/api/include",
"${env:CONDA_PREFIX}/envs/fvdb/lib/python3.11/site-packages/torch/include/THC",
"${env:CONDA_PREFIX}/envs/fvdb/lib/python3.11/site-packages/torch/include/c10",
"${default}"
],
"macFrameworkPath": [
"${default}"
],
"defines": [
"${default}",
"NDEVELOP_IDE_ONLY",
"__CUDA_ARCH__=800"
],
"forcedInclude": [
"${default}"
],
"compileCommands": "${default}",
"browse": {
"limitSymbolsToIncludedHeaders": true,
"databaseFilename": "${default}",
"path": [
"${default}"
]
},
"intelliSenseMode": "${default}",
"cStandard": "${default}",
"cppStandard": "c++20",
"compilerPath": "${default}",
"configurationProvider": "ms-vscode.cpptools"
}
],
"version": 4
}
+15
View File
@@ -0,0 +1,15 @@
{
// See https://go.microsoft.com/fwlink/?LinkId=827846 to learn about workspace recommendations.
// Extension identifier format: ${publisher}.${name}. Example: vscode.csharp
// List of extensions which should be recommended for users of this workspace.
"recommendations": [
"llvm-vs-code-extensions.vscode-clangd",
"ms-vscode.cpptools-extension-pack",
"Seaube.clangformat"
],
// List of extensions recommended by VS Code that should not be recommended for users of this workspace.
"unwantedRecommendations": [
]
}
+58
View File
@@ -0,0 +1,58 @@
{
"files.associations": {
"*.cu": "cuda-cpp",
"*.cuh": "cuda-cpp"
},
"files.trimTrailingWhitespace": true,
"files.insertFinalNewline": true,
"editor.rulers": [ 100 ],
"C_Cpp.intelliSenseEngine": "disabled",
"C_Cpp.enhancedColorization": "enabled",
"C_Cpp.errorSquiggles": "enabledIfIncludesResolve",
"C_Cpp.autocomplete": "default",
"C_Cpp.default.compilerPath": "${env:CONDA_PREFIX}/envs/fvdb/bin/x86_64-conda-linux-gnu-c++",
"terminal.integrated.defaultLocation": "editor",
"cmake.environment": {
"PATH": "${env:CONDA_PREFIX}/envs/fvdb/bin:${env:PATH}"
},
"cmake.configureOnOpen": false,
"C_Cpp.formatting": "clangFormat",
"C_Cpp.clang_format_path": "${env:CONDA_PREFIX}/envs/fvdb/bin/clang-format-18",
"C_Cpp.autoAddFileAssociations": false,
"[cpp]": {
"editor.defaultFormatter": "seaube.clangformat",
"editor.formatOnSave": true
},
"[cuda-cpp]": {
"editor.defaultFormatter": "seaube.clangformat",
"editor.formatOnSave": true
},
"[python]": {
"editor.defaultFormatter": "ms-python.black-formatter",
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": "explicit"
},
},
"python.analysis.typeCheckingMode": "basic",
"python.testing.unittestArgs": [
"-v", "-s", "."
],
"python.testing.pytestEnabled": false,
"python.testing.unittestEnabled": true,
"black-formatter.args": [
"--target-version=py311",
"--line-length=120"
],
"python.analysis.diagnosticSeverityOverrides": {
"reportPrivateImportUsage": "none",
},
"isort.args": [
"--profile", "black"
],
"json.format.keepLines": true,
"clangd.path": "${env:CONDA_PREFIX}/envs/fvdb/bin/clangd",
"clangFormat.executable.linux": "${env.CONDA_PREFIX}/envs/fvdb/bin/clang-format",
}
+98
View File
@@ -0,0 +1,98 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
cmake_minimum_required(VERSION 3.25 FATAL_ERROR)
# Set global download timeout to 600 seconds (10 minutes)
set(CMAKE_DOWNLOAD_TIMEOUT 600 CACHE STRING "Time allowed for downloads in seconds")
project(${SKBUILD_PROJECT_NAME} VERSION ${SKBUILD_PROJECT_VERSION} LANGUAGES CXX)
find_package(Python3 COMPONENTS Development Interpreter)
message(STATUS "CMAKE_BUILD_PARALLEL_LEVEL: ${CMAKE_BUILD_PARALLEL_LEVEL}")
include(${CMAKE_CURRENT_SOURCE_DIR}/src/cmake/get_cpm.cmake)
include(${CMAKE_CURRENT_SOURCE_DIR}/src/cmake/get_nvtx.cmake)
include(${CMAKE_CURRENT_SOURCE_DIR}/src/cmake/get_torch.cmake)
include(${CMAKE_CURRENT_SOURCE_DIR}/src/cmake/get_pybind11.cmake)
include(${CMAKE_CURRENT_SOURCE_DIR}/src/cmake/configure_torch_pybind11.cmake)
# Generate compile_commands.json
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
# configure libfvdb
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/src/)
# Source files
set(FVDB_BINDINGS_CPP_FILES
src/python/Bindings.cpp
src/python/GaussianSplatBinding.cpp
src/python/GridBatchBinding.cpp
src/python/JaggedTensorBinding.cpp)
# Build library
Python3_add_library(_Cpp ${FVDB_BINDINGS_CPP_FILES})
if (DEFINED TORCH_PYBIND11_INCLUDE_DIR)
target_link_libraries(_Cpp PUBLIC torch_pybind11_headers)
else()
target_link_libraries(_Cpp PUBLIC pybind11::headers)
endif()
set_target_properties(_Cpp PROPERTIES
INTERPROCEDURAL_OPTIMIZATION ON
CXX_VISIBILITY_PRESET hidden
VISIBILITY_INLINES_HIDDEN ON)
target_link_options(_Cpp PRIVATE "-Wl,--no-as-needed")
set_target_properties(_Cpp PROPERTIES
BUILD_RPATH "\$ORIGIN"
INSTALL_RPATH "\$ORIGIN"
INSTALL_RPATH_USE_LINK_PATH FALSE
BUILD_WITH_INSTALL_RPATH TRUE
CXX_STANDARD 20
CXX_STANDARD_REQUIRED ON
CXX_EXTENSIONS ON
POSITION_INDEPENDENT_CODE ON
INTERFACE_POSITION_INDEPENDENT_CODE ON)
target_include_directories(_Cpp PUBLIC
${CMAKE_CURRENT_SOURCE_DIR}/src
${TORCH_INCLUDE_DIRS})
target_link_libraries(_Cpp PRIVATE
${TORCH_PYTHON_LIBRARY}
${TORCH_LIBRARIES}
fvdb)
target_compile_options(_Cpp PRIVATE
$<$<COMPILE_LANGUAGE:CXX>:
#"-Wno-unused-variable"
"-fvisibility=hidden"
"-fvisibility-inlines-hidden"
"-Wall"
"-Werror"
"${TORCH_CXX_FLAGS}"
>)
target_compile_definitions(_Cpp PRIVATE
TORCH_EXTENSION_NAME=_Cpp
TORCH_API_INCLUDE_EXTENSION_H)
# Configure the pybind11 module with PyTorch compatibility settings
configure_torch_pybind11(_Cpp)
install(TARGETS _Cpp fvdb
LIBRARY
DESTINATION fvdb
COMPONENT PythonModule
RUNTIME
DESTINATION fvdb
COMPONENT PythonModule)
# Add at the beginning after project() call:
set(VERSION_PY_PATH "${CMAKE_CURRENT_SOURCE_DIR}/fvdb/version.py")
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/fvdb/version.py.in" "${VERSION_PY_PATH}" @ONLY)
# Make sure version.py is included in the installation
install(FILES "${VERSION_PY_PATH}" DESTINATION fvdb)
+19
View File
@@ -0,0 +1,19 @@
FROM condaforge/miniforge3:24.11.3-2
ARG MODE=dev
RUN echo "Building fVDB container in $MODE mode"
# used for cross-compilation in docker build
ENV FORCE_CUDA=1
WORKDIR /openvdb
# force this CUDA version to be used to build the docker container because `docker build` does not
# expose the GPU to the docker build process for it to be detected
ENV CONDA_OVERRIDE_CUDA=12.0
# copy env/dev_environment.yml to /tmp/
COPY fvdb/env/dev_environment.yml /tmp/
RUN conda env create -f /tmp/dev_environment.yml
RUN conda init
RUN echo "conda activate fvdb" >> ~/.bashrc
+244
View File
@@ -0,0 +1,244 @@
# *ƒ*(VDB)
This repository contains the code for *f*VDB, a data structure for encoding and operating on *sparse voxel hierarchies* of features in PyTorch. A sparse voxel hierarchy is a coarse-to-fine hierarchy of sparse voxel grids such that every fine voxel is contained within some coarse voxel. The image below illustrates an example. *f*VDB supports using PyTorch Tensors to represent features at the corners and centers of voxels in a hierarchy and enables a number of differentiable operations on these Tensors (*e.g.* trilinear interpolation, convolution, splatting, ray tracing).
<p align="center">
<img src="docs/imgs/fvdb_teaser.png" style="width: 40%;"alt="fVDB Teaser">
<!-- <img src="docs/imgs/readme/av_screenshot.png" style="width: 100%;"alt="fVDB Teaser"> -->
<figcaption style="text-align: center; font-style: italic;">An example of a sparse voxel hierarchy with 3 levels. Each fine voxel is contained within exactly one coarse voxel.</figcaption>
</p>
## Learning to Use *f*VDB
After [installing *f*VDB](#installing-fvdb), we recommend starting with our walk-through [notebooks](notebooks) which provide a gentle, illustrated introduction to the main concepts and operations in *f*VDB.
Once familiar with the basics, [Usage Examples](#usage-examples) introduces a few of the practical python scripts that can be further explored in the [examples](examples) directory.
Lastly, our [documentation](docs) provides deeper details on the concepts as well as an exhaustive set of illustrations of all the operations available in *f*VDB and an API reference. The [documentation can be built locally](#building-documentation) or can be accessed online at https://www.openvdb.org/documentation/fvdb/.
## Installing *f*VDB
During the project's initial development stages, it is necessary to [run the build steps](#building-fvdb-from-source) to install ƒVDB. Eventually, ƒVDB will be provided as a pre-built, installable package from anaconda. We support building the latest ƒVDB version for the following dependent library configurations:
| PyTorch | Python | CUDA |
| -------------- | ----------- | ------------ |
| 2.4.0-2.4.1 | 3.10 - 3.12 | 12.1 - 12.4 |
** Notes:**
* Linux is the only platform currently supported (Ubuntu >= 20.04 recommended).
* A CUDA-capable GPU with Ampere architecture or newer (i.e. compute capability >=8.0) is recommended to run the CUDA-accelerated operations in ƒVDB. A GPU with compute capabililty >=7.0 (Volta architecture) is the minimum requirement but some operations and data types are not supported.
## Building *f*VDB from Source
### Environment Management
ƒVDB is a Python library implemented as a C++ Pytorch extension. Of course you can build ƒVDB in whatever environment suits you, but we provide three paths to constructing reliable environments for building and running ƒVDB: using [docker](#setting-up-a-docker-container), [conda](#setting-up-a-conda-environment), or a Python virtual environment.
`conda` tends to be more flexible since reconfiguring toolchains and modules to suit your larger project can be dynamic, but at the same time this can be a more brittle experience compared to using a virtualized `docker` container. Using `conda` is generally recommended for development and testing, while using `docker` is recommended for CI/CD and deployment.
#### Setting up a Docker Container
Running a docker container is a great way to ensure that you have a consistent environment for building and running ƒVDB.
Our provided [`Dockerfile`](Dockerfile) constructs a Docker image which is ready to build ƒVDB. The docker image is configured to install miniforge and the `fvdb` conda environment with all the dependencies needed to build and run ƒVDB.
Building and starting the docker image is done by running the following command from the fvdb directory:
```shell
docker compose run --rm fvdb-dev
```
When you are ready to build ƒVDB, run the following command within the docker container. `TORCH_CUDA_ARCH_LIST` specifies which CUDA architectures to build for.
```shell
conda activate fvdb;
cd /openvdb/fvdb;
TORCH_CUDA_ARCH_LIST="7.0;7.5;8.0;8.6+PTX" \
./build.sh install
```
Additional information about the ƒVDB docker setup, with troubleshooting for common errors, can be found
here: [`ƒVDB Docker`](docs/markdown/docker_readme.md)
#### Setting up a Conda Environment
*f*VDB can be used with any Conda distribution installed on your system. Below is an installation guide using
[miniforge](https://github.com/conda-forge/miniforge). You can skip steps 1-3 if you already have a Conda installation.
1. Download and Run Install Script. Copy the command below to download and run the [miniforge install script](https://github.com/conda-forge/miniforge?tab=readme-ov-file#unix-like-platforms-macos--linux):
```shell
curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh
```
2. Follow the prompts to customize Conda and run the install. Note, we recommend saying yes to enable `conda-init`.
3. Start Conda. Open a new terminal window, which should now show Conda initialized to the `(base)` environment.
4. Create the `fvdb` conda environment. Run the following command from the directory containing this README:
```shell
conda env create -f env/dev_environment.yml
```
5. Activate the *f*VDB environment:
```shell
conda activate fvdb
```
##### Other available environments
* `fvdb_build`: Use `env/build_environment.yml` for a minimum set of dependencies needed just to build/package *f*VDB (note this environment won't have all the runtime dependencies needed to `import fvdb`).
* `fvdb_test`: Use `env/test_environment.yml` for a runtime environment which has only the packages required to run the unit tests after building ƒVDB. This is the environment used by the CI pipeline to run the tests after building ƒVDB in the `fvdb_build` environment.
* `fvdb_learn`: Use `env/learn_environment.yml` for additional runtime requirements and packages needed to run the [notebooks](notebooks) or [examples](examples) and view their visualizations.
#### Setting up a Python virtual environment
After creating a Python virtual environment, proceed to install the exact version of PyTorch that corresponds to your CUDA version. Then, install the rest of the build requirements.
```shell
python -m venv fvdb
source fvdb/bin/activate
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu126
pip install -r env/build_requirements.txt
```
When you're ready to build fVDB, run the following command after activating the Python virtual environment
```shell
TORCH_CUDA_ARCH_LIST="7.0;7.5;8.0;8.6+PTX" ./build.sh install
```
### Building *f*VDB
**:warning: Note:** Compilation can be very memory-consuming. As part of our build script, we set the `CMAKE_BUILD_PARALLEL_LEVEL` environment variable to control compilation job parallelism with a value that we find works well for most machines (allowing for one job every 2.5GB of memory) but this can be overridden by setting the `CMAKE_BUILD_PARALLEL_LEVEL` environment variable to a different value.
You can either perform an install:
```shell
./build.sh
```
or if you would like to build a packaged wheel for installing in other environments, you can run the following command:
```shell
./build.sh wheel
```
### Running Tests
To make sure that everything works by running tests:
```shell
cd tests
pytest unit
```
### Building Documentation
To build the documentation, simply run:
```shell
python setup.py build_ext --inplace
sphinx-build -E -a docs/ build/sphinx
# View the docs
open build/sphinx/index.html
```
### Setting up Intellisense with clangd in Visual Studio Code
Please see the guide [`Clangd for Intellisense in fVDB`](docs/markdown/clangd.md)
## Usage Examples
The [examples](examples) directory contains a number of useful illustrations using the `fvdb` Python package. The sections below show some notable examples and their outputs. Run all commands from the directory containing this README.
### Trilinear sampling of grids
```
python examples/sample_trilinear.py
```
This script generates a grid with scalars at the corners of each voxel and samples this grid at points. The visualization below shows the points colored according to their sampled values as well as the values at grid corners.
<p align="center">
<img src="docs/imgs/readme/trilerp.png" style="width: 40%;"alt="fVDB trilinear interpolation demo">
<figcaption style="text-align: center; font-style: italic;">Trilinearly interpolate the corner values at the points.</figcaption>
</p>
### Trilinear splatting into grids
```
python examples/splat_trilinear.py
```
This script splats normals of a point cloud onto grid centers. The green arrows represent the values of the normals splatted onto each grid center
<p align="center">
<img src="docs/imgs/readme/splat.png" style="width: 40%;"alt="fVDB trilinear splatting demo">
<figcaption style="text-align: center; font-style: italic;">Splat the normals at the blue points into the center of each grid cell. The green arrows are the splatted normals</figcaption>
</p>
### Tracing voxels along rays (hierarchical DDA)
```
python examples/ray_voxel_marching.py
```
This script demonstrates finding the first `N` voxels which lie along a ray (returning thier index as well as their entry and exit points).
<p align="center">
<img src="docs/imgs/readme/rayvox.png" style="width: 70%;"alt="fVDB ray voxel marching">
<figcaption style="text-align: center; font-style: italic;">Find the voxels (yellow) which intersect the pink rays eminating from the green dot.</figcaption>
</p>
### Tracing contiguous segments along rays
```
python examples/ray_segment_marching.py
```
This script demonstrates finding the first `N` continuous segments of voxels which lie along a ray (returning thier index as well as their entry and exit points).
<p align="center">
<img src="docs/imgs/readme/rayseg.png" style="width: 70%;"alt="fVDB ray voxel marching">
<figcaption style="text-align: center; font-style: italic;">Find the contiguous segments of voxels (red and blue lines) which intersect the cyan rays eminating from the pink dot.</figcaption>
</p>
### Backpropagating through sampling and splatting
```
python examples/overfit_sdf.py
```
This scripts fits SDF values at a grid corner to the SDF of a mesh using gradient descent.
<p align="center">
<img src="docs/imgs/readme/fitsdf.png" style="width: 70%;"alt="fVDB SDF fitting">
<figcaption style="text-align: center; font-style: italic;">SDF values at grid corners (colored dots) fitted using gradient descent to the SDF of a mesh.</figcaption>
</p>
The following scripts also show how to bakcprop through splatting and sampling with fVDB:
```
python scripts/debug_grad_trilerp.py
```
```
python scripts/debug_grad_splat.py
```
## Code Structure
The main source code for fVDB lives in the [src](src) directory. There are several important files here:
* `src/python/Bindings.cpp` exposes functionality directly to Python. It is mainly a wrapper around the core classes such as `fvdb::GridBatch` and `fvdb::JaggedTensor`.
* `src/GridBatch.h` contains the implementation of `fvdb::GridBatch` which is the core data structure on which fVDB is built. A `GridBatch` acts as a map between `(i, j, k)` integer coordinates and offsets in linear memory. This mapping can be used to perform a host of operations. The methods in this class are mostly lightweight wrappers around a set of CPU and CUDA *kernels*. The function prototypes for these kernels are defined in `src/detail/ops/Ops.h`.
* `src/detail/ops/Ops.h` contains the function prototypes for the main kernels used by fVDB. Host and device kernel implementations are provided in the `src/detail/ops/*.cu` source files.
* `src/detail/autograd` contains C++ implementations of PyTorch autograd functions for differentiable operations. `#include <detail/autograd/Autograd.h>` includes all of the functions in this directory.
* `src/detail/utils/nanovdb` contains a number of utilities which make it easier to use NanoVDB.
## References
Please consider citing this when using *f*VDB in a project. You can use the citation BibTeX:
```bibtex
@article{williams2024fvdb,
title={fVDB: A Deep-Learning Framework for Sparse, Large-Scale, and High-Performance Spatial Intelligence},
author={Williams, Francis and Huang, Jiahui and Swartz, Jonathan and Klar, Gergely and Thakkar, Vijay and Cong, Matthew and Ren, Xuanchi and Li, Ruilong and Fuji-Tsang, Clement and Fidler, Sanja and Sifakis, Eftychios and Museth, Ken},
journal={ACM Transactions on Graphics (TOG)},
volume={43},
number={4},
pages={133:1--133:15},
year={2024},
publisher={ACM New York, NY, USA}
}
```
## Contact
Executable
+195
View File
@@ -0,0 +1,195 @@
#!/bin/bash
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
usage() {
echo "Usage: $0 [-h|--help] [build_type] [options...]"
echo ""
echo "Builds or tests FVDB."
echo ""
echo "Arguments:"
echo " build_type Specifies the build operation. Can be one of:"
echo " install - Build and install the package (default)."
echo " wheel - Build the Python wheel."
echo " ctest - Run tests (requires tests to be built)."
echo " debug - Build in debug mode with full debug symbols and no optimizations."
echo ""
echo "Options:"
echo " -h, --help Display this help message and exit."
echo ""
echo "Build Modifiers (for 'install' and 'wheel' build types, typically passed after build_type):"
echo " gtests Enable building tests (sets FVDB_BUILD_TESTS=ON)."
echo " benchmarks Enable building benchmarks (sets FVDB_BUILD_BENCHMARKS=ON)."
echo " verbose Enable verbose build output for pip and CMake."
echo ""
echo " Any modifier arguments not matching above are passed through to pip."
exit 0
}
setup_parallel_build_jobs() {
# Calculate the optimal number of parallel build jobs based on available RAM
RAM_GB=$(free -g | awk '/^Mem:/{print $4}')
if [ -z "$RAM_GB" ]; then
echo "Error: Unable to determine available RAM"
exit 1
fi
JOB_RAM_GB=2.5
# Calculate max jobs based on RAM
RAM_JOBS=$(awk -v ram="$RAM_GB" -v job_ram="$JOB_RAM_GB" 'BEGIN { print int(ram / job_ram) }')
# Get number of processors
NPROC=$(nproc)
# if CMAKE_BUILD_PARALLEL_LEVEL is set, use that
if [ -n "$CMAKE_BUILD_PARALLEL_LEVEL" ]; then
echo "Using CMAKE_BUILD_PARALLEL_LEVEL=$CMAKE_BUILD_PARALLEL_LEVEL"
else
# Determine the minimum of RAM-based jobs and processor count
if [ "$RAM_JOBS" -lt "$NPROC" ]; then
CMAKE_BUILD_PARALLEL_LEVEL="$RAM_JOBS"
else
CMAKE_BUILD_PARALLEL_LEVEL="$NPROC"
fi
# Ensure at least 1 job
if [ "$CMAKE_BUILD_PARALLEL_LEVEL" -lt 1 ]; then
CMAKE_BUILD_PARALLEL_LEVEL=1
fi
echo "Setting CMAKE_BUILD_PARALLEL_LEVEL to $CMAKE_BUILD_PARALLEL_LEVEL based on available RAM to target $JOB_RAM_GB GB per translation unit"
export CMAKE_BUILD_PARALLEL_LEVEL
fi
}
if [[ "$1" == "-h" || "$1" == "--help" ]]; then
usage
fi
# Determine BUILD_TYPE from the first positional argument, default to 'install'.
# Handle shifting of arguments appropriately.
_first_arg_val="$1"
BUILD_TYPE="install" # Default build type
if [[ -n "$_first_arg_val" ]]; then
if [[ "$_first_arg_val" == "install" || "$_first_arg_val" == "wheel" || "$_first_arg_val" == "ctest" ]]; then
BUILD_TYPE="$_first_arg_val"
shift # Consume the build_type argument
else
# _first_arg_val is not a recognized build type. Print usage and exit.
echo "Error: Argument '$_first_arg_val' is not a recognized build_type."
usage # This will also exit
fi
fi
CONFIG_SETTINGS=""
PASS_THROUGH_ARGS=""
while (( "$#" )); do
is_config_arg_handled=false
if [[ "$BUILD_TYPE" == "install" || "$BUILD_TYPE" == "wheel" ]]; then
if [[ "$1" == "gtests" ]]; then
echo "Detected 'gtests' flag for $BUILD_TYPE build. Enabling FVDB_BUILD_TESTS."
CONFIG_SETTINGS+=" --config-settings=cmake.define.FVDB_BUILD_TESTS=ON"
is_config_arg_handled=true
elif [[ "$1" == "benchmarks" ]]; then
echo "Detected 'benchmarks' flag for $BUILD_TYPE build. Enabling FVDB_BUILD_BENCHMARKS."
CONFIG_SETTINGS+=" --config-settings=cmake.define.FVDB_BUILD_BENCHMARKS=ON"
is_config_arg_handled=true
elif [[ "$1" == "verbose" ]]; then
echo "Enabling verbose build"
CONFIG_SETTINGS+=" -v -C build.verbose=true"
is_config_arg_handled=true
elif [[ "$1" == "debug" ]]; then
echo "Enabling debug build"
CONFIG_SETTINGS+=" --config-settings=cmake.build-type=debug"
is_config_arg_handled=true
fi
fi
if ! $is_config_arg_handled; then
# Append other arguments, handling potential spaces safely
PASS_THROUGH_ARGS+=" $(printf "%q" "$1")"
fi
shift
done
# Construct PIP_ARGS with potential CMake args and other pass-through args
export PIP_ARGS="--no-build-isolation$CONFIG_SETTINGS$PASS_THROUGH_ARGS"
if [ "$BUILD_TYPE" != "ctest" ]; then
setup_parallel_build_jobs
fi
# if the user specified 'wheel' as the build type, then we will build the wheel
if [ "$BUILD_TYPE" == "wheel" ]; then
echo "Build wheel"
echo "pip wheel . --wheel-dir dist/ $PIP_ARGS"
pip wheel . --wheel-dir dist/ $PIP_ARGS
elif [ "$BUILD_TYPE" == "install" ]; then
echo "Build and install package"
echo "pip install --force-reinstall $PIP_ARGS ."
pip install --force-reinstall $PIP_ARGS .
# TODO: Fix editable install
# else
# echo "Build and install editable package"
# echo "pip install $PIP_ARGS -e . "
# pip install $PIP_ARGS -e .
elif [ "$BUILD_TYPE" == "ctest" ]; then
# --- Ensure Test Data is Cached via CMake Configure Step ---
echo "Ensuring test data is available in CPM cache..."
if [ -z "$CPM_SOURCE_CACHE" ]; then
echo "CPM_SOURCE_CACHE is not set"
else
echo "Using CPM_SOURCE_CACHE: $CPM_SOURCE_CACHE"
fi
# Assume this script runs from the source root directory
SOURCE_DIR=$(pwd)
TEMP_BUILD_DIR="build_temp_download_data"
# Clean up previous temp dir and create anew
rm -rf "$TEMP_BUILD_DIR"
mkdir "$TEMP_BUILD_DIR"
echo "Running CMake configure in temporary directory ($TEMP_BUILD_DIR) to trigger data download..."
pushd "$TEMP_BUILD_DIR" > /dev/null
cmake "$SOURCE_DIR/src/cmake/download_test_data"
popd > /dev/null # Back to SOURCE_DIR
# Clean up temporary directory
rm -rf "$TEMP_BUILD_DIR"
echo "Test data caching step finished."
# --- End Test Data Caching ---
# --- Find and Run Tests ---
echo "Searching for test build directory..."
# Find the directory containing the compiled tests (adjust if needed)
# Using -print -quit to stop after the first match for efficiency
BUILD_DIR=$(find build -name tests -type d -print -quit)
if [ -z "$BUILD_DIR" ]; then
echo "Error: Could not find build directory with tests"
echo "Please enable tests by building with pip argument"
echo "-C cmake.define.FVDB_BUILD_TESTS=ON"
exit 1
fi
echo "Found test build directory: $BUILD_DIR"
# Run ctest within the test build directory
pushd "$BUILD_DIR" > /dev/null
echo "Running ctest..."
ctest --output-on-failure
CTEST_EXIT_CODE=$?
popd > /dev/null # Back to SOURCE_DIR
echo "ctest finished with exit code $CTEST_EXIT_CODE."
exit $CTEST_EXIT_CODE
else
echo "Invalid build/run type: $BUILD_TYPE"
echo "Valid build/run types are: wheel, install, ctest"
exit 1
fi
+25
View File
@@ -0,0 +1,25 @@
version: '3'
services:
fvdb-dev:
build:
context: ../
dockerfile: fvdb/Dockerfile
args:
MODE: dev
volumes:
- ../:/openvdb
# Uncomment and adjust the following as needed
# ports:
# - "8888:8888" # Example for Jupyter notebook
entrypoint: /bin/bash
stdin_open: true
tty: true
environment:
- NVIDIA_VISIBLE_DEVICES=all
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
+5
View File
@@ -0,0 +1,5 @@
GridBatch
===============
.. autoclass:: fvdb.GridBatch
:members:
+5
View File
@@ -0,0 +1,5 @@
JaggedTensor
===============
.. autoclass:: fvdb.JaggedTensor
:members:
+20
View File
@@ -0,0 +1,20 @@
fvdb.nn
=======
`fvdb.nn` is a collection of neural network layers to build sparse neural networks.
.. autoclass:: fvdb.nn.VDBTensor
.. autoclass:: fvdb.nn.MaxPool
.. autoclass:: fvdb.nn.AvgPool
.. autoclass:: fvdb.nn.UpsamplingNearest
.. autoclass:: fvdb.nn.FillFromGrid
.. autoclass:: fvdb.nn.SparseConv3d
.. autoclass:: fvdb.nn.GroupNorm
.. autoclass:: fvdb.nn.Linear
.. autoclass:: fvdb.nn.ReLU
.. autoclass:: fvdb.nn.LeakyReLU
.. autoclass:: fvdb.nn.SELU
.. autoclass:: fvdb.nn.SiLU
.. autoclass:: fvdb.nn.Tanh
.. autoclass:: fvdb.nn.Sigmoid
.. autoclass:: fvdb.nn.Dropout
+4
View File
@@ -0,0 +1,4 @@
fvdb.utils
==================
.. autofunction:: fvdb.utils.build_ext.FVDBExtension
+91
View File
@@ -0,0 +1,91 @@
# Copyright Contributors to the OpenVDB Project
# SPDX-License-Identifier: Apache-2.0
#
# -- Path setup --------------------------------------------------------------
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
import os
import sys
sys.path.insert(0, os.path.abspath(".."))
# -- Project information -----------------------------------------------------
project = "fVDB"
copyright = "Contributors to the OpenVDB Project"
author = "Contributors to the OpenVDB Project"
# -- General configuration ---------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ["sphinx.ext.autodoc", "sphinx.ext.viewcode", "sphinx.ext.napoleon", "myst_parser"]
myst_enable_extensions = [
"amsmath",
"attrs_inline",
"colon_fence",
"deflist",
"dollarmath",
"fieldlist",
"html_admonition",
"html_image",
"linkify",
"replacements",
"smartquotes",
"strikethrough",
"substitution",
"tasklist",
]
# Fix return-type in google-style docstrings
napoleon_custom_sections = [("Returns", "params_style")]
# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]
source_suffix = [".rst", ".md"]
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
autodoc_default_options = {"undoc-members": "forward, extra_repr"}
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = "sphinx_rtd_theme"
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
# html_static_path = ['imgs']
# -- Custom hooks ------------------------------------------------------------
def process_signature(app, what, name, obj, options, signature, return_annotation):
if signature is not None:
signature = signature.replace("._Cpp", "")
signature = signature.replace("fvdb::", "fvdb.")
if return_annotation is not None:
return_annotation = return_annotation.replace("._Cpp", "")
return_annotation = return_annotation.replace("fvdb::", "fvdb.")
return signature, return_annotation
def setup(app):
app.connect("autodoc-process-signature", process_signature)
Binary file not shown.

After

Width:  |  Height:  |  Size: 418 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 415 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 53 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 362 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 811 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 193 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 958 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 622 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 190 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 348 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 352 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 69 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 234 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 813 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 912 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 162 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.5 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 806 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 480 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 265 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 216 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 382 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 312 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 256 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 395 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 287 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 767 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 628 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 428 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 988 KiB

+55
View File
@@ -0,0 +1,55 @@
Welcome to fVDB!
=================
fVDB, inspired by function notation to resemble :math:`f(VDB)`, is a data structure for encoding and operating on *sparse* voxel hierarchies of features in PyTorch.
A sparse voxel hierarchy is a coarse-to-fine hierarchy of sparse voxel grids such that every fine voxel is contained within some coarse voxel.
fvdb supports storing PyTorch tensors at the corners and centers of voxels in a hierarchy and enables a number of differentiable operations on these tensors (e.g. trilinear interpolation, splatting, ray tracing).
.. image:: imgs/fvdb_teaser.png
:align: center
:width: 400
Please refer to the tutorials for examples of how to install and use fVDB to build your own sparse voxel pipelines.
.. toctree::
:caption: Introduction
:hidden:
self
.. toctree::
:maxdepth: 1
:caption: Tutorials
tutorials/installation
tutorials/basic_concepts
tutorials/building_grids
tutorials/basic_grid_ops
tutorials/ray_tracing
tutorials/simple_unet
tutorials/io
tutorials/volume_rendering
.. toctree::
:maxdepth: 1
:caption: API References
api/grid_batch
api/jagged_tensor
.. toctree::
:maxdepth: 2
api/nn
api/utils
.. raw:: html
<hr>
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
+60
View File
@@ -0,0 +1,60 @@
# Clangd for "Intellisense" in fVDB
In fVDB we typically develop using Visual Studio Code (VSCode), both local, in containers, and
remove via SSH. To aid development we use `clangd` for intellisense (code navigation) and to
detect syntax and usage errors before compiling. With `clangd` active, you will see "red squiggles"
under actual syntax / compilation errors in your code as you edit; you will get code completion
suggestions; and you will get tooltips for definitions when you mouse over code.
The fVDB VSCode workspace includes configuration for clangd but developers new to fVDB should follow
these instructions.
## Setting up clangd in your environment
The fvdb Conda development environment installs the CUDA toolkit, but clangd runs outside the
environment and so may not find the correct CUDA installation. Therefore you should create the file
To fix this, you need to create a user clangd config file to point to the CUDA installation in the
FVDB conda environment: Create ~/.config/clangd/config.yaml with these contents. This assumes you
use the `dev_environment.yml` to create the `fvdb` Conda environment.
```yaml
# https://clangd.llvm.org/config
# Apply a config conditionally to all CUDA files
If:
PathMatch: .*\.cuh?
CompileFlags:
Add:
- "--cuda-path=/home/mharris/miniforge3/envs/fvdb"
```
The first time you load the fVDB VSCode workspace, it should prompt you to install recommended
extensions. Specifically: "llvm-vs-code-extensions.vscode-clangd", and
"ms-vscode.cpptools-extension-pack", as defined in the `extensions.json` file. Install these
extensions. With Clangd, the MS CPPTools is only used for syntax highlighting. Clangd is used for
intellisense and (CUDA) C++ file formatting.
## Troubleshooting Intellisense
Clangd uses the file `compile_commands.json` to determine the compiler command line used so that it
can build an index of the source code for "intellisense" navigation. This file is generated by
CMake when you configure the build. This happens when you run fVDB's `build.sh`. Here are some
tips for troubleshooting intellisense issues.
- If you get a red squiggle on the first character of a file, it’s usually a major failure. For
example when clangd fails to find the CUDA installation. This is often a configuration problem.
- A common problem is when you are editing a file that is not in `compile_commands.json`. if this is
the case, you may not have CMake-configured your build.
- A key indicator of this is seeing a message about ‘inferred’ commands like this in the clangd
output log `ASTWorker building file XXX with command inferred from YYY`
- Switching conda environments: if you activate a conda environment, e.g. `fvdb_build` and build
(or cmake configure), then switch to another conda environment, your `compile_commands.json` will
be out of date with the wrong include paths. You need to rerun cmake configure or rebuild in the
active conda environment.
- red squiggles in google test .cpp : did you build the tests? Are they in compile_commands.cpp?
Make sure you add the `gtests` argument to your `build.sh` command line:
```sh
./build.sh install gtests
```
+51
View File
@@ -0,0 +1,51 @@
# ƒVDB Docker Help
Running a docker container is a great way to ensure that you have a consistent environment for building and running ƒVDB.
Our provided [`Dockerfile`](../../Dockerfile) constructs a Docker image which is ready to build ƒVDB. The docker image is configured to install miniforge and the `fvdb` conda environment with all the dependencies needed to build and run ƒVDB.
## Setting up a Docker Container
Building and starting the docker image is done by running the following command from the fvdb directory:
```shell
docker compose run --rm fvdb-dev
```
When you are ready to build ƒVDB, run the following command within the docker container. `TORCH_CUDA_ARCH_LIST` specifies which CUDA architectures to build for.
```shell
conda activate fvdb;
cd /openvdb/fvdb;
TORCH_CUDA_ARCH_LIST="7.0;7.5;8.0;8.6+PTX" \
./build.sh install
```
### Troubleshooting
* **docker daemon runtime/driver errors**
Errors like these:
```shell
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].
```
```shell
docker: Error response from daemon: Unknown runtime specified nvidia.
```
most likely indicate that the [`NVIDIA Container Toolkit`](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/index.html) is not installed.
You can install it by following the [installation instructions here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
* **GPU access blocked by the operating system**
Errors like these:
```shell
Failed to initialize NVML: GPU access blocked by the operating system
Failed to properly shut down NVML: GPU access blocked by the operating system
```
may be solved by making a change to the file `/etc/nvidia-container-toolkit/config.toml`, setting
the property "no-cgroups" from "true" to "false":
```
# ...
[nvidia-container-cli]
# ...
# Below changed from previous no-cgroups = true
no-cgroups = false
# ...
```
+87
View File
@@ -0,0 +1,87 @@
# Basic Concepts
We give a high-level overview of the main concepts in fVDB. Namely, how fVDB encodes sparse voxel grids with attributes, as well as how fVDB efficiently manages batches with non-uniform numbers of elements.
## `GridBatch`: Voxel Grids with Attributes
At its core, fVDB provides means to efficiently encode mini-batches of sparse voxel grids where each voxel contains arbitrary vector or scalar attributes encoded as torch tensors. In practice, such an encoding is implemented via the `GridBatch` class.
![Minibatch2.png](../imgs/fig/Minibatch2.png)
A `GridBatch` is an indexing structure which maps 3D `ijk` coordinates to integer offsets which can be used to look up attributes in a tensor. The figure below illustrates this process for a GridBatch containing a single grid.
![gridbatch.png](../imgs/fig/gridbatch.png)
In the figure, the `GridBatch` acts as an acceleration structure which encodes the sparsity pattern of the grid (also known as the **********topology**********) and can translate grid coordinates into offsets into a data tensor.
By separating the grid topology and data tensors, the same grid can be used to operate on many different attributes without rebuilding.
Every operation in fVDB is built upon this kind of query (e.g. Sparse Convolution uses this query to look up features in the neighborhood of each voxel).
## `JaggedTensor` and Batching
Each grid in a `GridBatch` can have a different number of voxels (****e.g.**** in the mini batch of four cars above, each car has a different number of voxels). This means that unlike the dense case, fVDB needs to handle parallel operations over ***jagged batches***. I.e. batches containing different numbers of elements.
To handle jagged batches, fVDB provides a `JaggedTensor` class. Conceptually, a `JaggedTensor` is a list of tensors with shapes $[N_0, *], [N_1, *], \ldots, [N_{B-1}, *]$ where $B$ is the number of elements in the batch, $N_i$ is the number of elements in the $i^\text{th}$ batch item and $*$ is an arbitrary numer of additional dimensions that all match between the tensors. The figure below illustrates such a list of tensors pictorially.
![jaggedtensor1.png](../imgs/fig/jaggedtensor1.png)
In practice, `JaggedTensor`s are represented in memory by concatenating each tensor in the list into a single `jdata` (for Jagged Data) tensor of shape $[N_0 + N_1 + \ldots + N_{B-1}, *]$. Additionally, each `JaggedTensor` stores an additional `jidx` tensor (for Jagged Indexes) of shape $[N_0 + N_1 + \ldots + N_{B-1}]$ containing one int per element in the jagged tensor. `jidx[i]` is the batch index of the $i^\text{th}$ element of `jdata`. Finally, a `JaggedTensor` contains a `joffsets` tensor (for Jagged Offsets) of shape $[B, 2]$ which indicates the start and end positions of the $i^\text{th}$ tensor in the batch.
![jaggedtensor4.png](../imgs/fig/jaggedtensor4.png)
Similarly, each `GridBatch` also has `jidx` and `joffsets` corresponding to the batch index of each voxel in the grid, and the start and end offsets of each voxel index in a batch.
## A simple example
To illustrate the use of `GridBatch`and `JaggedTensor`, consider a simple example where we build a grid from a point cloud, splat some values onto the voxels of that grid, and then sample them again using a different set of points.
First, we construct a minibatch of grids using the input points. These input points have corresponding color attributes.
```python
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
import torch
import point_cloud_utils as pcu
# We're going to create a minibatch of two point clouds each of which
# has a different number of points
pts1, clrs1 = load_car_1_mesh(mode = "vn")
pts2, clrs2 = load_car_2_mesh(mode = "vn")
# Creating JaggedTensors: one for points and one for colors
points = fvdb.JaggedTensor([pts1, pts2])
colors = fvdb.JaggedTensor([clrs1, clrs2])
# Create a grid where the voxels each have unit sidelength
grid = fvdb.gridbatch_from_points(points, voxel_sizes=1.0)
# Indexing into a JaggedTensor returns a JaggedTensor
print(points[0].jdata.shape)
print(points[1].jdata.shape)
```
![Minibatch of grids constructed from the input points. These input points have corresponding color attributes.](../imgs/fig/screenshot_000000.png.trim.png)
Next, we splat the colors at the points to the constructed grid, yielding per-voxel colors.
```python continuation
# Splat the colors into the grid with trilinear interpolation
# vox_colors is a JaggedTensor of per-voxel normas
vox_colors = grid.splat_trilinear(points, colors)
```
![Colors splat at the input points to grid, yielding per-voxel colors.](../imgs/fig/screenshot_000006.png.trim.png)
Finally, we generate a new set of noisy points and sample the grid to recover colors at those new samples.
```python continuation
# Now let's generate some random points and sample the grid at those points
sample_points = fvdb.JaggedTensor([torch.rand(10_000, 3), torch.rand(11_000, 3)]).cuda()
# sampled_colors is a JaggedTensor with the same shape as sample_points with
# one color sampled from the grid at each point
sampled_colors = grid.sample_trilinear(sample_points, vox_colors)
```
![Colors resampled at random locations from the grid.](../imgs/fig/screenshot_000004.png.trim.png)
+868
View File
@@ -0,0 +1,868 @@
# Basic GridBatch Operations
Here we describe basic operations you can perform on a `GridBatch`. Generally, these operations involve a `GridBatch` and an associated `JaggedTensor` of data representing the attributes/features at each voxel within the grid.
## Sampling grids
You can differentiably sample data stored at the voxels of a grid using *trilinear* or *Bézier* interpolation as follows:
```python
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
import torch
import numpy as np
# We're going to create a minibatch of two meshes
v1, f1 = load_car_1_mesh(mode="vf")
v2, f2 = load_car_2_mesh(mode="vf")
# Build a GridBatch from two meshes
mesh_v_jagged = fvdb.JaggedTensor([v1, v2])
mesh_f_jagged = fvdb.JaggedTensor([f1, f2]).int()
grid = fvdb.gridbatch_from_mesh(mesh_v_jagged, mesh_f_jagged, voxel_sizes=0.1)
# Generate some sample points by adding random gaussian noise to the center of each voxel
world_space_centers = grid.grid_to_world(grid.ijk.float())
# 5 samples per voxel for the first grid and 7 samples per voxel for the second
sample_pts = fvdb.JaggedTensor([
torch.cat([world_space_centers[0].jdata] * 5),
torch.cat([world_space_centers[1].jdata] * 7)])
sample_pts += fvdb.JaggedTensor([
torch.randn(grid.num_voxels_at(0) * 5, 3).to(grid.device) * grid.voxel_sizes[0]*0.3,
torch.randn(grid.num_voxels_at(1) * 7, 3).to(grid.device) * grid.voxel_sizes[1]*0.3])
# Generate RGB values per voxel as the normalized absolute grid coordinate of the voxel
per_voxel_colors = world_space_centers.clone()
per_voxel_colors.jdata = torch.abs(world_space_centers.jdata) / torch.norm(world_space_centers.jdata, dim=-1, keepdim=True)
# Sample these RGB colors at each sample point with trilinear interpolation
# NOTE: You can use grid.sample_bezier to sample using bezier interpolation
sampled_colors = grid.sample_trilinear(sample_pts, per_voxel_colors)
```
Grid with color attributes | Points with sampled colors
:------------------------------------:|:------------------------------------:
![](../imgs/fig/sampling_1.png) | ![](../imgs/fig/sampling_2.png)
## Splatting point data to a grid
You can differentiably splat data at a set of points into voxels in a grid using *trilinear* or *Bézier* interpolation as follows:
```python
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
import torch
import point_cloud_utils as pcu
# We're going to create a minibatch of two point clouds each of which
# has a different number of points
pts1, clrs1 = load_car_1_mesh(mode="vn")
pts2, clrs2 = load_car_2_mesh(mode="vn")
# JaggedTensors of points and normals
points = fvdb.JaggedTensor([pts1, pts2])
colors = fvdb.JaggedTensor([clrs1, clrs2])
# Create a grid where the voxels each have unit sidelength
grid = fvdb.gridbatch_from_points(points, voxel_sizes=1.0)
# Splat the normals into the grid with trilinear interpolation
# vox_normals is a JaggedTensor of per-voxel normas
# NOTE: You can use grid.splat_bezier to splat using bezier interpolation
vox_colors = grid.splat_trilinear(points, colors)
```
Input grid and points with colors | Splat colors onto the grid
:----------------------------------------------------:|:----------------------------------------------------:
![](../imgs/fig/screenshot_000000.png.trim.png) | ![](../imgs/fig/screenshot_000006.png.trim.png)
## Checking if points are in a grid
You can query whether points lie in a grid as follows:
```python
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
import torch
import numpy as np
import point_cloud_utils as pcu
# We're going to create a minibatch of two meshes
v1, f1 = load_car_1_mesh(mode="vf")
v2, f2 = load_car_2_mesh(mode="vf")
f1, f2 = f1.to(torch.int32), f2.to(torch.int32)
# Build a GridBatch from two meshes
mesh_v_jagged = fvdb.JaggedTensor([v1, v2]).cuda()
mesh_f_jagged = fvdb.JaggedTensor([f1, f2]).cuda()
grid = fvdb.gridbatch_from_mesh(mesh_v_jagged, mesh_f_jagged, voxel_sizes=0.1)
# Generate some points and check if they lie within the grid
bbox_sizes = grid.bbox[:, 1] - grid.bbox[:, 0]
bbox_origins = grid.bbox[:, 0]
pts = fvdb.JaggedTensor([
(torch.randn(10_000, 3, device='cuda') - bbox_origins[0]) * bbox_sizes[0],
(torch.randn(11_000, 3, device='cuda') - bbox_origins[0]) * bbox_sizes[0],
])
# Get a mask indicating which points lie in the grid
mask = grid.points_in_active_voxel(pts)
```
We visualize the points which intersect the grid (yellow points intersect and purple points do not).
![](../imgs/fig/pts_in_grid.png)
## Checking if ijk coordinates are in a grid
Similar to querying whether world space points lie in a grid, you can query whether the index space `ijk` integer coordinates lie in a grid as follows:
```python
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
import torch
import numpy as np
import point_cloud_utils as pcu
import polyscope as ps
import os
# We're going to create a minibatch of two meshes
v1, f1 = load_car_1_mesh(mode="vf")
v2, f2 = load_car_2_mesh(mode="vf")
f1, f2 = f1.to(torch.int32), f2.to(torch.int32)
# Build a GridBatch from two meshes
mesh_v_jagged = fvdb.JaggedTensor([v1, v2]).cuda()
mesh_f_jagged = fvdb.JaggedTensor([f1, f2]).cuda()
grid = fvdb.gridbatch_from_mesh(mesh_v_jagged, mesh_f_jagged, voxel_sizes=0.025)
rand_idx_pts = []
for b, b_grid in enumerate(grid.bbox):
pts = [torch.randint(int(b_grid[0][i]), int(b_grid[1][i]), size=(2_000 * (b+1),), device='cuda') for i in range(3)]
rand_idx_pts.append(torch.stack(pts, dim=1))
pts = fvdb.JaggedTensor(rand_idx_pts)
coords_in_grid = grid.coords_in_active_voxel(pts)
```
We visualize the coordinates which intersect the grid (yellow coordinates intersect and purple coordinates do not).
![](../imgs/fig/coords_in_grid.png)
## Checking if axis aligned cubes intersect a grid
There are methods of a grid to help ascertain whether a provided set of axis-aligned cubes (all of the same size) each are contained within voxels of the grid or intersect with any voxel of the grid. These methods are called `cubes_in_grid` and `cubes_intersect_grid`, respectively, and return a `JaggedTensor` of boolean values which indicate the result.
In this example we create some random points to represent the centers of cubes of size `0.03` units along-a-side. Then we use these two methods to discover which of those cubes would intersect a voxel of the grid and which would be entirely enclosed by a voxel in the grid.
```python
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
import torch
import numpy as np
import point_cloud_utils as pcu
import polyscope as ps
import os
# We're going to create a minibatch of two meshes
v1, f1 = load_car_1_mesh(mode="vf")
v2, f2 = load_car_2_mesh(mode="vf")
f1, f2 = f1.to(torch.int32), f2.to(torch.int32)
# Build a GridBatch from two meshes
mesh_v_jagged = fvdb.JaggedTensor([v1, v2]).cuda()
mesh_f_jagged = fvdb.JaggedTensor([f1, f2]).cuda()
grid = fvdb.gridbatch_from_mesh(mesh_v_jagged, mesh_f_jagged, voxel_sizes=0.025)
# Generate some points and check if they lie within the grid
bbox_sizes = (grid.bbox[:, 1] - grid.bbox[:, 0]) * grid.voxel_sizes
bbox_origins = (grid.bbox[:, 0] + grid.origins) * grid.voxel_sizes
pts = fvdb.JaggedTensor(
[
(torch.rand(3000, 3, device="cuda")) * bbox_sizes[0] + bbox_origins[0],
(torch.rand(2000, 3, device="cuda")) * bbox_sizes[1] + bbox_origins[1],
]
)
cube_size = 0.03
# We can check if the axis-aligned cubes intersect any voxels of the grid...
cubes_intersect_grid = grid.cubes_intersect_grid(pts, -cube_size / 2, cube_size / 2)
# ... or if they are fully contained within the grid
cubes_in_grid = grid.cubes_in_grid(pts, -cube_size / 2, cube_size / 2)
```
We visualize the cubes which intersect a voxel in the grid (yellow cubes intersect and purple cubes do not).
![](../imgs/fig/cubes_intersect_grid.png)
And which cubes lie entirely within a voxel of the grid
![](../imgs/fig/cubes_inside_grid.png)
## Converting ijk values to indexes
There is a convenient method to convert ijk values, which are integer coordinates in the index space of a grid, to indexes, which are the linearized indices of the voxels in the GridBatch. This method, `ijk_to_index`, returns a `JaggedTensor` of indexes. If the provided ijk values do not correspond to any voxels in the grid, the returned index will be `-1`.
```python
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
import torch
import numpy as np
import point_cloud_utils as pcu
import polyscope as ps
import os
torch.random.manual_seed(0)
def generate_random_points(bounding_box, num_points):
min_i, min_j, min_k = bounding_box[0]
max_i, max_j, max_k = bounding_box[1]
# Generate random integer points within the bounding box
random_i = torch.randint(min_i, max_i, size=(num_points,))
random_j = torch.randint(min_j, max_j, size=(num_points,))
random_k = torch.randint(min_k, max_k, size=(num_points,))
random_points = torch.stack([random_i, random_j, random_k], dim=1)
return random_points
# We're going to create a minibatch of two meshes
v1, f1 = load_car_1_mesh(mode="vf")
v2, f2 = load_car_1_mesh(mode="vf")
f1, f2 = f1.to(torch.int32), f2.to(torch.int32)
# Build a GridBatch from two meshes
mesh_v_jagged = fvdb.JaggedTensor([v1, v2]).cuda()
mesh_f_jagged = fvdb.JaggedTensor([f1, f2]).cuda()
grid = fvdb.gridbatch_from_mesh(mesh_v_jagged, mesh_f_jagged, voxel_sizes=0.025)
rand_pts = fvdb.JaggedTensor([generate_random_points(bbox, 1000) for bbox in grid.bbox]).cuda()
rand_pts_indices= grid.ijk_to_index(rand_pts)
print(rand_pts_indices.jdata)
```
```bash
tensor([ -1, -1, 121, ..., 4614, 5695, -1], device='cuda:0')
```
## Converting indexes to ijk values
If we have the value of an index into the feature data and want to obtain its corresponding ijk value, this is as simple as indexing into the `ijk` attribute of the grid which is itself just a `JaggedTensor` representing the `ijk` values of each index. Here we get the `ijk` values for 1000 random indexes of a `GridBatch`.
```python
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
import torch
import numpy as np
import point_cloud_utils as pcu
import polyscope as ps
import os
# We're going to create a minibatch of two meshes
v1, f1 = load_car_1_mesh(mode="vf")
v2, f2 = load_car_2_mesh(mode="vf")
f1, f2 = f1.to(torch.int32), f2.to(torch.int32)
# Build a GridBatch from two meshes
mesh_v_jagged = fvdb.JaggedTensor([v1, v2]).cuda()
mesh_f_jagged = fvdb.JaggedTensor([f1, f2]).cuda()
grid = fvdb.gridbatch_from_mesh(mesh_v_jagged, mesh_f_jagged, voxel_sizes=0.025)
rand_indexes = torch.randint(0, grid.total_voxels, size=(1000,)).cuda()
print(grid.ijk.jdata[rand_indexes])
print(grid.ijk.jidx[rand_indexes])
```
```bash
tensor([[-11, 5, 7],
[ 8, -4, -5],
[-15, 3, -2],
...,
[ 18, 1, 6],
[ 19, -3, 6],
[-14, -7, 8]], device='cuda:0', dtype=torch.int32)
```
However, you may also want to obtain the batch index of the grid that the `ijk` coordinate belongs to. This is just as simple by using the `jidx` attribute of the `JaggedTensor` which represents the batch index of each element in the `JaggedTensor`. Using the same random indexes as above, we can get the batch index of the grid for each random index.
```python continuation
print(grid.ijk.jidx[rand_indexes])
```
```bash
tensor([0, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1,
1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1,
0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1,
1, 0, 0, 0, 0, 1, 0, 1, 1, 0,…], device='cuda:0')
```
While this case of having a random set of `ijk` values is very contrived, it is more likely that we would have a `JaggedTensor` of features that correspond to each `ijk` value but are out-of-order in relation to another `grid.ijk`'s and we need to reorder these features to for this other grid. For this, there is the `ijk_to_inv_index` function.
Given a `JaggedTensor` of `ijk` values, `grid_batch.ijk_to_inv_index` will return a scalar integer `JaggedTensor` of size `[B, -1]`, where `B` is the number of grids in the batch and -1 represents the number of voxels in each grid. This `JaggedTensor` of indexes can be used to permute the input to `ijk_to_inv_index` (or the feature `JaggedTensor` that correspond to those `ijk`'s) to match the ordering of the `grid_batch`.
To concisely illustrate the properties of this function,
* if `idx = grid.ijk_to_inv_index(misordered_ijk)`, then `grid.ijk == misordered_ijk[idx]`
* if `idx = grid.ijk_to_index(misordered_ijk)`, then `grid.ijk[idx] == misordered_ijk`
Note: If any `ijk` values in the grid are not present in the input to `ijk_to_inv_index`, the returned index will be `-1` at that position.
In this example, let's illustrate the more useful case where we have corresponding features and `ijk` values from a grid that are ordered differently from another reference grid and we want to re-order the features to match the order they should be in the reference grid.
```python continuation
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
import torch
import numpy as np
import point_cloud_utils as pcu
# We're going to create a minibatch of two meshes
v1, f1 = load_car_1_mesh(mode="vf")
v2, f2 = load_car_2_mesh(mode="vf")
f1, f2 = f1.to(torch.int32), f2.to(torch.int32)
# Build a GridBatch from two meshes
mesh_v_jagged = fvdb.JaggedTensor([v1, v2]).cuda()
mesh_f_jagged = fvdb.JaggedTensor([f1, f2]).cuda()
reference_grid = fvdb.gridbatch_from_mesh(mesh_v_jagged, mesh_f_jagged, voxel_sizes=0.025)
# 7 random feature values for each voxel in the grid
features = reference_grid.jagged_like(torch.rand(grid.total_voxels, 7).to(grid.device))
# Create a set of randomly shuffled corresponding ijk/features from our original grid/features
shuffled_ijks = []
shuffled_features = []
for i in range(reference_grid.grid_count):
perm = torch.randperm(reference_grid.num_voxels_at(i))
shuffled_ijks.append(reference_grid.ijk.jdata[grid.ijk.jidx==i][perm])
shuffled_features.append(features.jdata[features.jidx==i][perm])
shuffled_ijks = fvdb.JaggedTensor(shuffled_ijks)
shuffled_features = fvdb.JaggedTensor(shuffled_features)
# Get the indexes to reorder the shuffled features to match the grid's original ijk ordering
idx = reference_grid.ijk_to_inv_index(shuffled_ijks)
# Permute the shuffled features based on the ordering from `ijk_to_inv_index`
unshuffled_features = shuffled_features.jdata[idx.jdata]
print("Do the shuffled features that have been permuted based on `ijk_to_inv_index` match the original features? ", "Yes!" if torch.all(unshuffled_features == features.jdata) else "No!")
```
```bash
Do the shuffled features that have been permuted based on `ijk_to_inv_index` match the original features? Yes!
```
## Getting indexes of neighbors
If we want to get the indexes of all the spatial neighbors of a set of ijk values, we can use the `neighbor_indexes` method of a `GridBatch`. This method receives a set of `ijk` values and an `extent` (the number of voxels away from the ijk value to consider neighbors) and returns a `JaggedTensor` of indexes of neighbors for each ijk value. The returned `JaggedTensor` has `jdata` of size `[N, extent*2+1, extent*2+1, extent*2+1]` where `N` is the number of requested ijk values.
In this example we create a set of 24 random ijk values per grid and get the indexes of all neighbors 2 voxels away from each ijk.
```python
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
import torch
import numpy as np
import point_cloud_utils as pcu
import polyscope as ps
import os
# We're going to create a minibatch of two meshes
v1, f1 = load_car_1_mesh(mode="vf")
v2, f2 = load_car_2_mesh(mode="vf")
f1, f2 = f1.to(torch.int32), f2.to(torch.int32)
# Build a GridBatch from two meshes
mesh_v_jagged = fvdb.JaggedTensor([v1, v2]).cuda()
mesh_f_jagged = fvdb.JaggedTensor([f1, f2]).cuda()
grid = fvdb.gridbatch_from_mesh(mesh_v_jagged, mesh_f_jagged, voxel_sizes=0.025)
# Build a set of 24 randomly selected ijk values per grid
rand_ijks = fvdb.JaggedTensor(
[
grid.ijk.jdata[
torch.randint(int(grid.ijk.joffsets[b]), int(grid.ijk.joffsets[b + 1]), (24,))
]
for b in range(grid.grid_count - 1)
],
)
# Get the indexes of all neighbors 2 voxels away from each ijk
# Returns a JaggedTensor with jdata of size [48, 5, 5, 5] ([ N x (extent*2+1)^3 ])
# where each [5, 5, 5] describes the layout of neighbouring indexes of each ijk
neighbor_idxs = grid.neighbor_indexes(rand_ijks, 2)
```
We visualize the voxels we selected in red and their neighbors in blue.
![](../imgs/fig/neighbor_indexes.png)
## Clipping a grid to a bounding box
If we want to 'clip' a grid, meaning remove all the voxels that are outside of a bounding box, we can use the `clipped_grid` method of a `GridBatch`. The minimum and maximum `ijk` extents of the bounding box used for this clipping are provided as arguments and can be specified per-grid of the batch.
If we want to both clip the batch of grids and the accompanying `JaggedTensor` of batched features, we can use the `clip` method of a `GridBatch` which can be provided the `features` to be clipped along with the grid.
In this example we clip a batch of grids to independent bounding boxes and visualize the result.
```python
import os
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh, load_car_3_mesh, load_car_4_mesh
import torch
import numpy as np
import point_cloud_utils as pcu
device = 'cuda'
mesh_vs = []
mesh_fs = []
mesh_load_funcs = [load_car_1_mesh, load_car_2_mesh, load_car_3_mesh, load_car_4_mesh]
for func in mesh_load_funcs:
v, f = func(mode="vf")
mesh_vs.append(v)
mesh_fs.append(f.to(torch.int32))
mesh_vs = fvdb.JaggedTensor(mesh_vs)
mesh_fs = fvdb.JaggedTensor(mesh_fs)
vox_size = 0.01
# Build GridBatch from meshes
grid = fvdb.gridbatch_from_mesh(mesh_vs, mesh_fs, vox_size)
# Use `clipped_grid` to clip the grids outside of the specified minimum and maximum bounding boxes for each grid…
clipped_grid = grid.clipped_grid(
ijk_min=[[-200, -200, -200], [0, -200, -200], [-200, -200, -200], [-200, 0, -200]],
ijk_max=[[0, 200, 200], [200, 200, 200], [200, 0, 200], [200, 200, 200]])
# Use `clip` to both clip the grids and the accompanying JaggedTensor of features
features = grid.jagged_like(torch.rand(grid.total_voxels, 7).to(device))
clipped_features, clipped_grid = grid.clip(
features=features,
ijk_min=[[-200, -200, -200], [0, -200, -200], [-200, -200, -200], [-200, 0, -200]],
ijk_max=[[0, 200, 200], [200, 200, 200], [200, 0, 200], [200, 200, 200]])
```
We visualize these grids, each clipped to a different bounding box.
![](../imgs/fig/clip.png)
## Max/Mean Pooling
'Pooling' a grid has the effect of coarsening the resolution of the grid by a constant factor (and can be performed anisotropically for different factors in each `xyz` spatial dimension). When pooling a grid, the values of the new, coarser grid can be determined by the maximum or mean of the values of the voxels in the original grid that are covered by each voxel in the new grid. Maximum and mean pooling can be accomplished by the `max_pool` and `avg_pool` operators.
In this example, we create a grid from a mesh and then perform max and mean pooling on it to illustrate the difference between the two.
```python
import os
import fvdb
from fvdb.utils.examples import load_car_1_mesh
import torch
import numpy as np
import point_cloud_utils as pcu
vox_size = 0.02
num_pts = 10_000
mesh_load_funcs = [load_car_1_mesh]
points = []
normals = []
for func in mesh_load_funcs:
pts, nms = func(mode="vn")
pmt = torch.randperm(pts.shape[0])[:num_pts]
pts, nms = pts[pmt], nms[pmt]
points.append(pts)
normals.append(nms)
# JaggedTensors of points and normals
points = fvdb.JaggedTensor(points)
normals = fvdb.JaggedTensor(normals)
# Create a grid
grid = fvdb.gridbatch_from_points(points, voxel_sizes=vox_size)
# Splat the normals into the grid with trilinear interpolation
vox_normals = grid.splat_trilinear(points, normals)
# Mean Pooling of normals features
avg_normals, avg_grid = grid.avg_pool(4, vox_normals)
# Max Pooling of normals features
max_normals, max_grid = grid.max_pool(4, vox_normals)
```
We visualize the original grid with values of the mesh normals as features, the features/grid after mean pooling, and the features/grid after max pooling to illustrate how these pooling modes affect the resulting features of the grid.
![](../imgs/fig/max_avg_pool.png)
## Subdividing a grid
Subdividing a grid has the effect of increasing the resolution of the grid by a constant factor (and can be performed anisotropically for different factors in each `xyz` spatial dimension).
The most straightforward way to use the `subdivide` method is to provide an integer value for the subdivision factor. This will result in a grid that is `subdiv_factor` times the resolution of the original grid in each spatial dimension.
```python
import os
import fvdb
from fvdb.utils.examples import load_car_1_mesh
import torch
import numpy as np
import point_cloud_utils as pcu
vox_size = 0.02
num_pts = 10_000
mesh_load_funcs = [load_car_1_mesh]
points = []
normals = []
for func in mesh_load_funcs:
pts, nms = func(mode="vn")
pmt = torch.randperm(pts.shape[0])[:num_pts]
pts, nms = pts[pmt], nms[pmt]
points.append(pts)
normals.append(nms)
# JaggedTensors of points and normals
points = fvdb.JaggedTensor(points)
normals = fvdb.JaggedTensor(normals)
# Create a grid
grid = fvdb.gridbatch_from_points(points, voxel_sizes=vox_size)
# Splat the normals into the grid with trilinear interpolation
vox_normals = grid.splat_trilinear(points, normals)
# Subdivide by a constant factor of 2
subdiv_normals, subdiv_grid = grid.subdivide(2, vox_normals)
```
Here we visualize the original grid on the right and the grid after subdivision by a factor of 2 on the left.
![](../imgs/fig/subdivide.png)
In practice, in a deep neural network like a U-Net architecture, the resolution of the grid can be decreased early in the network and then the grid's features need to be concatenated with features of the grid after its resolution is increased again. When working with traditional, dense 2D or 3D data, cropping any mismatched outputs is straightforward to be able to concatenate these features. However, in a sparse 3D grid, this is not straightforward and it's entirely unclear how to align the features.
Take this simple example to illustrate the difficulties created by changing the grid topology in this way. We take the grid created from our mesh, perform Pooling by a factor of 2 and then try to Subdivide by an equal factor of 2 to invert the Pooling operation.
```python continuation
max_normals, max_grid = grid.max_pool(2, vox_normals)
subdiv_normals, subdiv_grid = max_grid.subdivide(2, max_normals)
```
Let's visualize these results from left to right of the original grid, the grid after max pooling, and the grid after max pooling and then subdividing again by the same factor.
![](../imgs/fig/subdiv_pool_incorrect.png)
Notice how we have not obtained the topology of the original grid and have obtained a grid with many more voxels than the original.
To correctly invert the Pooling operation, we can provide the `subdivide` function with a `fine_grid` optional argument which describes the topology we want the grid to have after the subdivision. The original grid before the Pooling operation can be used as this `fine_grid`.
```python continuation
max_normals, max_grid = grid.max_pool(2, vox_normals)
# Providing the original grid as our fine_grid target
subdiv_normals, subdiv_grid = max_grid.subdivide(2, max_normals, fine_grid=grid)
```
![](../imgs/fig/subdiv_pool_correct.png)
Note the matching topology of the grid after the Subdivision operation to the original grid (though the features are different due to the Pooling operation).
One other useful feature of the `subdivide` operator is that this operation can be *masked* so that subdivision is only performed on a subset of the voxels in the grid.
The optional `mask` argument to `subdivide` is a `JaggedTensor` of boolean values that indicates which voxels should be subdivided. Given this `mask` is simply a `JaggedTensor`, this operation can be made differentiable in a neural network and the `subdivide` operator can be learned.
Let's illustrate a very simple example of how to use the `mask` argument to only subdivide the voxels which have a feature value greater than a certain threshold.
```python continuation
# Mask the grid with the normals where only the normals with a value greater than 0.5 on the x-axis are subdivided
mask = vox_normals.jdata[:, 0] > 0.5
subdiv_normals, subdiv_grid = grid.subdivide(2, vox_normals, mask=mask)
```
![](../imgs/fig/subdiv_mask.png)
## Getting the number of voxels per grid in a batch
Getting the number of voxels in the grids of a `GridBatch` can be easily accomplished with `num_voxels`:
```python
import fvdb
import torch
# Create a GridBatch of random points
batch_size = 4
pts = fvdb.JaggedTensor([torch.rand(10_000*(i+1),3) for i in range(batch_size)])
grid = fvdb.GridBatch()
grid.set_from_points(pts, voxel_sizes=0.02)
# Get the number of voxels per grid in the batch
for batch, num_voxels in enumerate(grid.num_voxels):
print(f"Grid {batch} has {num_voxels} voxels")
```
```bash
Grid 0 has 9645 voxels
Grid 1 has 18503 voxels
Grid 2 has 26749 voxels
Grid 3 has 34378 voxels
```
## Converting between ijk (grid) coordinates and world coordinates
A `GridBatch` can contain multiple grids, each with its own coordinate system that relate the grids' `ijk` integer index space coordinates to `xyz` floating-point world-space coordinates. These axis-aligned coordinate systems are defined by specifying a list of three-dimensional world-space origin and scale values when we define the topology of the grids in the `GridBatch` like so:
```python
import fvdb
import torch
# Create a GridBatch of random points
batch_size = 4
pts = fvdb.JaggedTensor([torch.rand(10_000*(i+1),3) for i in range(batch_size)])
grid = fvdb.GridBatch()
grid.set_from_points(pts,
voxel_sizes=[[0.02, 0.02, 0.02], [0.03, 0.03, 0.03], [0.04, 0.04, 0.04], [0.05, 0.05, 0.05]],
origins=[[-.1,-.1,-.1], [0,0,0], [.1,.1,.1], [.2,-.2,.2]])
```
When `voxel_sizes` and `origins` are not defined, it is assumed all grids have a unit scale voxel_size and an origin at `[0.0, 0.0, 0.0]`.
We can use `GridBatch`'s `grid_to_world` function to convert between `ijk` index coordinates and their corresponding world-space `xyz` coordinates. In this example, let's obtain the world-space position that would lie at the index-space `[1, 1, 1]` point of each grid:
```python continuation
# Convert ijk coordinates to world coordinates
ijk = fvdb.JaggedTensor([torch.ones(1,3, dtype=torch.float) for _ in range(batch_size)])
world_coords = grid.grid_to_world(ijk)
for i in range(grid.grid_count):
print(f"World-space point that lies at index [1,1,1] for Grid {i} is positioned at {world_coords.jdata[world_coords.jidx==i].tolist()}")
```
```bash
World-space point that lies at index [1,1,1] for Grid 0 is positioned at [[-0.07999999821186066, -0.07999999821186066, -0.07999999821186066]]
World-space point that lies at index [1,1,1] for Grid 1 is positioned at [[0.030000001192092896, 0.030000001192092896, 0.030000001192092896]]
World-space point that lies at index [1,1,1] for Grid 2 is positioned at [[0.14000000059604645, 0.14000000059604645, 0.14000000059604645]]
World-space point that lies at index [1,1,1] for Grid 3 is positioned at [[0.25, -0.15000000596046448, 0.25]]
```
We can also do the inverse operation and convert world-space `xyz` coordinates to their corresponding `ijk` index-space coordinates using `world_to_grid`. In this example, let's find the `ijk` index coordinates of the voxel which would contain the world-space point located at `[1.0, 1.0, 1.0]` for each grid in our `GridBatch`:
```python continuation
# Convert world coordinates to ijk coordinates
xyz = fvdb.JaggedTensor([torch.ones(1,3, dtype=torch.float) for _ in range(batch_size)])
ijk_coords = grid.world_to_grid(xyz)
for i in range(grid.grid_count):
print(f"Index-space voxel that contains the point [1.0, 1.0, 1.0] for Grid {i} {ijk_coords.jdata[ijk_coords.jidx==i].int().tolist()}")
```
```bash
Index-space voxel that contains the point [1.0, 1.0, 1.0] for Grid 0 [[55, 55, 55]]
Index-space voxel that contains the point [1.0, 1.0, 1.0] for Grid 1 [[33, 33, 33]]
Index-space voxel that contains the point [1.0, 1.0, 1.0] for Grid 2 [[22, 22, 22]]
Index-space voxel that contains the point [1.0, 1.0, 1.0] for Grid 3 [[16, 24, 16]]
```
While `grid_to_world` and `world_to_grid` are convenient functions for these purposes, the row-major transformation matrices used for these calculations can be obtained directly from a `GridBatch` for use in your own logic:
```python continuation
print(f"Grid to world matrices:\n{grid.grid_to_world_matrices}")
print(f"World to grid matrices:\n{grid.grid_to_world_matrices}")
```
```bash
Grid to world matrices:
tensor([[[ 0.0200, 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0200, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0200, 0.0000],
[-0.1000, -0.1000, -0.1000, 1.0000]],
[[ 0.0300, 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0300, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0300, 0.0000],
[ 0.0000, 0.0000, 0.0000, 1.0000]],
[[ 0.0400, 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0400, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0400, 0.0000],
[ 0.1000, 0.1000, 0.1000, 1.0000]],
[[ 0.0500, 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0500, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0500, 0.0000],
[ 0.2000, -0.2000, 0.2000, 1.0000]]])
World to grid matrices:
tensor([[[ 0.0200, 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0200, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0200, 0.0000],
[-0.1000, -0.1000, -0.1000, 1.0000]],
[[ 0.0300, 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0300, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0300, 0.0000],
[ 0.0000, 0.0000, 0.0000, 1.0000]],
[[ 0.0400, 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0400, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0400, 0.0000],
[ 0.1000, 0.1000, 0.1000, 1.0000]],
[[ 0.0500, 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0500, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0500, 0.0000],
[ 0.2000, -0.2000, 0.2000, 1.0000]]])
```
## Convolution
Convolving the features of a `GridBatch` can be accomplished with either a high-level `torch.nn.Module` derived class provided by `fvdb.nn` or with more low-level methods available with `GridBatch`, we will illustrate both techniques.
### High-level Usage with `fvdb.nn`
`fvdb.nn.SparseConv3d` provides a high-level `torch.nn.Module` class for convolution on `fvdb` classes that is an analogue to the use of `torch.nn.Conv3d`. Using this module is the recommended functionality for performing convolution with `fvdb` because it not only manages functionality such as initializing the weights of the convolution and calling appropriate backend implementation functions but it also provides certain backend optimizations which will be illustrated in the [Low-level usage](#low-level-usage-with-gridbatch) section.
One thing to note is `fvdb.nn.SparseConv3d` operates on a class that wraps a `GridBatch` and `JaggedTensor` together into a convenience object, `fvdb.VDBTensor`, which is used by all the `fvdb.nn` modules.
A simple example of using `fvdb.nn.SparseConv3d` is as follows:
```python
import fvdb
import fvdb.nn as fvdbnn
from fvdb.utils.examples import load_car_1_mesh
import torch
import numpy as np
import point_cloud_utils as pcu
num_pts = 10_000
vox_size = 0.02
mesh_load_funcs = [load_car_1_mesh]
points = []
normals = []
for func in mesh_load_funcs:
pts, nms = func(mode="vn")
pmt = torch.randperm(pts.shape[0])[:num_pts]
pts, nms = pts[pmt], nms[pmt]
points.append(pts)
normals.append(nms)
# JaggedTensors of points and normals
points = fvdb.JaggedTensor(points)
normals = fvdb.JaggedTensor(normals)
# Create a grid
grid = fvdb.gridbatch_from_points(points, voxel_sizes=vox_size)
# Splat the normals into the grid with trilinear interpolation
vox_normals = grid.splat_trilinear(points, normals)
# VDBTensor is a simple wrapper of a grid and a feature tensor
vdbtensor = fvdbnn.VDBTensor(grid, vox_normals)
# fvdb.nn.SparseConv3d is a convenient torch.nn.Module implementing the fVDB convolution
conv = fvdbnn.SparseConv3d(in_channels=3, out_channels=3, kernel_size=3, stride=1, bias=False).to(vdbtensor.device)
output = conv(vdbtensor)
```
Let's visualize the original grid with normals visualized as colours alongside the result of these features after a convolution initialized with random weights:
![](../imgs/fig/simple_conv.png)
For stride values greater than 1, the output of the convolution will be a grid with a smaller resolution than the input grid (similar in topological effect to the output of a Pooling operator). Let's illustrate this:
```python continuation
# We would expect for stride=2 that the output grid would have half the resolution (or twice the world-space size) of the input grid
conv = fvdbnn.SparseConv3d(in_channels=3, out_channels=3, kernel_size=3, stride=2, bias=False).to(vdbtensor.device)
output = conv(vdbtensor)
```
![](../imgs/fig/stride_conv.png)
Transposed convolution can be performed with `fvdb.nn.SparseConv3d` which can increase the resolution of the grid. It only really makes sense to perform transposed sparse convolution with a target grid topology we wish to produce with this operation (see the [Pooling Operators](#maxmean-pooling) for an explanation). Therefore, an `out_grid` argument must be provided in this case to specify the target grid topology:
```python continuation
# Tranposed convolution operator, stride=2
transposed_conv = fvdbnn.SparseConv3d(in_channels=3, out_channels=3, kernel_size=3, stride=2, bias=False, transposed=True).to(vdbtensor.device)
# Note the use of the `out_grid` argument to specify the target grid topology
transposed_output = transposed_conv(output, out_grid=vdbtensor.grid)
```
Here we visuzlie the original grid, the grid after strided convolution and the grid after transposed convolution inverts the topological operation of the strided convolution to produce the same topology as the original grid with the features convolved by our two layers:
![](../imgs/fig/transposed_stride_conv.png)
### Low-level Usage with `GridBatch`
The [high-level `fvdb.nn.SparseConv3d` class](#high-level-convolution-with-fvdbnn) wraps several pieces of `GridBatch` functionality to provide a convenient `torch.nn.Module` for convolution. However, for a more low-level approach that accomplishes the same outcome, the `GridBatch` class itself can be the starting point for performing convolution on the grid and its features. We will illustrate this approach for completeness, though we do recommend the use of the `fvdb.nn.SparseConv3d` Module for most use-cases.
Using the `GridBatch` convolution functions directly requires a little more knowledge about the implementation under-the-hood. Due to the nature of a sparse grid, in order to make convolution performant, it is useful to pre-compute a mapping of which features in the input grid will contribute to the values of the output grid when convolved by a kernel of a particular dimension and stride. This mapping structure is called a 'kernel map'.
The kernel map, as well as the functionality for using it to compute the convolution, is contained within a `fvdb.SparseConvPackInfo` class which can be constructed by `GridBatch.sparse_conv_kernel_map`. A `fvdb.SparseConvPackInfo` must perform a pre-computation of the kernel map based on the style expected by the backend implementation of the convolution utilized by `fvdb.SparseConvPackInfo.sparse_conv_3d`. Here is an example of how to construct a `fvdb.SparseConvPackInfo` and use it to perform a convolution:
```python
import fvdb
import fvdb.nn as fvdbnn
from fvdb.utils.examples import load_car_1_mesh
import torch
import numpy as np
import point_cloud_utils as pcu
num_pts = 10_000
vox_size = 0.02
mesh_load_funcs = [load_car_1_mesh]
points = []
normals = []
for func in mesh_load_funcs:
pts, nms = func(mode="vn")
pmt = torch.randperm(pts.shape[0])[:num_pts]
pts, nms = pts[pmt], nms[pmt]
points.append(pts)
normals.append(nms)
# JaggedTensors of points and normals
points = fvdb.JaggedTensor(points)
normals = fvdb.JaggedTensor(normals)
# Create a grid
grid = fvdb.gridbatch_from_points(points, voxel_sizes=vox_size)
# Splat the normals into the grid with trilinear interpolation
vox_normals = grid.splat_trilinear(points, normals)\
# Create the kernel map (housed in a SparseConvPackInfo) and the output grid's topology based on the kernel parameters
sparse_conv_packinfo, out_grid = grid.sparse_conv_kernel_map(kernel_size=3, stride=1)
# The kernel map must be pre-computed based on the backend implementation we plan on using, here we use gather/scatter, the default implementation
sparse_conv_packinfo.build_gather_scatter()
# Create random weights for our convolution kernel of size 3x3x3 that takes 3 input channels and produces 3 output channels
kernel_weights = torch.randn(3, 3, 3, 3, 3, device=grid.device)
# Perform convolution on the normals colours. Gather/scatter is used as the backend, it is the default
conv_vox_normals = sparse_conv_packinfo.sparse_conv_3d(vox_normals, weights=kernel_weights, backend=fvdb.ConvPackBackend.GATHER_SCATTER)
```
Here we visualize the output of our convolution alongside the original grid with normals visualized as colours:
![](../imgs/fig/gridbatch_conv.png)
The kernel map can potentially be expensive to compute, so it is often useful to re-use the `SparseConvPackInfo` in the same network to perform a convolution on other features or with different weights. This optimization is something `fvdb.nn.SparseConv3d` attempts to do where appropriate and is one reason we recommend using `fvdb.nn.SparseConv3d` over this low-level approach.
+186
View File
@@ -0,0 +1,186 @@
# Building Sparse Grids
We introduce several ways to construct sparse voxel grids from various data sources including point clouds, coordinate lists, triangle meshes, and deriving from other grids.
All the examples below could be found in full version (including visualization) at `examples/grid_building.py` and `examples/grid_subdivide_coarsen.py`.
## From Coordinate Lists
If you already have an integer list of the `ijk` coordinates for each voxel in the grid, you could directly build grids from the list.
Additionally, you will have to specify the voxel sizes and voxel origins for the grid.
An example is as follows:
```python
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
coords_1, _ = load_car_1_mesh()
coords_2, _ = load_car_2_mesh()
coords_jagged = fvdb.JaggedTensor([
coords_1.long().cuda(),
coords_2.long().cuda()
])
voxel_sizes = [[0.1, 0.1, 0.1], [0.15, 0.15, 0.15]]
grid = fvdb.gridbatch_from_ijk(coords_jagged, voxel_sizes=voxel_sizes, origins=[0.0] * 3)
```
The above code assumes that you want to build a grid with two batch elements, one with voxel size `[0.1, 0.1, 0.1]`, and the other with voxel size `[0.15, 0.15, 0.15]` (although usually you just want all the elements in your batch to have the same size, in which case you could just pass in `voxel_sizes=[0.1, 0.1, 0.1]`).
The same logic applies for `origins` that specifies the world coordinates of voxel `(0, 0, 0)`, which we set to the origin here.
The grid will be constructed on the same device as `coords_jagged`, which is a `JaggedTensor`. The JaggedTensor is built from two numpy `int64` arrays with size `[*, 3]` called `coords_1` and `coords_2`.
![build_from_coordinates.png](../imgs/fig/build_from_coordinates.png)
## From Point Clouds
You could either choose to quantize the coordinates of your point cloud into `ijk` coordinates yourself (e.g. using `np.unique((xyz / voxel_size).floor(), axis=0)`), or let `fvdb` handle this logic. Specifically, you could do:
```python
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
coords_1, _ = load_car_1_mesh()
coords_2, _ = load_car_2_mesh()
# Assemble point clouds into JaggedTensor
pcd_jagged = fvdb.JaggedTensor([
coords_1.cuda(),
coords_2.cuda()
])
voxel_sizes = [[0.1, 0.1, 0.1], [0.15, 0.15, 0.15]]
# Method 1:
grid_a1 = fvdb.gridbatch_from_points(pcd_jagged, voxel_sizes=voxel_sizes, origins=[0.0] * 3)
# Method 2:
grid_a2 = fvdb.GridBatch(device=pcd_jagged.device)
grid_a2.set_from_points(pcd_jagged, voxel_sizes=voxel_sizes, origins=[0.0] * 3)
```
Above we show two methods of building grids from points. Similar functions exist for other grid building approaches. The built grids are shown as following:
![build_from_points.png](../imgs/fig/build_from_points.png)
In some applications, you may want to build a dilated version of the grid by ensuring that all $2\times 2 \times 2$ voxels around each point are included in the built grid. That said, you could build the grid by:
```python
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
coords_1, _ = load_car_1_mesh()
coords_2, _ = load_car_2_mesh()
# Assemble point clouds into JaggedTensor
pcd_jagged = fvdb.JaggedTensor([
coords_1.cuda(),
coords_2.cuda()
])
voxel_sizes = [[0.1, 0.1, 0.1], [0.15, 0.15, 0.15]]
# Build grid from containing nearest voxels to the points
grid_b = fvdb.gridbatch_from_nearest_voxels_to_points(pcd_jagged, voxel_sizes=voxel_sizes, origins=[0.0] * 3)
```
![build_from_points_nn.png](../imgs/fig/build_from_points_nn.png)
## From Meshes
We allow building grids enclosing a triangle mesh easily. The given triangle mesh does not have to be manifold nor watertight and it will be treated as a triangle soup internally.
An example to build grids from meshes is shown as follows:
```python
import fvdb
from fvdb.utils.examples import load_car_1_mesh, load_car_2_mesh
mesh_1_v, mesh_1_f = load_car_1_mesh(mode='vf')
mesh_2_v, mesh_2_f = load_car_2_mesh(mode='vf')
mesh_v_jagged = fvdb.JaggedTensor([
mesh_1_v.float().cuda(),
mesh_2_v.float().cuda()
])
mesh_f_jagged = fvdb.JaggedTensor([
mesh_1_f.long().cuda(),
mesh_2_f.long().cuda()
])
voxel_sizes = [[0.1, 0.1, 0.1], [0.15, 0.15, 0.15]]
grid = fvdb.gridbatch_from_mesh(mesh_v_jagged, mesh_f_jagged, voxel_sizes=voxel_sizes, origins=[0.0] * 3)
```
Here `mesh_1_v` and `mesh_1_f` are the vertex array and triangle array of the mesh to build grid from, with the shape of $(V, 3)$ and $(F, 3)$. The triangle array is an integer array that indexes into the vertex array (starting from 0 for each element in the batch). Same for another `mesh_2_v` and `mesh_2_f`.
![build_from_mesh.png](../imgs/fig/build_from_mesh.png)
## From Dense
We have APIs for you to build dense grids of shape $(D, H, W)$ containing the full $D\times H \times W$ voxels.
```python
import fvdb
grid = fvdb.gridbatch_from_dense(num_grids=1, dense_dims=[32, 32, 32], device="cuda")
```
![build_from_dense.png](../imgs/fig/build_from_dense.png)
If you are comparing the performance of dense pytorch 3D tensors vs sparse grids, it is usually very helpful to build the exact same input (including grid and features). In `fvdb.nn`, we provide a thin wrapper class `VDBTensor` that works like a `torch.Tensor`, yet enclosing the grid topology. To convert data back and forth from dense PyTorch `Tensor`s, we could do:
```python
import torch
import fvdb
from fvdb.nn import VDBTensor
# Easy way to initialize a VDBTensor from a torch 3D tensor [B, D, H, W, C]
dense_data = torch.ones(2, 32, 32, 32, 16).cuda()
sparse_data = fvdb.nn.vdbtensor_from_dense(dense_data, voxel_sizes=[0.1] * 3)
dense_data_back = sparse_data.to_dense()
assert torch.all(dense_data == dense_data_back)
```
Here `sparse_data` will be a `fvdb.nn.VDBTensor` class, containing both `feature` and `grid` attribute.
Such a class could be fed into all the neural network components available in `fvdb.nn`.
## Deriving from other grids
### Dual grid
A dual grid (of a primal grid) is also a grid, with its voxel *centers* covering the *corners* of the primal grid, and *corners* taking the positions of the *centers* of the primal grid.
A dual grid shares the same voxel sizes as its primal grid, and is just a simple shift of half the voxel size in translation.
In the picture below, green grid is the primal grid while purple grid is its dual.
![build_dual.png](../imgs/fig/build_dual.png)
To create a dual grid from a given primal grid, use `GridBatch.dual_grid()`:
```python
import fvdb
from fvdb.utils.examples import load_dragon_mesh
coords_1, _ = load_dragon_mesh()
grid_primal = fvdb.gridbatch_from_points(fvdb.JaggedTensor([coords_1]))
grid_dual = grid_primal.dual_grid()
```
### Subdividing and Coarsening a grid
The grid could be subdivided or coarsened with a subdivision/coarsening factor provided:
```python
import fvdb
from fvdb.utils.examples import load_happy_mesh
coords_1, _ = load_happy_mesh(mode='vf')
grid = fvdb.gridbatch_from_points(fvdb.JaggedTensor([coords_1]))
grid_subdivided = grid.subdivided_grid(2)
grid_coarsened = grid.coarsened_grid(2)
```
![build_coarse_subdivide.png](../imgs/fig/build_coarse_subdivide.png)
The features associated with the grids could be processed via the examples in `Grid Operations`.
Refer to the corresponding section for more details.
+150
View File
@@ -0,0 +1,150 @@
# Sparse Grid I/O
We give an overview of ways to save and load sparse grids including how fVDB's serialized format relates to other libraries such as OpenVDB and NanoVDB. All of the examples in this tutorial are available in the `examples/io.py` file of the fVDB repository.
In these examples we will be using tools which are part of the `NanoVDB` project such as `nanovdb_convert`. It is assumed that the `NanoVDB` tools are available to call (i.e. findable on the system's `$PATH`). If you have not already installed these tools, you can find instructions on building NanoVDB with OpenVDB from its documentation:
https://www.openvdb.org/documentation/doxygen/NanoVDB_HowToBuild.html
While not necessary for fVDB's functionality, they are useful utilities to have available for inspecting and manipulating sparse grids.
## Python Serialization
Batches of sparse grids can be serialized to a NanoVDB file using the `fvdb.save` method. Here, we create two grids of different sizes and numbers of points and save them to a compressed NanoVDB file with specified names. The names are optional.
```python
import torch
import fvdb
import tempfile
import os
import subprocess
p = fvdb.JaggedTensor(
[
torch.randn(10, 3),
torch.randn(100, 3),
]
)
grid = fvdb.gridbatch_from_points(
p, voxel_sizes=[[0.1, 0.1, 0.1], [0.15, 0.15, 0.15]], origins=[0.0] * 3
)
# save the grid and features to a compressed nvdb file
path = os.path.join(tempfile.gettempdir(), "two_random_grids.nvdb")
fvdb.save(path, grid, names=["taco1", "taco2"], compressed=True)
```
We can use the `nanovdb_print` command line tool to show information about our saved file. Note how our grids have the `INDEX` class since they have no features and only store the voxel indices.
```bash
The file "/tmp/tmpwnu7qc_7/two_random_grids.nvdb" contains the following 2 grids:
# Name Type Class Version Codec Size File Scale # Voxels Resolution
1 taco1 OnIndex INDEX 32.6.0 BLOSC 1.453 MB 7.181 KB (0.1,0.1,0.1) 10 22 x 43 x 37
2 taco2 OnIndex INDEX 32.6.0 BLOSC 2.326 MB 12.7 KB (0.15,0.15,0.15) 100 33 x 39 x 38
```
We can include N-dimensional features by passing a JaggedTensor as the second argument (or `data` kwarg) to `fvdb.save`. Here, we create a grid with a single, float feature channel for our grids and save it to a `nvdb` file.
```python continuation
# a single, scalar float feature per grid
feats = fvdb.JaggedTensor([torch.randn(x, 1) for x in grid.num_voxels])
# save the grid and features to a compressed nvdb file
path = os.path.join(tempfile.gettempdir(), "two_random_grids.nvdb")
fvdb.save(path, grid, feats, names=["taco1", "taco2"], compressed=True)
```
Again, we can use the `nanovdb_print` command line tool to show information about our saved file.
```bash
The file "/tmp/tmpg9ol5841/two_random_grids.nvdb" contains the following 2 grids:
# Name Type Class Version Codec Size File Scale # Voxels Resolution
1 taco1 float ? 32.6.0 BLOSC 1.47 MB 7.959 KB (1,1,1) 10 26 x 30 x 36
2 taco2 float ? 32.6.0 BLOSC 2.411 MB 16.48 KB (1,1,1) 100 32 x 33 x 34
```
Note how our serialized NanoVDB grids are now of type `float`. fVDB will automatically map N-dimensional features to appropriate NanoVDB types. For feature sizes that don't naturally map to any NanoVDB data types, fVDB will save the feature data as NanoVDB blind-data which will be appropriately read back as N-dimensional features by fVDB.
Let's try to save the same two grids with a `Vec3d` type by creating a JaggedTensor of 3-dimensional double-precision features.
```python continuation
# a 3-vector double feature per grid
feats = fvdb.JaggedTensor([torch.randn(x, 3, dtype=torch.float64) for x in grid.num_voxels])
# save the grid and features to a compressed nvdb file
saved_nvdb = os.path.join(tempfile.gettempdir(), "two_random_vec3d_grids.nvdb")
fvdb.save(saved_nvdb, grid, feats, names=["taco1", "taco2"], compressed=True)
```
```bash
The file "/tmp/tmpwnu7qc_7/two_random_grids.nvdb" contains the following 2 grids:
# Name Type Class Version Codec Size File Scale # Voxels Resolution
1 taco1 Vec3d ? 32.6.0 BLOSC 6.077 MB 28.18 KB (1,1,1) 10 23 x 36 x 35
2 taco2 Vec3d ? 32.6.0 BLOSC 7.346 MB 41.37 KB (1,1,1) 100 37 x 40 x 34
```
## Loading NanoVDB Files
Loading NanoVDB files is as simple as calling `fvdb.load`. You can optionally supply a PyTorch device you'd like the grids and features loaded onto. Here, we load the two grids we saved in the previous section onto our GPU.
```python continuation
# Load the grid and features from the compressed nvdb file
grid_batch, features, names = fvdb.load(saved_nvdb, device=torch.device("cuda:0"))
print("Loaded grid batch total number of voxels: ", grid_batch.total_voxels)
print("Loaded grid batch data type: %s, device: %s" % (features.dtype, features.device))
```
```bash
Loaded grid batch total number of voxels: 110
Loaded grid batch data type: torch.float64, device: cuda:0
```
## Saving/Loading OpenVDB Files
While saving and loading from OpenVDB files is not directly supported by fVDB, it is possible to easily convert between NanoVDB and OpenVDB files using the `nanovdb_convert` command line tool. Here, we convert our previously saved NanoVDB file to an OpenVDB file.
```python notest
vdb_path = os.path.join(tempfile.gettempdir(), "two_random_grids.vdb")
convert_cmd = "nanovdb_convert -v %s %s" % (saved_nvdb, vdb_path)
print("nanovdb_convert our nvdb to vdb: ", convert_cmd)
print(subprocess.check_output(convert_cmd.split()).decode("utf-8"))
```
```bash
nanovdb_convert our nvdb to vdb: nanovdb_convert -v /tmp/tmpnr7gk0tf/two_random_vec3d_grids.nvdb /tmp/tmpnr7gk0tf/two_random_grids.vdb
Opening NanoVDB file named "/tmp/tmpnr7gk0tf/two_random_vec3d_grids.nvdb"
Read 1 NanoGrid(s) from the file named "/tmp/tmpnr7gk0tf/two_random_vec3d_grids.nvdb"
Converting NanoVDB grid named "taco1" to OpenVDB
Converting NanoVDB grid named "taco2" to OpenVDB
```
From here, our grid can be loaded by OpenVDB tools. Roundtripping our converted cache back to NanoVDB is possible with the `nanovdb_convert` tool as well.
Loading the converted OpenVDB file into fVDB shows our familiar grids and features as we expect:
```python notest
convert_cmd = "nanovdb_convert -v -f %s %s" % ( # -f flag forces overwriting existing file
vdb_path,
saved_nvdb,
)
print("nanovdb_convert roundtrip the vdb to nvdb: ", convert_cmd)
print(subprocess.check_output(convert_cmd.split()).decode("utf-8"))
# Load the nvdb file of the converted vdb
grid_batch, features, names = fvdb.load(saved_nvdb, device=torch.device("cuda:0"))
print("Loaded grid batch total number of voxels: ", grid_batch.total_voxels)
print("Loaded grid batch data type: %s, device: %s" % (features.dtype, features.device))
print("\n")
```
```bash
nanovdb_convert roundtrip the vdb to nvdb: nanovdb_convert -v -f /tmp/tmpuwq2r5mx/two_random_grids.vdb /tmp/tmpuwq2r5mx/two_random_vec3d_grids.nvdb
Opening OpenVDB file named "/tmp/tmpuwq2r5mx/two_random_grids.vdb"
Converting OpenVDB grid named "taco1" to NanoVDB
Converting OpenVDB grid named "taco2" to NanoVDB
Loaded grid batch total number of voxels: 110
Loaded grid batch data type: torch.float64, device: cuda:0
```
+1
View File
@@ -0,0 +1 @@
# Ray Tracing
+272
View File
@@ -0,0 +1,272 @@
# A Simple Convolutional U-Net
In this tutorial, you will be guided on how to build a simple sparse convolutional neural network using fVDB.
If you were using MinkowskiEngine to tackle sparse 3D data previously, we will also guide you step-by-step to help you smoothly transfer from it and enjoy speed-ups and memory-savings.
In our simplistic U-Net case, we want to build a Res-UNet with four layers, and each layer contains several blocks.
First, we import basic `fvdb` libraries:
```python
import fvdb
import fvdb.nn as fvnn
from fvdb.nn import VDBTensor
import torch
```
Here `fvdb.nn` is a namespace similar to `torch.nn`, containing a broad definition of different neural layers.
`VDBTensor` is a very thin wrapper around a grid (with type `GridBatch`) and its corresponding feature (with type `JaggedTensor`), and internally makes sure that the two members align.
It also overloads a bunch of operators such as arithmetic computations. Please refer to our API docs to learn more.
We could then build a basic block as follows:
```python continuation
class BasicBlock(torch.nn.Module):
expansion = 1
def __init__(self, in_channels: int, out_channels: int, downsample=None, bn_momentum: float = 0.1):
super().__init__()
self.conv1 = fvnn.SparseConv3d(in_channels, out_channels, kernel_size=3, stride=1)
self.norm1 = fvnn.BatchNorm(out_channels, momentum=bn_momentum)
self.conv2 = fvnn.SparseConv3d(out_channels, out_channels, kernel_size=3, stride=1)
self.norm2 = fvnn.BatchNorm(out_channels, momentum=bn_momentum)
self.relu = fvnn.ReLU(inplace=True)
self.downsample = downsample
def forward(self, x: VDBTensor):
residual = x
out = self.conv1(x)
out = self.norm1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.norm2(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
out = self.relu(out)
return out
```
This defines a similar block as `MinkowskiEngine`:
```python notest
import MinkowskiEngine as ME
class BasicBlock(torch.nn.Module):
expansion = 1
def __init__(self, in_channels: int, out_channels: int, downsample=None, bn_momentum: float = 0.1):
super().__init__()
self.conv1 = ME.MinkowskiConvolution(
in_channels, out_channels, kernel_size=3, stride=1, dilation=1, dimension=3)
self.norm1 = ME.MinkowskiBatchNorm(out_channels, momentum=bn_momentum)
self.conv2 = ME.MinkowskiConvolution(
out_channels, out_channels, kernel_size=3, stride=1, dilation=1, dimension=3)
self.norm2 = ME.MinkowskiBatchNorm(out_channels, momentum=bn_momentum)
self.relu = ME.MinkowskiReLU(inplace=True)
self.downsample = downsample
def forward(self, x):
residual = x
out = self.conv1(x)
out = self.norm1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.norm2(out)
if self.downsample is not None:
residual = self.downsample(x)
out += residual
out = self.relu(out)
return out
```
All the network layers are fully compatible with `torch.nn`. The only difference is that they take `VDBTensor` as input and return a `VDBTensor`.
A full network definition could then be built as:
```python continuation
class FVDBUNetBase(torch.nn.Module):
LAYERS = (2, 2, 2, 2, 2, 2, 2, 2)
CHANNELS = (32, 64, 128, 256, 256, 128, 96, 96)
INIT_DIM = 32
OUT_TENSOR_STRIDE = 1
def __init__(self, in_channels, out_channels, D=3):
super().__init__()
# Output of the first conv concated to conv6
self.inplanes = self.INIT_DIM
self.conv0p1s1 = fvnn.SparseConv3d(in_channels, self.inplanes, kernel_size=5, stride=1, bias=False)
self.bn0 = fvnn.BatchNorm(self.inplanes)
self.conv1p1s2 = fvnn.SparseConv3d(self.inplanes, self.inplanes, kernel_size=2, stride=2, bias=False)
self.bn1 = fvnn.BatchNorm(self.inplanes)
self.block1 = self._make_layer(BasicBlock, self.CHANNELS[0], self.LAYERS[0])
self.conv2p2s2 = fvnn.SparseConv3d(
self.inplanes, self.inplanes, kernel_size=2, stride=2, bias=False)
self.bn2 = fvnn.BatchNorm(self.inplanes)
self.block2 = self._make_layer(BasicBlock, self.CHANNELS[1], self.LAYERS[1])
self.conv3p4s2 = fvnn.SparseConv3d(
self.inplanes, self.inplanes, kernel_size=2, stride=2, bias=False)
self.bn3 = fvnn.BatchNorm(self.inplanes)
self.block3 = self._make_layer(BasicBlock, self.CHANNELS[2], self.LAYERS[2])
self.conv4p8s2 = fvnn.SparseConv3d(
self.inplanes, self.inplanes, kernel_size=2, stride=2, bias=False)
self.bn4 = fvnn.BatchNorm(self.inplanes)
self.block4 = self._make_layer(BasicBlock, self.CHANNELS[3], self.LAYERS[3])
self.convtr4p16s2 = fvnn.SparseConv3d(
self.inplanes, self.CHANNELS[4], kernel_size=2, stride=2, transposed=True, bias=False)
self.bntr4 = fvnn.BatchNorm(self.CHANNELS[4])
self.inplanes = self.CHANNELS[4] + self.CHANNELS[2]
self.block5 = self._make_layer(BasicBlock, self.CHANNELS[4], self.LAYERS[4])
self.convtr5p8s2 = fvnn.SparseConv3d(
self.inplanes, self.CHANNELS[5], kernel_size=2, stride=2, transposed=True, bias=False)
self.bntr5 = fvnn.BatchNorm(self.CHANNELS[5])
self.inplanes = self.CHANNELS[5] + self.CHANNELS[1]
self.block6 = self._make_layer(BasicBlock, self.CHANNELS[5], self.LAYERS[5])
self.convtr6p4s2 = fvnn.SparseConv3d(
self.inplanes, self.CHANNELS[6], kernel_size=2, stride=2, transposed=True, bias=False)
self.bntr6 = fvnn.BatchNorm(self.CHANNELS[6])
self.inplanes = self.CHANNELS[6] + self.CHANNELS[0]
self.block7 = self._make_layer(BasicBlock, self.CHANNELS[6], self.LAYERS[6])
self.convtr7p2s2 = fvnn.SparseConv3d(
self.inplanes, self.CHANNELS[7], kernel_size=2, stride=2, transposed=True, bias=False)
self.bntr7 = fvnn.BatchNorm(self.CHANNELS[7])
self.inplanes = self.CHANNELS[7] + self.INIT_DIM
self.block8 = self._make_layer(BasicBlock, self.CHANNELS[7], self.LAYERS[7])
self.final = fvnn.SparseConv3d(self.CHANNELS[7], out_channels, kernel_size=1)
self.relu = fvnn.ReLU(inplace=True)
def _make_layer(self, block, planes, blocks):
downsample = None
if self.inplanes != planes * block.expansion:
downsample = torch.nn.Sequential(
fvnn.SparseConv3d(
self.inplanes,
planes * block.expansion,
kernel_size=1,
stride=1
),
fvnn.BatchNorm(planes * block.expansion),
)
layers = []
layers.append(
BasicBlock(
self.inplanes, planes,
downsample=downsample
)
)
self.inplanes = planes * block.expansion
for _ in range(1, blocks):
layers.append(BasicBlock(self.inplanes, planes))
return torch.nn.Sequential(*layers)
def forward(self, x):
out = self.conv0p1s1(x)
out = self.bn0(out)
out_p1 = self.relu(out)
grid1 = out_p1.grid
out = self.conv1p1s2(out_p1)
out = self.bn1(out)
out = self.relu(out)
out_b1p2 = self.block1(out)
grid2 = out_b1p2.grid
out = self.conv2p2s2(out_b1p2)
out = self.bn2(out)
out = self.relu(out)
out_b2p4 = self.block2(out)
grid4 = out_b2p4.grid
out = self.conv3p4s2(out_b2p4)
out = self.bn3(out)
out = self.relu(out)
out_b3p8 = self.block3(out)
grid8 = out_b3p8.grid
# tensor_stride=16
out = self.conv4p8s2(out_b3p8)
out = self.bn4(out)
out = self.relu(out)
out = self.block4(out)
# tensor_stride=8
out = self.convtr4p16s2(out, out_grid=grid8)
out = self.bntr4(out)
out = self.relu(out)
out = fvdb.jcat([out, out_b3p8], dim=1)
out = self.block5(out)
# tensor_stride=4
out = self.convtr5p8s2(out, out_grid=grid4)
out = self.bntr5(out)
out = self.relu(out)
out = fvdb.jcat([out, out_b2p4], dim=1)
out = self.block6(out)
# tensor_stride=2
out = self.convtr6p4s2(out, out_grid=grid2)
out = self.bntr6(out)
out = self.relu(out)
out = fvdb.jcat([out, out_b1p2], dim=1)
out = self.block7(out)
# tensor_stride=1
out = self.convtr7p2s2(out, out_grid=grid1)
out = self.bntr7(out)
out = self.relu(out)
out = fvdb.jcat([out, out_p1], dim=1)
out = self.block8(out)
return self.final(out)
```
Please note that here, when we apply transposed convolution layers, we additionally introduce the `out_grid` keyword arguments.
This is needed to guide the output domain of the network, because for perception networks, the output grid topology should align with the input topology.
Note that fVDB will NOT cache the grids to maintain maximum flexibility.
To perform inference with the network, you could simply create a VDBTensor and feed it into the model:
```python continuation
coords = fvdb.JaggedTensor([
(torch.randn(10_000, 3, device='cuda')),
(torch.randn(11_000, 3, device='cuda')),
])
grid = fvdb.gridbatch_from_points(coords)
features = grid.jagged_like(torch.randn(grid.total_voxels, 32, device='cuda'))
sinput = fvnn.VDBTensor(grid, features)
model = FVDBUNetBase(32, 1).to('cuda')
soutput = model(sinput)
```
The output `soutput` will carry gradients during training, and you could train the sparse network accordingly.
Please find a fully working example at `examples/perception_example.py`. The same network is implemented using `MinkowskiEngine` for reference.
+641
View File
@@ -0,0 +1,641 @@
# Volume Rendering
In this example we replace `nerfacc`'s acceleration structure with fVDB and hence scale to unbounded scenes:
```python
import json
import math
import os
from typing import Optional, Tuple, Union
import imageio.v2 as imageio
import matplotlib.pyplot as plt
import numpy as np
import polyscope as ps
import torch
import tqdm
from fvdb import GridBatch
from fvdb import volume_render
TensorPair = Tuple[torch.Tensor, torch.Tensor]
TensorTriple = Tuple[torch.Tensor, torch.Tensor, torch.Tensor]
class _TruncExp(torch.autograd.Function):
@staticmethod
def forward(ctx, x):
ctx.save_for_backward(x)
return torch.exp(x)
@staticmethod
def backward(ctx, dL_dout):
x = ctx.saved_tensors[0]
return dL_dout * torch.exp(x.clamp(-15, 15))
# SH
MAX_SH_BASIS = 10
def eval_sh_bases(basis_dim : int, dirs : torch.Tensor):
"""
Evaluate spherical harmonics bases at unit directions,
without taking linear combination.
At each point, the final result may the be
obtained through simple multiplication.
:param basis_dim: int SH basis dim. Currently, 1-25 square numbers supported
:param dirs: torch.Tensor (..., 3) unit directions
:return: torch.Tensor (..., basis_dim)
"""
SH_C0 = 0.28209479177387814
SH_C1 = 0.4886025119029199
SH_C2 = [
1.0925484305920792,
-1.0925484305920792,
0.31539156525252005,
-1.0925484305920792,
0.5462742152960396
]
SH_C3 = [
-0.5900435899266435,
2.890611442640554,
-0.4570457994644658,
0.3731763325901154,
-0.4570457994644658,
1.445305721320277,
-0.5900435899266435
]
SH_C4 = [
2.5033429417967046,
-1.7701307697799304,
0.9461746957575601,
-0.6690465435572892,
0.10578554691520431,
-0.6690465435572892,
0.47308734787878004,
-1.7701307697799304,
0.6258357354491761,
]
result = torch.empty((*dirs.shape[:-1], basis_dim), dtype=dirs.dtype, device=dirs.device)
result[..., 0] = SH_C0
if basis_dim > 1:
x, y, z = dirs.unbind(-1)
result[..., 1] = -SH_C1 * y;
result[..., 2] = SH_C1 * z;
result[..., 3] = -SH_C1 * x;
if basis_dim > 4:
xx, yy, zz = x * x, y * y, z * z
xy, yz, xz = x * y, y * z, x * z
result[..., 4] = SH_C2[0] * xy;
result[..., 5] = SH_C2[1] * yz;
result[..., 6] = SH_C2[2] * (2.0 * zz - xx - yy);
result[..., 7] = SH_C2[3] * xz;
result[..., 8] = SH_C2[4] * (xx - yy);
if basis_dim > 9:
result[..., 9] = SH_C3[0] * y * (3 * xx - yy);
result[..., 10] = SH_C3[1] * xy * z;
result[..., 11] = SH_C3[2] * y * (4 * zz - xx - yy);
result[..., 12] = SH_C3[3] * z * (2 * zz - 3 * xx - 3 * yy);
result[..., 13] = SH_C3[4] * x * (4 * zz - xx - yy);
result[..., 14] = SH_C3[5] * z * (xx - yy);
result[..., 15] = SH_C3[6] * x * (xx - 3 * yy);
if basis_dim > 16:
result[..., 16] = SH_C4[0] * xy * (xx - yy);
result[..., 17] = SH_C4[1] * yz * (3 * xx - yy);
result[..., 18] = SH_C4[2] * xy * (7 * zz - 1);
result[..., 19] = SH_C4[3] * yz * (7 * zz - 3);
result[..., 20] = SH_C4[4] * (zz * (35 * zz - 30) + 3);
result[..., 21] = SH_C4[5] * xz * (7 * zz - 3);
result[..., 22] = SH_C4[6] * (xx - yy) * (7 * zz - 1);
result[..., 23] = SH_C4[7] * xz * (xx - 3 * yy);
result[..., 24] = SH_C4[8] * (xx * (xx - 3 * yy) - yy * (3 * xx - yy));
return result
def speherical_harmonics(deg: int, sh: torch.Tensor, dirs: torch.Tensor):
C0 = 0.28209479177387814
C1 = 0.4886025119029199
C2 = [
1.0925484305920792,
-1.0925484305920792,
0.31539156525252005,
-1.0925484305920792,
0.5462742152960396
]
C3 = [
-0.5900435899266435,
2.890611442640554,
-0.4570457994644658,
0.3731763325901154,
-0.4570457994644658,
1.445305721320277,
-0.5900435899266435
]
C4 = [
2.5033429417967046,
-1.7701307697799304,
0.9461746957575601,
-0.6690465435572892,
0.10578554691520431,
-0.6690465435572892,
0.47308734787878004,
-1.7701307697799304,
0.6258357354491761,
]
# sh is a tensor of shape [N, C, (deg+1)**2]
# dirs is a tensor of shape [N, 3]
assert 0 <= deg <= 4
assert (deg + 1) ** 2 == sh.shape[-1]
# C = sh.shape[-2]
result = C0 * sh[..., 0]
if deg > 0:
x, y, z = dirs[..., 0:1], dirs[..., 1:2], dirs[..., 2:3]
result = (result -
C1 * y * sh[..., 1] +
C1 * z * sh[..., 2] -
C1 * x * sh[..., 3])
if deg > 1:
xx, yy, zz = x * x, y * y, z * z
xy, yz, xz = x * y, y * z, x * z
result = (result +
C2[0] * xy * sh[..., 4] +
C2[1] * yz * sh[..., 5] +
C2[2] * (2.0 * zz - xx - yy) * sh[..., 6] +
C2[3] * xz * sh[..., 7] +
C2[4] * (xx - yy) * sh[..., 8])
if deg > 2:
result = (result +
C3[0] * y * (3 * xx - yy) * sh[..., 9] +
C3[1] * xy * z * sh[..., 10] +
C3[2] * y * (4 * zz - xx - yy)* sh[..., 11] +
C3[3] * z * (2 * zz - 3 * xx - 3 * yy) * sh[..., 12] +
C3[4] * x * (4 * zz - xx - yy) * sh[..., 13] +
C3[5] * z * (xx - yy) * sh[..., 14] +
C3[6] * x * (xx - 3 * yy) * sh[..., 15])
if deg > 3:
result = (result + C4[0] * xy * (xx - yy) * sh[..., 16] +
C4[1] * yz * (3 * xx - yy) * sh[..., 17] +
C4[2] * xy * (7 * zz - 1) * sh[..., 18] +
C4[3] * yz * (7 * zz - 3) * sh[..., 19] +
C4[4] * (zz * (35 * zz - 30) + 3) * sh[..., 20] +
C4[5] * xz * (7 * zz - 3) * sh[..., 21] +
C4[6] * (xx - yy) * (7 * zz - 1) * sh[..., 22] +
C4[7] * xz * (xx - 3 * yy) * sh[..., 23] +
C4[8] * (xx * (xx - 3 * yy) - yy * (3 * xx - yy)) * sh[..., 24])
return result
def compute_psnr(rgb_gt: torch.Tensor, rgb_est: torch.Tensor) -> torch.Tensor:
x = torch.mean((rgb_gt - rgb_est)**2)
return -10. * torch.log10(x)
def nerf_matrix_to_ngp(pose: np.ndarray, scale: float = 0.33, offset: Union[tuple, list, torch.Tensor] = (0, 0, 0)) -> np.ndarray:
new_pose = np.array([
[pose[1, 0], -pose[1, 1], -pose[1, 2], pose[1, 3] * scale + offset[0]],
[pose[2, 0], -pose[2, 1], -pose[2, 2], pose[2, 3] * scale + offset[1]],
[pose[0, 0], -pose[0, 1], -pose[0, 2], pose[0, 3] * scale + offset[2]],
[0, 0, 0, 1],], dtype=np.float32)
return new_pose
def get_rays(pose: torch.Tensor, intrinsic: torch.Tensor, H: int, W: int, depth: Optional[torch.Tensor] = None) -> TensorTriple:
fx, fy, cx, cy = intrinsic
i, j = torch.meshgrid(torch.linspace(0, W-1, W, device='cpu'), torch.linspace(0, H-1, H, device='cpu'), indexing='ij')
i = i.t().reshape([1, H*W]).expand([1, H*W]) + 0.5
j = j.t().reshape([1, H*W]).expand([1, H*W]) + 0.5
zs = torch.ones_like(i)
xs = (i - cx) / fx * zs
ys = (j - cy) / fy * zs
directions = torch.cat((xs.reshape(-1,1), ys.reshape(-1,1), zs.reshape(-1,1)), dim=-1)
# compute distances
if depth is not None:
dist = torch.norm(directions * depth[:,None], dim=-1, keepdim=True)
else:
dist = torch.empty([])
directions = directions / torch.norm(directions, dim=-1, keepdim=True)
rays_d = (pose[:3,:3] @ directions.transpose(0,1)).transpose(0,1)
rays_o = pose[:3, 3] # [3]
rays_o = rays_o[None, :].expand_as(rays_d) # [N, 3]
return rays_o.squeeze(), rays_d.squeeze(), dist.squeeze()
class NeRFDataset:
def __init__(self, root_path: str = 'data/lego/', scale: float = 1.0, num_rays: int = 4096, mode: str = 'train'):
super().__init__()
self.root_path = root_path
self.scale = scale
self.num_rays = num_rays
self.mode = mode
with open(os.path.join(self.root_path, f'transforms_{self.mode}.json'), 'r', encoding='utf-8') as f:
transform = json.load(f)
# read images
frames = transform["frames"]
self.n_frames = len(frames)
# Read the intrinsics
image = imageio.imread(os.path.join(self.root_path, frames[0]['file_path'] + '.png')) # [H, W, 3] o [H, W, 4]
self.H, self.W = image.shape[:2]
fl_x = fl_y = self.W / (2 * np.tan(transform['camera_angle_x'] / 2)) if 'camera_angle_x' in transform else None
cx = (transform['cx']) if 'cx' in transform else (self.W / 2)
cy = (transform['cy']) if 'cy' in transform else (self.H / 2)
self.intrinsics = np.array([fl_x, fl_y, cx, cy])
self.rays = []
self.rgbs = []
self.depths = []
self.poses = []
self.pc = []
self.pc_rgbs = []
for f in tqdm.tqdm(frames, desc=f'Loading {self.mode} data'):
f_path = os.path.join(self.root_path, f['file_path'] + '.png')
pose = nerf_matrix_to_ngp(np.array(f['transform_matrix'], dtype=np.float32), scale=self.scale)
image = imageio.imread(f_path) / 255.0 # [H, W, 3] o [H, W, 4]
depth = None
if self.mode == 'train':
f_path_depth = os.path.join(self.root_path, f['file_path'] + '_depth.npy')
depth = np.load(f_path_depth).reshape(-1)
ray_o, ray_d, depth = get_rays(torch.from_numpy(pose), torch.from_numpy(self.intrinsics), self.H, self.W, depth)
# Scale the depth
depth_mask = depth < 1000
depth *= scale
rgbs = torch.from_numpy(image).reshape(self.H * self.W, -1)
self.poses.append(pose)
self.rays.append(torch.cat([ray_o, ray_d], 1))
self.rgbs.append(rgbs)
if self.mode == 'train':
self.depths.append(depth)
self.pc.append(ray_o[depth_mask, :3] + ray_d[depth_mask, :3] * depth[depth_mask,None])
self.pc_rgbs.append(rgbs[depth_mask,:3])
self.rays = torch.vstack(self.rays)
self.rgbs = torch.vstack(self.rgbs)
if self.mode == 'train':
self.depths = torch.cat(self.depths) # Note that depth denotes the distance along the ray
self.pc = torch.vstack(self.pc)
self.pc_rgbs = torch.vstack(self.pc_rgbs)
def get_point_cloud(self, downsample_ratio: float = 1.0, return_color: bool = False) -> Union[torch.Tensor, TensorPair]:
if self.mode == 'train':
assert isinstance(self.pc, torch.Tensor)
if return_color:
assert isinstance(self.pc_rgbs, torch.Tensor)
dri = int(1 / downsample_ratio)
pts = self.pc[::dri, :]
rgb = self.pc_rgbs[::dri, :]
return pts, rgb
return self.pc[::int(1/downsample_ratio),:]
else:
raise ValueError('Only training data has depth information!')
def __len__(self):
if self.mode == 'train':
return 1000
else:
return self.n_frames
def __getitem__(self, idx):
# raise an error to now iterate in infinity
if idx >= len(self): raise IndexError
if self.mode == 'train':
assert isinstance(self.rays, torch.Tensor)
idxs = np.random.choice(self.rays.shape[0], self.num_rays)
return {'rays_o': self.rays[idxs,:3],
'rays_d': self.rays[idxs,3:6],
'rgba': self.rgbs[idxs],
'depth': self.depths[idxs],
'idxs': idxs}
else:
# raise an error to now iterate in infinity
if idx >= len(self): raise IndexError
assert isinstance(self.rays, torch.Tensor)
assert isinstance(self.rgbs, torch.Tensor)
return {'rays_o': self.rays[idx * self.W * self.H : (idx + 1) * self.W * self.H, :3],
'rays_d': self.rays[idx * self.W * self.H : (idx + 1) * self.W * self.H, 3:6],
'rgba': self.rgbs[idx * self.W * self.H : (idx + 1) * self.W * self.H,],
'depth': None}
def make_ray_grid(origin, nrays, minb=(-0.45, -0.45), maxb=(0.45, 0.45),
device: Union[str, torch.device]='cpu', dtype=torch.float32):
ray_o = torch.tensor([origin] * nrays**2) #+ p.mean(0, keepdim=True)
ray_d = torch.from_numpy(
np.stack([a.ravel() for a in
np.mgrid[minb[0]:maxb[0]:nrays*1j,
minb[1]:maxb[1]:nrays*1j]] +
[np.ones(nrays**2)], axis=-1).astype(np.float32))
ray_d /= torch.norm(ray_d, dim=-1, keepdim=True)
ray_o, ray_d = ray_o.to(device).to(dtype), ray_d.to(device).to(dtype)
return ray_o, ray_d
def evaluate_density_and_color(dual_grid: GridBatch, sh_features: torch.Tensor, o_features: torch.Tensor,
ray_d: torch.Tensor, pts: torch.Tensor) -> TensorPair:
pt_features = dual_grid.sample_trilinear(pts, sh_features.view(sh_features.shape[0], -1)).jdata.view(pts.shape[0], 3, 9)
pt_o_features = dual_grid.sample_trilinear(pts, o_features.unsqueeze(-1)).jdata.squeeze(-1)
return _TruncExp.apply(pt_o_features), torch.sigmoid(speherical_harmonics(2, pt_features, ray_d))
def render(primal_grid: GridBatch, dual_grid: GridBatch,
sh_features: torch.Tensor, o_features: torch.Tensor,
ray_o: torch.Tensor, ray_d: torch.Tensor,
tmin: torch.Tensor, tmax: torch.Tensor, step_size: float,
t_threshold: float = 0.0, chunk: bool = False) -> TensorTriple:
pack_info, ray_idx, ray_intervals = \
primal_grid.uniform_ray_samples(ray_o, ray_d, tmin, tmax, step_size)
ray_t = ray_intervals.jdata.mean(1)
ray_delta_t = (ray_intervals.jdata[:, 1] - ray_intervals.jdata[:, 0]).contiguous()
ray_pts = ray_o[ray_idx.jdata] + ray_t[:, None] * ray_d[ray_idx.jdata]
if chunk:
ray_density = []
ray_color = []
ray_d = ray_d[ray_idx.jdata]
chunk_size = 400000
for i in range(ray_d.shape[0]//chunk_size + 1):
ray_density_chunk, ray_color_chunk = evaluate_density_and_color(dual_grid, sh_features, o_features,
ray_d[i*chunk_size:(i+1)*chunk_size, :],
ray_pts[i*chunk_size:(i+1)*chunk_size, :])
ray_density.append(ray_density_chunk)
ray_color.append(ray_color_chunk)
ray_density = torch.cat(ray_density, 0)
ray_color = torch.vstack(ray_color)
else:
ray_density, ray_color = evaluate_density_and_color(dual_grid, sh_features, o_features,
ray_d[ray_idx.jdata], ray_pts)
# Do the volume rendering
# print(ray_density.shape, ray_color.shape, ray_delta_t.shape, ray_t.shape, pack_info.jdata.shape)
rgb, depth, opacity, _, _ = volume_render(ray_density, ray_color, ray_delta_t,
ray_t, pack_info.jdata, t_threshold)
return rgb, depth, opacity[:, None]
def tv_loss(dual_grid: GridBatch, ijk: torch.Tensor, sh_features: torch.Tensor, o_features: torch.Tensor, res) -> TensorPair:
nhood = dual_grid.neighbor_indexes(ijk, 1).jdata.view(-1, 3, 3, 3)
n_up = nhood[:, 1, 0, 0]
n_right = nhood[:, 0, 1, 0]
n_front = nhood[:, 0, 0, 1]
n_center = nhood[:, 0, 0, 0]
mask = torch.logical_and(torch.logical_and(n_center != -1, n_up != -1), n_front != -1)
fmask = mask.float()
n_up_mask, n_right_mask, n_center_mask, n_front_mask = n_up[mask], n_right[mask], n_center[mask], n_front[mask]
diff_up_sh = (sh_features[n_up_mask] - sh_features[n_center_mask]) / (256.0 / res)
diff_right_sh = (sh_features[n_right_mask] - sh_features[n_center_mask]) / (256.0 / res)
diff_front_sh = (sh_features[n_front_mask] - sh_features[n_center_mask]) / (256.0 / res)
diff_up_o = (o_features[n_up] * fmask - o_features[n_center]) / (256.0 / res)
diff_right_o = (o_features[n_right] * fmask- o_features[n_center]) / (256.0 / res)
diff_front_o = (o_features[n_front] * fmask - o_features[n_center]) / (256.0 / res)
tv_reg_sh = (diff_up_sh ** 2.0 + diff_right_sh ** 2.0 + diff_front_sh ** 2.0).sum(-1).sum(-1)
tv_reg_o = (diff_up_o ** 2.0 + diff_right_o ** 2.0 + diff_front_o ** 2.0)
return tv_reg_sh.mean(), tv_reg_o.mean()
def main():
# Configuration parameters
device = torch.device('cuda')
dtype = torch.float32
# scene_aabb = 1.0
starting_resolution = 256
resolution = starting_resolution
vox_size = (1.0 / resolution, 1.0 / resolution, 1.0 / resolution)
vox_origin = (vox_size[0]/2, vox_size[1]/2, vox_size[2]/2)
ray_step_size = math.sqrt(3) / 512
rays_per_batch = 4096
lr_o = 1e-1
lr_sh = 1e-2
plot_every = 2
num_epochs = 30
bg_color = (0.0, 0.0, 0.0)
t_threshold = 1e-5
# Create the dataset. Assumes there is a file <repository_root>/data/lego_test.h5
data_path = os.path.join(os.path.dirname(__file__), "..", "data/lego/")
if not os.path.exists(data_path):
data_url = "https://drive.google.com/drive/folders/1i6qMn-mnPwPEioiNIFMO8QJlTjU0dS1b?usp=share_link"
raise RuntimeError(f"You need to download the data at {data_url} "
"into <repository_root>/data "
"in order to run this script")
train_dataset = NeRFDataset(data_path, scale=0.33, num_rays=rays_per_batch, mode='train')
test_dataset = NeRFDataset(data_path, scale=0.33, num_rays=rays_per_batch, mode='test')
# Create a sparse grid used to support features and do ray queries
print("Building grid...")
primal_grid = GridBatch(device=device)
primal_grid.set_from_dense_grid(1, [resolution]*3, [-resolution//2]*3, voxel_sizes=vox_size, voxel_origins=vox_origin)
dual_grid = primal_grid # primal_grid.dual_grid()
print("Done bulding the grid!")
# Initialize features at the voxel centers
xyz = primal_grid.ijk.jdata / (resolution / 2.0)
print(xyz.min(0)[0], xyz.max(0)[0])
sh_features = torch.stack([eval_sh_bases(9, xyz)]*3, dim=1)
print(sh_features.shape)
sh_features = sh_features.to(device=device, dtype=dtype)
o_features = torch.rand(dual_grid.total_voxels)
o_features = o_features.to(device=device, dtype=dtype)
o_features.requires_grad = True
sh_features.requires_grad = True
# Init optimizer
param_group = []
param_group.append({'params': o_features, 'lr': lr_o })
param_group.append({'params': sh_features, 'lr': lr_sh })
# optimizer = torch.optim.Adam(param_group)
optimizer = torch.optim.RMSprop(param_group)
# scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=30, eta_min=lr/30)
print("Starting training!")
ps.init() # Initialize 3d plotting
for epoch in tqdm.trange(num_epochs):
if resolution <= starting_resolution:
all_ijk = dual_grid.ijk.jdata
else:
all_ijk = None
pbar = tqdm.tqdm(enumerate(train_dataset)) # type: ignore
for _, batch in pbar: # type: ignore
optimizer.zero_grad()
ray_o, ray_d = batch['rays_o'].to(device=device, dtype=dtype), \
batch['rays_d'].to(device=device, dtype=dtype)
tmin = torch.zeros(ray_o.shape[0]).to(ray_o)
tmax = torch.full_like(tmin, 1e10)
# Render color and depth along rays
rgb, depth, opacity = render(primal_grid, dual_grid, sh_features, o_features, ray_o, ray_d,
tmin, tmax, ray_step_size, t_threshold=t_threshold)
rgb = opacity * rgb + (1.0 - opacity) * torch.tensor(bg_color).to(rgb)[None, :]
# RGB loss
rgb_gt = batch['rgba'].to(rgb)[:,:3]
loss_rgb = torch.nn.functional.mse_loss(rgb, rgb_gt) # torch.nn.functional.huber_loss(rgb, rgb_gt) / 5.
# Depth loss
# depth_gt = batch['depth'].to(rgb)
# depth_mask = depth_gt < np.sqrt(3) # Mask out rays that miss the object
# loss_depth = torch.nn.functional.l1_loss(depth[depth_mask], depth_gt[depth_mask]) / 100
if resolution <= starting_resolution:
assert all_ijk is not None
random_ijk = all_ijk[torch.randperm(all_ijk.shape[0])[:int(.1*dual_grid.total_voxels)]]
tv_reg_sh, tv_reg_o = tv_loss(dual_grid, random_ijk, sh_features, o_features, resolution)
tv_reg = 1e-1 * tv_reg_sh + 1e-2 * tv_reg_o
else:
tv_reg_sh = torch.tensor([0.0]).to(device)
tv_reg_o = torch.tensor([0.0]).to(device)
tv_reg = torch.tensor([0.0]).to(device)
# Total loss to minimize
loss = loss_rgb + tv_reg #+ 0.1 * loss_depth
loss.backward()
optimizer.step()
# Compute current PSNR
psnr = compute_psnr(rgb, rgb_gt)
# Log losses in tqdm progress bar
pbar.set_postfix({"Loss": f"{loss.item():.4f}",
"Loss RGB": f"{loss_rgb.item():.4f}",
# "Loss Depth": f"{loss_depth.item():.4f}",
"Loss TV (sh)": f"{tv_reg_sh.item():.4f}",
"Loss TV (o)": f"{tv_reg_o.item():.4f}",
"PSNR": f"{psnr.item():.2f}"})
# scheduler.step()
if epoch % plot_every == 0:
with torch.no_grad():
torch.cuda.empty_cache()
grid_res = 512
ray_o, ray_d = make_ray_grid((0., 0.15, 1.2), grid_res, device=device, dtype=dtype)
ray_d = - ray_d
tmin = torch.zeros(ray_o.shape[0]).to(ray_o)
tmax = torch.full_like(tmin, 1e10)
rgb, depth, opacity = render(primal_grid,dual_grid, sh_features, o_features, ray_o, ray_d,
tmin, tmax, ray_step_size, t_threshold=t_threshold, chunk=True)
rgb = opacity * rgb + (1.0 - opacity) * torch.tensor(bg_color).to(rgb)[None, :]
rgb_img = rgb.clip(0.0, 1.0).detach().cpu().numpy().reshape([grid_res, grid_res, 3])
depth_img = depth.detach().cpu().numpy().reshape([grid_res, grid_res])
plt.figure()
plt.imshow(rgb_img)
plt.figure()
plt.imshow(depth_img)
plt.show()
ray_v = torch.cat([ray_o, ray_o + ray_d*0.33]).cpu().numpy()
ray_e = np.array([[i, i + ray_o.shape[0]] for i in range(ray_o.shape[0])])
ps.register_curve_network("rays", ray_v, ray_e, radius=0.00002)
vox_ijk = primal_grid.ijk.jdata
vox_ctrs = primal_grid.grid_to_world(vox_ijk.to(dtype)).jdata
vox_density, vox_color = evaluate_density_and_color(dual_grid, sh_features, o_features,
torch.ones_like(vox_ctrs), vox_ctrs)
# Subdivide
if epoch > 0:
sh_features, sub_grid = dual_grid.subdivide(2, sh_features.view(sh_features.shape[0], -1), mask=vox_density > 0.25)
o_features, sub_grid = dual_grid.subdivide(2, o_features.unsqueeze(-1), mask=vox_density > 0.25)
o_features = o_features.jdata.squeeze(-1)
sh_features = sh_features.jdata.reshape(sh_features.rshape[0], 3, -1)
sh_features.requires_grad = True
o_features.requires_grad = True
resolution *= 2.0
ray_step_size /= 2.0
print(f"Subdivided grid with {dual_grid.total_voxels} to {sub_grid.total_voxels}")
dual_grid = sub_grid
primal_grid = sub_grid
camera_origins = []
for pose in test_dataset.poses:
camera_origins.append(pose @ np.array([0.0, 0.0, 0.0, 1.0]))
camera_origins = np.stack(camera_origins)[:, :3]
ps.register_point_cloud("camera origins", camera_origins)
v, e = primal_grid.viz_edge_network
v, e = v.jdata, e.jdata
ps.register_curve_network("grid", v.cpu(), e.cpu(), radius=0.0001)
pc = ps.register_point_cloud("vox centers", vox_ctrs.cpu(),
point_render_mode='quad')
pc.add_scalar_quantity("density", vox_density.cpu(), enabled=True)
pc.add_scalar_quantity("density thresh", (vox_density.cpu() < .25).float(), enabled=True)
pc.add_color_quantity("rgb", vox_color.cpu(), enabled=False)
ps.show()
print("Starting testing!")
pbar = tqdm.tqdm(enumerate(test_dataset)) # type: ignore
psnr_test = []
for _, batch in pbar: # type: ignore
with torch.no_grad():
ray_o, ray_d = batch['rays_o'].to(device=device, dtype=dtype), \
batch['rays_d'].to(device=device, dtype=dtype)
tmin = torch.zeros(ray_o.shape[0]).to(ray_o)
tmax = torch.full_like(tmin, 1e10)
rgb_gt = batch['rgba'].to(rgb)[:,:3] # type: ignore
# Render color and depth along rays
rgb, depth, opacity = render(primal_grid, dual_grid, sh_features, o_features, ray_o, ray_d,
tmin, tmax, ray_step_size, t_threshold=t_threshold, chunk=True)
rgb = opacity * rgb + (1.0 - opacity) * torch.tensor(bg_color).to(rgb)[None, :]
# Compute current PSNR
psnr = compute_psnr(rgb, rgb_gt)
psnr_test.append(psnr.item())
# Log losses in tqdm progress bar
pbar.set_postfix({"PSNR": f"{psnr.item():.2f}"})
print(f"Mean PSNR on the test set across {len(test_dataset)} images: {torch.tensor(psnr_test).mean().item()} ")
if __name__ == "__main__":
main()
```
+23
View File
@@ -0,0 +1,23 @@
diff --git a/CMakeLists.txt b/CMakeLists.txt
index f726a0a..78dde8b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -72,7 +72,7 @@
# DEV: static includes blosc.a and blosc.h
-cmake_minimum_required(VERSION 2.8.12)
+cmake_minimum_required(VERSION 3.5)
if(NOT CMAKE_VERSION VERSION_LESS 3.3)
cmake_policy(SET CMP0063 NEW)
endif()
diff --git a/internal-complibs/zlib-1.2.13/CMakeLists.txt b/internal-complibs/zlib-1.2.13/CMakeLists.txt
index b412dc7..3efb674 100644
--- a/internal-complibs/zlib-1.2.13/CMakeLists.txt
+++ b/internal-complibs/zlib-1.2.13/CMakeLists.txt
@@ -1,4 +1,4 @@
-cmake_minimum_required(VERSION 2.4.4)
+cmake_minimum_required(VERSION 3.5)
set(CMAKE_ALLOW_LOOSE_LOOP_CONSTRUCTS ON)
project(zlib C)
+38
View File
@@ -0,0 +1,38 @@
name: fvdb_build
channels:
- conda-forge
- nodefaults
dependencies:
- blas=*=mkl
- ca-certificates
- certifi
- cmake
- cuda-command-line-tools
- cuda-compiler
- cuda-nvrtc-dev
- cuda-nvtx-dev
- cuda-version=12.0
- cxx-compiler
- gcc_linux-64=11
- git
- gitpython
- gxx_linux-64=11
- libcublas-dev
- libcusolver-dev
- libcusparse-dev
- libpng
- linkify-it-py
- make
- myst-parser
- ninja
- numpy
- openssl
- psutil
- python=3.11
- pytorch-gpu=2.4.1[build=cuda120*]
- scikit-build-core
- setuptools>=68.2.2
- sphinx_rtd_theme
- sphinx>=7.0.0
- tqdm
- zlib
+9
View File
@@ -0,0 +1,9 @@
pip >= 23.3.1
scikit_build_core
wheel
GitPython
ninja
nvidia-cudnn-cu12
tqdm
numpy
torch

Some files were not shown because too many files have changed in this diff Show More