AnshulandClaude Sonnet 4.6
ff42ddc885
* support weekly benchmarks * updated for perftest * rem clipper * other tests * parse cmake output + use ember-benchmark-cases.json * add only required STl files for ember tests * remove ember tests+fix corrupted stl file * shell script to py + minor changes * mac-os * more info of machine with linux fallback * storage for weekly benchmarks * rebase * binary stl * clean * . * fix ember phase capture and drop removed phases - merge stderr into stdout in run_ember_case so man_bench diagnostic output is not silently dropped - remove Assembly/Triangulation/Simplification/Sorting from INDEPENDENT_PHASES; their Timer::Print calls were removed in PR #1745 and they always produced 0.0; only Intersect12 and Winding03 phases are still reported - update build_ember_summary table to match the 4 remaining phases Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * revert runner * add frontend + trigger on publish + san extended test * fix formatting * TEMP: allow workflow_dispatch to exercise data-publish steps for testing Not for merge - revert before this PR lands. Only relaxing the if: gate so a manual run can validate the folded run: > block and sanitizer arg passing reviewers asked about, without waiting for a scheduled/release run. * Revert "TEMP: allow workflow_dispatch to exercise data-publish steps for testing" This reverts commit e00c5f5784eefa27255681dce4105bd45be3daa0. * Move weekly sanitizer lane to Linux, fix artifact path, derive compiler - Move build_sanitizer/test_sanitizer in weekly_benchmark.yml from macos-15 to ubuntu-24.04, matching the existing PR-triggered sanitizer lane in manifold.yml, since nothing about ASan/UBSan requires macOS here. Drops the compiler overrides and macOS-only setup steps. - Download sanitizer build artifacts into build/ instead of the workspace root, so manifold_test's baked-in RPATH to libsamples can actually resolve. It was crashing on load with "Library not loaded: @rpath/libsamples.3.dylib" because the artifact download flattened the build/ directory away. - Capture the sanitizer build's own cmake configure log and derive SANITIZER_SUMMARY_COMPILER from it at runtime instead of hardcoding the string, reusing system_metadata.py's existing helpers. - Trim set -euo pipefail to set -u in the data-publish steps, since GitHub Actions' default bash invocation already applies -e/pipefail. - Remove the duplicate weeklybenchmark-test job from manifold.yml in favor of testing weekly_benchmark.yml directly via workflow_dispatch. * Merge dashboard pages + remove dead JS fallbacks + gtest JSON output + simplify perf summary formatting + clean up .gitignore * Stop forcing clang-18 for sanitizer builds, use runner default (gcc) * small changes * -Wno-maybe-uninitialized * review changes * dead code * fix cmke read * change flags * remove redundant flags and schema version * /dashboard route + remove upload route + change dead and retention changes --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1.7 KiB
1.7 KiB
Ember Benchmark
The paper EMBER: Exact Mesh Boolean via Efficient & Robust Local Arrangments (see https://dl.acm.org/doi/10.1145/3528223.3530181) has supplemental material that describes 1000 tests using meshes from the Thinki10K dataset (see https://ten-thousand-models.appspot.com/)
Dataset
Download the dataset from https://ten-thousand-models.appspot.com ,
decompress and put the raw_meshes directory inside testfiles.
Running the benchmark
- Download the dataset.
- Build the binary with
MANIFOLD_PARandASSIMP_ENABLEenabled. The binary should be insidebuild/extras(relative to the project root). - Run the selected Ember benchmark cases through
scripts/run_weekly_benchmarks.py, or runman_benchdirectly with the case arguments fromtestfiles/ember-benchmark-cases.json. - Run
python analyze_ember_tests.pyto analyze the fullbenchmark.csvresult.
Files
man_benchis a binary that reads two .stl files and has corresponding transform arguments. It converts those to MeshGL, then to Manifold, then runs a boolean difference, and converts the result back to MeshGL. It times the total time take to do this (not including loading the stl files into MeshGL format), and appends a line to a benchmarks.csv file with the argument and timing information.testfiles/test_file/ember-benchmark-cases.json: JSON file describing each test case.analyze_ember_tests.pyreads the benchmark.csv file and calculates statistics and does a plot.bug-case.jsonis a single test case that takes abnormally long.m3max_benchmarks.csvis the benchmarks file gotten by running on a Macbook Pro M3 Max (12 performance cores, 4 efficiency cores).