Disable leak detection only while the instrumented exp2cxx build tool
generates schema sources. LeakSanitizer cannot run under common tracers
and sandboxes, which otherwise makes an ASan build fail before its tests
can start.
Keep full descriptor metadata as the default and add an opt-in
structural profile for conversion-oriented late-bound API v2 users.
Strip descriptive and constraint text while retaining the complete
runtime descriptor graph. Pass the profile through CMake and correct
the scanner's compact late-bound file lists.
AP242e4 full-to-structural measurements:
Generated output falls from 54,839 lines and 3,304,007 bytes to
29,993 lines and 1,525,897 bytes. The packed schema source shrinks
68.1%. The schema DSO falls from 2,178,296 to 720,120 bytes, and peak
compile RSS drops 6.8%. File count and 49 ELF relocations are
unchanged.
The NIST AP242e4 sample still creates and validates all 10,609
instances with no errors or warnings and writes the same 949,685-byte
semantic result. Twenty read/write runs improve from 4.76 to 4.71
seconds and peak RSS falls from 82,276 to 78,900 KiB.
Add stable entity, type, and attribute IDs through SchemaModule. Make
late-bound compatibility names opt-in, table-drive rules and complex
metadata, and emit entity metadata directly into bounded chunks.
AP242e4 late-bound output changed as follows:
* generated files: 5,719 to 905 (-84.2%)
* generated lines: 254,220 to 123,552 (-51.4%)
* generated bytes: 13,942,523 to 9,804,680 (-29.7%)
* generation wall time: 2.74s to 0.56s (-79.6%)
* compile/link wall time: 157.68s to 37.02s (-76.5%)
* compile/link peak RSS: 593,320 KiB to 199,156 KiB (-66.4%)
* shared library size: 9,956,640 to 7,716,096 bytes (-22.5%)
* read/write wall time: 0.26s to 0.25s
ELF relocations increased from 28,843 to 39,719 due to descriptor-slot
tables. The packed schema image stage is intended to remove this cost.
The real AP242 sample created and validated 10,609 instances with zero
errors and warnings. Rewritten output matched except for its timestamp.
- API v2 table-driven schema, entity, type, attribute, and inheritance
initialization via include/clstepcore/ schemaInit.h.
- --late-bound/-T, producing descriptor-backed entities without
generated early-bound entity classes.
- CMake controls:
- SC_EXP2CXX_API_VERSION=2
- SC_EXP2CXX_LATE_BOUND=ON
- Runtime late-bound construction and owned attribute storage in
src/clstepcore/entityDescriptor.cc and src/clstepcore/STEPattribute.cc.
- Documentation and regression coverage, including incompatible option
handling.
AP242e4 generated-output results:
Mode Lines Bytes
━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━ ━━━━━━━━━━
Default v1 1,210,640 57.57 MB
──────────────────── ─────────── ──────────
API v2 early-bound 509,001 25.20 MB
──────────────────── ─────────── ──────────
API v2 late-bound 254,123 13.94 MB
The late-bound AP242 library compiled under C++11 and successfully
processed the BRL-CAD NIST sample: 10,609 instances
created and validated, with zero errors and warnings.
Opt in with either:
exp2cxx --api-version 2 schema.exp
or:
cmake -DSC_EXP2CXX_API_VERSION=2 ...
Version 1 remains the default, and the regression test verifies default
output is byte-identical to explicit v1
output.
API v2 now provides:
- Compact SELECT classes backed by shared descriptor-driven runtime code
instead of repeated generated parsing, assignment, and serialization
bodies.
- Generic Set* and typed getter APIs in
include/clstepcore/sdaiSelect.h:109.
- Dependency-bounded entity headers and source-owned dependencies in
src/exp2cxx/classes_entity.c:968.
- Out-of-line entity creators.
- Unity chunks that no longer include the all-entity header.
- C++11 output; C++17 did not offer enough additional benefit for this
tranche.
The command-line handling is in src/exp2cxx/fedex_main.c:85, while
compact SELECT generation begins in src/exp2cxx/selects.c:702.
Measured using the BRL-CAD ap242e4.exp, Release builds, eight parallel
jobs, and 64:8 unity chunks:
Metric API v1 Final API v2 Reduction
━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━ ━━━━━━━━━━━
Generated C++ lines 1,210,640 522,445 56.8%
──────────────────────── ─────────────── ────────────── ───────────
Generated bytes 57,566,051 27,202,644 52.7%
──────────────────────── ─────────────── ────────────── ───────────
Schema build wall time 183.22 s 88.31 s 51.8%
──────────────────────── ─────────────── ────────────── ───────────
Compiler CPU time 641.53 s 378.49 s 41.0%
──────────────────────── ─────────────── ────────────── ───────────
Peak RSS 1,035,112 KiB 916,412 KiB 11.5%
──────────────────────── ─────────────── ────────────── ───────────
Object-file bytes 70,472,952 40,031,088 43.2%
──────────────────────── ─────────────── ────────────── ───────────
Shared-library text 34,361,400 19,420,467 43.5%
──────────────────────── ─────────────── ────────────── ───────────
Generation wall time 1.52 s 0.69 s 54.6%
Schemas commonly contain many more entity implementations than type
implementations. Accept N:N chunk limits and use smaller defaults so
both kinds of generated unity files stay manageable.
Generate schema source manifests and bounded unity translation
units, then make schema targets depend on the generated absolute
paths. This avoids oversized compiler inputs while retaining
unity-build performance.
Use modern target_link_libraries scopes so libraries publish their
transitive requirements and executables keep theirs private. This
preserves schema test linkage with current CMake versions.
Occasional compile problems with libexpress appear to be an attempt to
compile with an incompletely generated header. Make the dependency
explicit to assist CMake in generating proper build ordering rules.
In case we do end up with buffers with contents at the BUFSIZ limit,
make sure we leave room to set a null termination char. Not always
clear where these buffers might get used or re-used, so just bump their
size across the board.
Getting an intermittent failure on Windows where the expscan.h header
apparently isn't present on the file system soon enough for the build to
succeed (a subsequent reinvocation of the build command does succeed.)
Try the standard technique of making another file write depend on the
outputs of the header and src file, then define a target depending on
that sentinel file and make the obj libraries depend on that target.
BRL-CAD's supporting system for building this will need some updates
once we require a newer re2c - to make testing easier in the meantime,
don't require the newer version.
This is a classic problem - the file copy will begin, the file will be
"present" in some sense on the file system, and build steps that need it
will begin - but they won't find what they actually need because the
copy isn't fully executed. The standard approach for this is to add a
second file which is only created after the first is done, and then make
the target depend on the second file rather than the first.
The relatively complicated logic trying to pull in the git sha1 id for
each build is proving fragile in some scenarios. Just go with a
simpler encoding of the defined library version.
This has proven to be more trouble than it is worth - changes
to the lexer and parser inputs are relatively rare at this point,
and different versions of re2c/lemon can break the checks.
Clifford Yapp