STEPnode::_null had no default: STEPnode has no constructor, so a node built
by a subclass that does not assign it starts out with whatever was on the
heap, and is_null() then answers at random. Give STEPnode a constructor that
starts nodes null.
EntityNode::StrToVal() and ::STEPread() additionally have to set it per
outcome - they resolve an entity reference and must report null exactly when
the reference did not resolve. Before this, a node whose reference failed to
resolve could still report itself non-null.
A data section indexes its instances from its own constructor, through
addLazyInstance(), under the section id it WILL be given - and
lazyFileReader::initP21() registers the section only afterwards, and only
if its scan succeeded. A section that fails part way therefore leaves
every instance it had already indexed naming a section that does not
exist.
typeFromFile() then indexes _dataSections with it unconditionally, and
loadInstance() does the same behind an assert() that Release compiles
out. On an empty vector that is an access violation. sourceRecord()
already makes exactly this test; the other two now make it too and answer
0 with a diagnostic, as they already do for an instance found in no
section.
The instances stay in the index - totalInstanceCount() still counts them
and getInstances() still lists them. What changes is that asking for one
of them fails the way every other unloadable instance fails.
Reachable from a thirteen-line file, added as test/lazy_broken_section.stp
and checked by the lazy_index test: a DATA section holding one instance
and then a line that is not one. Without the two changes below it, that
test dies with SIGSEGV inside typeFromFile().
Note that the new break in loadInstance() leaves `inst` null, and
isNilSTEPentity( 0 ) is false, so the null would be cached as a loaded
instance were it not for the guard added in "lazy: do not cache a null
instance as a loaded one" earlier in this branch. With the bounds test
alone the process survives the two calls and then dies on the way out.
Materializing each instance repeated the same FILE_SCHEMA warning.
Keep a thread-safe claim flag on the lazy file reader so concurrent
loads and multiple DATA sections report the ambiguity once per input
file.
Preserve the component keyword vector already parsed while indexing a
complex Part 21 instance, and expose it through a read-only lazy manager
API. This lets clients interpret complex instances without reparsing raw
source text or materializing the SDAI graph.
Construct SELECT, enumeration, and aggregate values from descriptors so
compact output needs no generated type classes or factories. Preserve
enumeration elements and renamed schema items in the packed image.
Make each public schema header independently includable through the small
SchemaModule interface. Keep runtime and model declarations internal.
AP242e4 output falls from 865 to 10 files and clean compile/link time
falls from 18.71 to 1.21 seconds. The schema DSO falls from 3,503,352
to 2,178,296 bytes, with ELF relocations falling from 9,620 to 49.
Replace late-bound descriptor procedures with validated, offset-based
schema and complex metadata images. Use an explicit per-load context
for reentrant module construction while retaining the version 1
compatibility path.
AP242e4 generated output falls from 9,060,249 to 4,865,657 bytes and
clean compile/link time falls from 41.49 to 18.71 seconds. The schema
DSO falls from 7,470,304 to 3,503,352 bytes and ELF relocations fall
from 33,221 to 9,620.
Replace generated SchemaModuleSlot pointer arrays with a versioned,
16-byte SchemaModuleImage. Shared initialization helpers capture
entity, type, and attribute descriptors in stable generated-ID order
and transfer them to SchemaModule without one relocated pointer per
descriptor.
Retain the slot-based initialization entry point for existing API v2
output, and leave API v1 generation unchanged.
For AP242e4, generated output falls from 123,552 to 117,046 lines
(-5.3%) and from 9,804,680 to 9,060,249 bytes (-7.6%). The schema
module source falls from 808,907 to 64,367 bytes (-92.0%).
The schema DSO falls from 7,728,384 to 7,470,304 bytes (-3.3%), while
ELF relocations fall from 39,719 to 33,221 (-16.4%). Comparable clean
compile times are unchanged, and paired read/write runs improve by 0.6%.
The real file still creates and validates 10,609 instances with zero
errors and warnings. Its 949,685-byte rewrite differs only by its
timestamp.
Resolve each flattened inherited attribute layout once after schema
initialization. Late-bound instance construction now follows descriptor
and index records directly instead of recursively traversing supertypes
and repeating derived and redefined attribute name searches.
Keep the cache outside EntityDescriptor so its object layout and API v1
ABI remain unchanged. Invalidate cached subtype layouts when descriptor
graphs are edited, and retain lazy preparation for non-module callers.
For the AP242e4 sample's most common entity, constructing 1,531,000
DIRECTION instances improved from 0.84s to 0.63s (-25.0%); user time
improved from 0.82s to 0.61s (-25.6%). Peak RSS increased by 0.9% and
libstepcore grew by 1.6%. Twenty full read/write runs remained at 4.9s.
The real file still created and validated 10,609 instances with zero
errors and warnings. Its 949,685-byte rewrite changed only by its
timestamp.
Add stable entity, type, and attribute IDs through SchemaModule. Make
late-bound compatibility names opt-in, table-drive rules and complex
metadata, and emit entity metadata directly into bounded chunks.
AP242e4 late-bound output changed as follows:
* generated files: 5,719 to 905 (-84.2%)
* generated lines: 254,220 to 123,552 (-51.4%)
* generated bytes: 13,942,523 to 9,804,680 (-29.7%)
* generation wall time: 2.74s to 0.56s (-79.6%)
* compile/link wall time: 157.68s to 37.02s (-76.5%)
* compile/link peak RSS: 593,320 KiB to 199,156 KiB (-66.4%)
* shared library size: 9,956,640 to 7,716,096 bytes (-22.5%)
* read/write wall time: 0.26s to 0.25s
ELF relocations increased from 28,843 to 39,719 due to descriptor-slot
tables. The packed schema image stage is intended to remove this cost.
The real AP242 sample created and validated 10,609 instances with zero
errors and warnings. Rewritten output matched except for its timestamp.
- API v2 table-driven schema, entity, type, attribute, and inheritance
initialization via include/clstepcore/ schemaInit.h.
- --late-bound/-T, producing descriptor-backed entities without
generated early-bound entity classes.
- CMake controls:
- SC_EXP2CXX_API_VERSION=2
- SC_EXP2CXX_LATE_BOUND=ON
- Runtime late-bound construction and owned attribute storage in
src/clstepcore/entityDescriptor.cc and src/clstepcore/STEPattribute.cc.
- Documentation and regression coverage, including incompatible option
handling.
AP242e4 generated-output results:
Mode Lines Bytes
━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━ ━━━━━━━━━━
Default v1 1,210,640 57.57 MB
──────────────────── ─────────── ──────────
API v2 early-bound 509,001 25.20 MB
──────────────────── ─────────── ──────────
API v2 late-bound 254,123 13.94 MB
The late-bound AP242 library compiled under C++11 and successfully
processed the BRL-CAD NIST sample: 10,609 instances
created and validated, with zero errors and warnings.
Opt in with either:
exp2cxx --api-version 2 schema.exp
or:
cmake -DSC_EXP2CXX_API_VERSION=2 ...
Version 1 remains the default, and the regression test verifies default
output is byte-identical to explicit v1
output.
API v2 now provides:
- Compact SELECT classes backed by shared descriptor-driven runtime code
instead of repeated generated parsing, assignment, and serialization
bodies.
- Generic Set* and typed getter APIs in
include/clstepcore/sdaiSelect.h:109.
- Dependency-bounded entity headers and source-owned dependencies in
src/exp2cxx/classes_entity.c:968.
- Out-of-line entity creators.
- Unity chunks that no longer include the all-entity header.
- C++11 output; C++17 did not offer enough additional benefit for this
tranche.
The command-line handling is in src/exp2cxx/fedex_main.c:85, while
compact SELECT generation begins in src/exp2cxx/selects.c:702.
Measured using the BRL-CAD ap242e4.exp, Release builds, eight parallel
jobs, and 64:8 unity chunks:
Metric API v1 Final API v2 Reduction
━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━ ━━━━━━━━━━━
Generated C++ lines 1,210,640 522,445 56.8%
──────────────────────── ─────────────── ────────────── ───────────
Generated bytes 57,566,051 27,202,644 52.7%
──────────────────────── ─────────────── ────────────── ───────────
Schema build wall time 183.22 s 88.31 s 51.8%
──────────────────────── ─────────────── ────────────── ───────────
Compiler CPU time 641.53 s 378.49 s 41.0%
──────────────────────── ─────────────── ────────────── ───────────
Peak RSS 1,035,112 KiB 916,412 KiB 11.5%
──────────────────────── ─────────────── ────────────── ───────────
Object-file bytes 70,472,952 40,031,088 43.2%
──────────────────────── ─────────────── ────────────── ───────────
Shared-library text 34,361,400 19,420,467 43.5%
──────────────────────── ─────────────── ────────────── ───────────
Generation wall time 1.52 s 0.69 s 54.6%
Handle nested SCOPE constructs and export lists while indexing Part 21
records, preserving scope-owned source records and their references.
Add scoped-record coverage.
Track indexed record bounds and expose the source-byte footprint of
materialized instances in cache statistics. Avoid promoting batch-owned
cache hits during reference resolution.
Add a lazy Part 21 reader and instance manager that index records
and resolve SDAI instances on demand. Include diagnostics, cache
management, reference indexes, and baseline coverage for lazy reads.
We want to be consistent with the previous layout, to avoid needlessly
breaking existing codes. The header update means client codes are not
required to include all of the stepcode library includes explicitly -
i.e., stepcode itself does not force this requirement with its own
internal includes. However, if client codes do not want to add the
library prefixes to their own includes as we have for stepcode, they
should still be able to add the various includes and work as before.
Adjust the include directory names accordingly.
I'm not sure if this logic was ever brought to a production level -
needs investigateion... maybe we should just arcive it if there isn't
any active development? Regardless, there appears to be no upstream
activity on the judy code, so just make it a part of cllazy directly.
Goal is for client codes to be able to simply include the stepcode
direcotry and have everything work, rather than having to worry about
individual step libraries. Will also make it clear with the dir
prefixes when libraries are using .h files from other libraries, rather
than their own.
Clifford Yapp