Move the decompression bomb detection in the TIFF reader into into
ImageInput::check_compression_ratio() so it can be used uniformly for
other input types as well.
Assisted-by: Claude Code / Claude Sonnet 5
Signed-off-by: Larry Gritz <lg@larrygritz.com>
A Softimage PIC file stores a chain of channel packets, each carrying
its own bit depth (8 or 16). The reader assumed one uniform per-channel
size for the whole image, so a file mixing depths (e.g. 16-bit R with
8-bit G) wrote past the end of the scanline buffer.
Fix: always expose one uniform pixel format, the widest depth among the
file's packets, and promote narrower channels to it on read via exact
bit replication (v*257 for 8->16), which preserves the normalized value
exactly. This keeps the pixel layout naturally aligned; representing
channels with their true differing formats instead would pack them at
non-uniform, sometimes misaligned byte offsets.
Assisted-by: Claude Code / claude-sonnet-5
Signed-off-by: Larry Gritz <lg@larrygritz.com>
read_layer_record computed a layer's width/height as `std::abs((int)right
- (int)left)`. The rectangle coordinates are signed 32-bit values (layers
can have negative offsets), but were stored in unsigned (uint32_t)
struct fields, and subtracting two ints near the extremes overflows --
undefined behavior that UBSan flagged on a fuzzed file (1615681375 -
-1605526113 cannot be represented in type 'int').
Retype Layer's and MaskData's top/left/bottom/right fields as int32_t,
matching what they actually hold, instead of reinterpreting bits at the
point of use. Widen to int64_t before subtracting so the computation
can't overflow; the absolute difference of two int32 values always fits
in uint32, and validate_resolution still rejects implausibly large
extents.
The mask data rect (used for layer masks) had the identical
std::abs((int)right - (int)left) overflow hazard in load_layer_channel
and gets the same fix.
Assisted-by: Claude Code / Sonnet 5
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Read10bitFilled() unpacks a scanline of 10-bit "filled" (Method A/B)
samples backwards, and for 1-channel images applies a workaround that
swaps the first and third datum of each group of 3 packed samples (so
column order comes out right). The swap didn't check that a full group
of 3 remained in bounds. Guard the swap so it only fires when a complete
group of 3 remains.
Add a regression test in testsuite/dpx.
Assisted-by: Claude Code / Sonnet 5
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Find OpenCV via its exported CMake config (PREFER_CONFIG) so OpenCV 5 is
located correctly.
Modify our imagebufalgo_opencv.h to work with OpenCV 5. But how to test
whether we did it correctly?
We never had an actual test for the OpenCV interop. So set one up.
Assisted-by: Claude Code / Sonnet 5
Signed-off-by: Larry Gritz <lg@larrygritz.com>
One small improvement in how decode_exif detects corruption, to ensure
that the initial header examination doesn't read past the array bounds
-- by starting off with a check that the exif blob's size is at least a
big as a TIFFHeader, otherwise parsing it is pointless.
Then the rest of the changes are finding places where readers call
decode_exif but ignore the results, effectively ignoring corrupt Exif
blobs. That is in fact the default intended behavior (if the rest of the
image file can be read properly), but we are supposed to be checking the
global "imageinput:strict" attribute and considering it a hard error
when in strict mode. There were several places where we didn't handle
that case, in the JPEG, PNG, WebP, and Heif readers.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Adds a hint "raw:bad_pixels" which passes a filename for bad pixel data
to libraw.
I haven't added any tests, can't think of a good way of testing this.
Open to suggestions.
This functionality has been tested in a client code.
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
read_native_tiles() on a tiled EXR whose dimensions aren't a multiple of
the tile size wrote past the caller's buffer when reading an edge tile
range: both readers used a row stride padded up to a whole number of
tiles (nxtiles*tile_width) instead of the caller's actual rectangle
width, overrunning the buffer whenever the last tile column is partial.
Fixed in both exrinput_c.cpp (Core reader) and exrinput.cpp (classic
reader) to use the requested rectangle width as the destination row
stride instead. exrinput.cpp additionally needs the pre-clamp width
specifically, since read_native_tile() forwards a tile-aligned xend past
the image edge for ordinary full-tile reads.
Adds testsuite/openexr-partialtile, a compiled regression test that
calls read_native_tiles() directly for both readers -- the only path
that exercises this code -- and checks pixels and buffer bounds.
Assisted-by: Claude Code / Claude Opus 4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
(The same problem and set of fixes were applied to both ImageInput and
ImageOutput. For brevity, we will only discuss ImageInput below.)
ImageInput error messages are stored in a thread_local map keyed by a
unique per-instance id (m_id). The destructor's erase of that entry was
commented out (over caution about the static destruction order fiasco),
so every ImageInput that accumulated an error and was destroyed without
its error being cleared via geterror() leaked one map entry. The id is
monotonically increasing, so entries never get reused and the map grows
without bound.
How did we find this? A fuzzer opening hundreds of thousands of corrupt
files -- each of which errors, and each ImageInput::create() attempt
tries several readers -- grew the map to tens of MB and eventually
OOMed. The leak was traced to this issue.
So we re-enable the erase, guarded against the destruction order fiasco:
the thread_local map is wrapped in a struct whose destructor clears an
'alive' flag, and the ImageInput destructor only touches the map while
it is alive. Each ImageInput removes only its own entry (keyed by its
unique id); other inputs' errors are untouched.
Cross-thread cleanup is still a known limitation -- errors accumulated
in a different thread than the one that descroys the ImageInput are
still leaked, noted in a TODO. However, this should be exceptionally
rare, since not only do you need the two-thread situation, but the one
that made the call that produced an error needs to not check for errors
and return/clear the message. So only badly behaving user code that
fails to check and retrieve errors should have this minor leak. And we
are in the process of adding compile-time warnings for not capturing the
error status values from our APIs.
Assisted-by: Claude Code / Claude Opus 4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
A fuzzed JPEG2000 caused a heap-buffer-overflow (caught by ASan) when
copy_scanline read past the end of openjpeg's decoded component buffer.
The file's image canvas was 32x32, but one of its three components had a
horizontal subsampling factor dx=249 (larger than the entire image).
OIIO derives the ImageSpec size from the union of each component's
x0+w*dx / y0+h*dy window, so that bogus component inflated the spec to
249x32. copy_scanline then iterated x over the inflated width while
indexing a component whose data array only held 1 column, walking off
the end. The per-sample guard was also broken: it compared the component
row index against reference-grid offsets (wrong units) and used `>`
instead of `>=` on the column.
Two fixes:
1. Detect the corruption at open(): the JPEG2000 canvas (x1-x0, y1-y0)
is the authoritative image size, and a component's subsampling factor
must be nonzero and no larger than the canvas. Reject files that violate
this instead of decoding a nonsensical, inflated image.
2. Harden copy_scanline by bounds-checking each access against the
actual component array geometry and skip the channel (zero-fill) when
comp.data is null or a subsampling factor is zero, which also closes a
latent divide-by-zero on comp.dx/comp.dy.
Assisted-by: Claude Code / Claude Opus 4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Previously, we did not have a PtexInput::valid_file() method, and our
open() method just directly called PtexTexture::open(), which is really
bad with validity checks and can crash or OOM with corrupted files.
But Ptex has a fixed 64 byte header (cheap to read and inspect) that has
lots of fields that let us more accurately check for reasonable file
validity. Therefore, we both implement valid_file, and also have open()
do these checks (rejecting many forms of corruption) before turning the
job over to the Ptex library itself.
Assisted-by: Claude Code / opus-4.8
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Change several build-time options from `option()` to our `set_option()`,
which mostly has the effect of allowing them to be overridden by
environment variables of the same name, which is really handy for CI
variants.
In the process, realized that we had some real problems with handling of
EMBEDPLUGINS=0, it probably hasn't worked properly for years and went
unnoticed because the one CI job variant that was trying to test
EMBEDPLUGINS=0 actually *was not* because it was trying to do it by
setting the environment variable, unaware that it could not be used to
set the option until this patch. So there is a bit of cmake and linkage
fixing that needed to be done to get it squared away.
Assisted-by: Claude Code / opus-4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The existing "limits:imagesize_MB" rejects files whose total
uncompressed size exceeds a cap, but it cannot catch an image that is
tiny in one dimension yet absurdly large in another (e.g. an image
claiming 32 x 50331680, which slips under the 32 GB default). Add a
companion global attribute, "limits:resolution", that caps the maximum
number of pixels along any single dimension.
Like imagesize_MB, this is enforced only in ImageInput::check_open(),
intended for validity checks when reading untrusted files.
Applications may set the attribute larger or smaller than this default,
depending on their need to deal with images larger than these limits,
and/or willingness to risk mistaking a corrupted or maliciously crafted
image for a legitimate one on this basis.
Now then, with this in place, we can clean up some of the values we used
in passing valid resolution ranges to check_open(). That was originally
meant to express the true limits of what was valid in the file formats,
but had in many cases been conflated with common sense "what value is so
big that it's probably bogus". Now we can disentangle these ideas -- the
value passed to check_open is the definitional limit of the file format
itself, and the "limits:resolution" is the overrideable global policy
for what we consider more likely to be malicious/corrupted than merely
legit big.
Assisted-by: Claude Code / Claude Opus 4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Validate channel-row byte availability before interleaving scanlines in
PSD reads, and fail gracefully with an error instead of risking
out-of-bounds access on malformed files.
Assisted-by: GitHub Copilot / GPT-5.3-Codex
Signed-off-by: Larry Gritz <lg@larrygritz.com>
- Add recently published security advisories to SECURITY.md
- Revision of security policy and definitions of vulnerability.
- Also some minor forward-ports and adjustments of CHANGES.
The substantive change is explaining that our policy is to only consider
vulnerabilities if they (a) allow remote code execution, privilege
escalation, and other very severe problems (not crashes, OOM, or most
"DoS" scenaries), and (b) are exploitable in practice by untrusted
actors merely by presenting malicious routine input such as image files
(misuse of APIs, for example, is not a vulnerability).
Everything else should be reported through the usual Issue/PR mechanism,
and not use the security vulnerability reporting regime, which is
massively more overhead for the maintainers.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
PRs #5260 and #5254 interact: #5254 exposed the default type arg on the
module getattribute (`"type"_a = TypeUnknown`), so a one-arg call now
returns None instead of raising TypeError. #5260's test still expected
the old pybind behavior. Update the test and ref to the intended new
behavior.
Assisted-by: Claude Code / Claude Opus 4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Exploring an idea Anton had about unifiying two python backends into
single source since they are so similar.
## Summary
This PR unifies the early nanobind migration with the existing pybind11
Python bindings by compiling **shared sources** from `src/python/` for
both backends, instead of maintaining parallel copies under
`src/python-nanobind/`.
Migrated modules so far: `ROI`, `TypeDesc`, `ImageSpec`, `ParamValue` /
`ParamValueList`, plus the core `OpenImageIO` module helpers in
`py_oiio.cpp`. Backend differences are isolated in `py_backend.h` (type
aliases, binding API macros like `.OIIO_PY_RW(...)`, and small
`oiio_py::` helpers). Shared Python↔C++ conversion logic lives in
`py_oiio.h` with `#if` only where the backends genuinely differ.
CMake is updated so that when `-DOIIO_PYTHON_BINDINGS_BACKEND=nanobind`
is selected, the module is built from the unified sources and installed
as the main `OpenImageIO` package in the expected location. With `both`,
pybind11 remains the primary module and nanobind builds as
`PyOpenImageIONanobind` for side-by-side testing.
Duplicate nanobind-only `.cpp` files are removed (~1.8k lines deleted
net). Testsuite coverage is extended with explicit roundtrip checks for
shared helper paths (attributes, buffers, tuples).
---
## Pros
- **Single source of truth** for migrated bindings — no more keeping two
copies of `py_roi.cpp`, `py_imagespec.cpp`, etc. in sync by hand.
- **Lower risk of backend drift** — a fix or API tweak in a migrated
module automatically applies to both pybind11 and nanobind.
- **Net code reduction** — removes a large duplicated tree under
`src/python-nanobind/` while centralizing shared logic in one place.
- **Clearer migration path** — `MIGRATION_STATUS.md` documents what is
fully migrated, partially migrated, and still pybind-only.
- **Better nanobind-only build layout** — on `main`, choosing nanobind
does not install into the same package layout as pybind11; this PR fixes
that for the migrated surface.
- **Side-by-side validation** — `OIIO_PYTHON_BINDINGS_BACKEND=both`
builds both modules so existing `testsuite/python-*` and `*.nanobind`
variants can be run against the same C++ sources.
- **Explicit parity tests** — new/extended tests cover shared conversion
paths (typed attributes, buffer→vector, tuple roundtrips) rather than
relying on import smoke tests alone.
- **Maintainable binding style** — method-name macros
(`.OIIO_PY_RW(...)`) keep clang-format happy without `clang-format off`
regions.
---
## Cons
- **Added abstraction in the pybind11 path** — even pybind-only builds
now go through `py_backend.h`, macros, and shared helpers; slightly more
indirection than plain `.def_readwrite(...)`.
- **Regression risk for pybind11** — adapting existing pybind code to
work for both backends could subtly change pybind11 behavior at the
edges (init, casting, error paths, ownership).
- **Incomplete migration** — most of the Python API (`ImageBuf`,
`ImageInput`/`ImageOutput`, `ImageBufAlgo`, texture system, etc.) is
still pybind-only; this PR is a foundation, not full nanobind coverage.
- **Known nanobind gaps remain** — module-level exports and helpers
listed in `MIGRATION_STATUS.md` are still missing or intentionally
different vs pybind11.
- **Higher review and test burden** — reviewers need to think about two
backends; CI/local testing should cover both pybind and `.nanobind`
variants for touched code.
- **Conditional compilation complexity** — `#if
defined(OIIO_PY_BACKEND_NANOBIND)` remains in a few places (e.g.
`ParamValue` constructors, buffer dispatch); easy to get wrong if
extended carelessly.
- **Slightly less obvious binding code** — `.OIIO_PY_RW("x", ...)` reads
less directly than `.def_readwrite("x", ...)` until you know the macro
map.
---------
Signed-off-by: Aleksandr Motsjonov <soswow@gmail.com>
Run check_open on JXL basic-info dimensions/channels before decode
output allocation to prevent oversized allocation paths on malformed
input.
Also update ref output for libjxl 0.12.0.
Assisted-by: GitHub Copilot / GPT-5.3-Codex
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Find corruptions by validating layer resolutions.
* Find corruptions when eof is hit in the middle of a string.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Truncating a corrupt uint32 length_tab entry into `int` could yield a
huge negative value, causing a runaway allocation and crash in
uncompress_rle_channel. Validate offset/length against the file size
instead.
Assisted-by: Claude Code / Sonnet 5
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This aligns FFmpeg/libswscale destination image buffers and validates
the buffer-size, frame-fill, and swscale return paths before decoded
pixels are consumed.
The change prevents narrow high-bit-depth VP9/GBR inputs from writing
past OIIO's destination buffer during swscale conversion, and adds a
regression case for the width-1 high-bit-depth FFmpeg path.
Signed-off-by: Dongju Lee <dongpago2@gmail.com>
This handles malformed GIF files whose logical canvas dimensions can
overflow legacy int-indexed canvas calculations.
The patch rejects oversized canvas dimensions before scanline/canvas
indexing and adds a regression test for the malformed oversized-canvas
case.
Signed-off-by: Dongju Lee <dongpago2@gmail.com>
This treats non-positive tiff:RowsPerStrip metadata as absent before
using it to align generic scanline read/write chunks. That avoids
feeding invalid chunking metadata into round_to_multiple while
preserving normal positive RowsPerStrip behavior.
The patch adds regression coverage for an OpenEXR file carrying
RowsPerStrip=0 through iinfo --hash, iinfo --stats, and an oiiotool
conversion path.
---------
Signed-off-by: Dongju Lee <dongpago2@gmail.com>
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Co-authored-by: Larry Gritz <lg@larrygritz.com>
GifSplitPalette() computed `numPixels * (splitElt - firstElt)` as a
32-bit int before dividing. For any image over roughly 16.9M pixels
(e.g. 4117x4117), that intermediate multiplication overflows INT_MAX and
is undefined behavior; in practice it produced a bogus negative/garbage
split count that could drive later indexing into `image` out of its
bounds.
Do the multiply in 64 bits and narrow back afterward -- the quotient is
always <= numPixels, so the narrowing itself is safe.
Assisted-by: Claude Code / Sonnet 5
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
TBHD width/height/channel fields were never cross-checked against how
much tile data the file actually contains, so a corrupt header could
claim an arbitrarily large image (e.g. 16k x 16k) backed by only a few
hundred bytes of real data. readimg() would then resize() a
multi-gigabyte buffer for a tiny malformed file, hanging (or very slowly
succeeding) instead of failing -- a resource-exhaustion DoS on untrusted
input, found via fuzzing.
Bound the claimed image size against how much tile data remains in the
file, generously allowing for RLE's worst-case 128x expansion ratio, and
fail fast if it's implausible.
Assisted-by: Claude Code / claude-sonnet-5
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
CineonInput::open() accepted any per-channel bit depth in [1,32], but
libcineon's ComponentDataSize()/ComponentByteCount() only recognize
{8,10,12,16,32,64} and assert(0) on anything else. A crafted file with
an unsupported bit depth (e.g. 26) passed the OIIO-side check and then
aborted deep inside the vendored library instead of failing cleanly.
Tighten the check to libcineon's actual whitelist so bogus bit depths
are rejected with a proper errorfmt() before ever reaching the vendored
code. Adds a regression fixture (broken_bitdepth2.cin) to the cineon
testsuite.
Assisted-by: Claude Code / Sonnet 5
Signed-off-by: Larry Gritz <lg@larrygritz.com>
open_raw set imgdata.rawparams.max_raw_memory_mb *after* calling
m_processor->unpack(), so LibRaw's allocation cap never protected the
unpack step -- which is exactly where LibRaw allocates the raw image
buffer. A corrupt raw file claiming huge dimensions could therefore
allocate unbounded and OOM (showing up on the fuzzer as a 0-length crash
artifact from an out-of-memory trip rather than a normal reproducer).
Move the max_raw_memory_mb assignment to immediately after open_file
succeeds, before unpack(), so the cap (default 2048 MB) actually bounds
the allocation. LibRaw then fails the file cleanly instead of OOMing.
Assisted-by: Claude Code / Claude Opus 4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
While working on
https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5254,
where we're introducing changes that could surface bugs in pybind11 code
that's in production today, I figured I'd try adding some tests.
This was a fairly loose-handed effort. To be upfront: most of it was
LLM-generated. I did some research into which modules were lacking
coverage the most, and those are what this PR targets. I haven't gone
through every line of the generated code.
I hate being asked to review something that was clearly vibe-coded with
no real effort put in, so I'm saying that here rather than pretending
otherwise. I can try to review it properly, but without deep codebase
knowledge it'll be slow and painful. Since these are just tests, my
thinking was: even if there are silly bits here and there, it shouldn't
hurt much, as long as things go green and we cover more logic and
branches. Worth offering as-is.
---------
Signed-off-by: Aleksandr Motsjonov <soswow@gmail.com>
We had missed a case where bit_convert needed outputs to go to the
scratch buffer: PHOTOMETRIC_SEPARATED, but not requesting raw color.
(Because the real number of channels differ from what we are presenting
to the caller.)
Assisted-by: Claude Code / Sonnet 5
Signed-off-by: Larry Gritz <lg@larrygritz.com>
10MB was conservatively chosen as big enough that anything larger is
definitely bogus for an ICC, Exif, or XMP block in a PSD file.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
read_native_scanlines declared task_set before the locals its queued
decompression lambdas capture (ok, compressed_scratch). On the
leftover-scanline early-return path those locals were destroyed before
~task_set() waited for the queued work, so workers could write through a
dangling ok and hand a freed compressed buffer to zlib.
Declare task_set after the captured locals so its destructor wait runs
while they are still alive, and propagate async decompression failures
via return ok instead of unconditionally returning true.
Assisted-by: Claude Code / opus-4.8
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
validate_header() let the oiio:RawColor/psd:RawData path return true
before the switch that rejects unknown m_header.color_mode values, so an
out-of-range or "hole" (undocumented) color mode reached setup(), which
unconditionally indexes the fixed 10-entry mode_channel_count /
mode_channel_names tables with the attacker-controlled value -- a global
out-of-bounds read (and, per external report, a path to unbounded/bogus
allocation) reachable via the public RawColor config attribute on a
crafted PSD.
Move the "is this a color mode we know about at all" check ahead of the
RawColor early return so it always runs, and keep the existing
Duotone/Lab-unsupported-without-raw check as a second pass after the
early return, preserving current behavior for every valid mode.
Assisted-by: Claude Code / claude-sonnet-5
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
An 18-byte TGA file consisting of just a header (no pixel data) can
claim an arbitrary width/height/bpp -- e.g. a 1472x12000 24bpp image --
and readimg() would allocate the full ~53MB buffer and only discover the
truncation once the first read hits EOF. Worse, the resulting
partially-formed ImageSpec could still propagate to a downstream writer
before the read failure was acted on.
Cross-check the header's width/height/bpp against how many bytes of
pixel data the file could plausibly contain before committing to the
allocation, mirroring the existing pattern in iff.imageio and
jpeg2000.imageio. The check compares against the *on-disk* pixel size
(palette/16-bit storage is smaller per pixel than the decoded, unpacked
ImageSpec) and allows up to 128x expansion for RLE-compressed images (a
1-byte run-length count can encode up to 128 output pixels).
Updated testsuite/targa/ref/out.txt: several existing corrupt-file
regression cases (crash1.tga, crash6.tga, crash3952.tga) are now caught
earlier with clearer messages, and two real oiio-images TGA files
(UCM8.TGA, UTC16.TGA) that use palette/16-bit storage smaller than their
decoded size continue to read correctly.
Assisted-by: Claude Code / Sonnet 5
Signed-off-by: Larry Gritz <lg@larrygritz.com>
PngReadCallback passed input->geterror(false).c_str() to
png_chunk_error() when a PNG read failed. png_chunk_error() does not
return -- it longjmp's back to libpng's setjmp point, which skips C++
destructors, so the temporary std::string returned by geterror() was
never freed. The fuzzer surfaced this as a small, intermittent leak (a
few KB across hundreds of thousands of runs).
Copy the message into a plain stack buffer (no destructor) before
calling png_chunk_error, so nothing heap-allocated is alive across the
longjmp.
Assisted-by: Claude Code / Claude Opus 4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The mask-based uncompressed decode path copies m_Bpp bytes into a
uint32_t pixel variable. m_Bpp is validated up to 16 (for 128-bit DXGI
formats), but the legacy fmt.masks[] are uint32_t and can only address
32 bits. A corrupt file with m_Bpp > 4 caused a stack-buffer-overflow.
Clamp the copy to sizeof(pixel) = 4 bytes, which is all the masks can
reference anyway.
Assisted-by: Claude Code / claude-sonnet-4-6
Signed-off-by: Larry Gritz <lg@larrygritz.com>
A DDS file with a garbage pixel-format bpp field (e.g. fmt.bpp ~2.5e9)
yields a native bytes-per-pixel (m_Bpp) of hundreds of millions.
internal_readimg() then does `new uint8_t[width * m_Bpp]`, a multi-GB
allocation that OOMs. The core resolution/imagesize limits do not catch
this: they only see the small output ImageSpec, not the native pixel
size, and the existing fmt.bpp check is gated on RGB/LUM/YUV/ALPHA flags
that a corrupt header can simply omit.
Validate m_Bpp for uncompressed formats right after it is computed in
open(): the largest real uncompressed DDS/DXGI pixel is 128-bit (16
bytes), so anything outside [1,16] is a corrupt header and is rejected
before any allocation. The check is confined to uncompressed formats
because m_Bpp is neither set nor used for compressed (block) formats,
which size their data via GetStorageRequirements.
Assisted-by: Claude Code / Claude Opus 4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Detect and reject corrupt chunk sizes, flags, and channel configurations
in the header.
Assisted-by: Claude Code / Claude Opus 4.8
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Somehow the check_open validation was only in exrinput.cpp (using the
old C++ AP), but not exrinput_c.cpp (using the C "Core API").
Also adjust the allowable z resolution to reflect that OpenEXR doesn't
support volumetric images. (This was obviously a cut and paste error
long ago.)
Signed-off-by: Larry Gritz <lg@larrygritz.com>
read_offset_tables() sized the RLE start/length tables from the raw
header (`ysize * zsize`), but read_native_scanline() indexes them by the
spec dimensions (`height * nchannels`, derived from the image's
dimension field). A corrupt header (e.g. dimension=1 with zsize=0) makes
`ysize*zsize` = 0 so the tables are empty, while indexing still uses
`height*nchannels` = 1 -- a null-pointer dereference and crash.
Size the tables by `m_spec.height * m_spec.nchannels`, the same quantity
used to index them, so the two can never desync. For well-formed files
this equals ysize*zsize, so behavior is unchanged; a corrupt file now
fails cleanly when the (too-short) table read hits EOF. Also guard
against non-positive table sizes, use vector::data() instead of `&v[0]`
(UB on empty), and fix an off-by-one in the scanline bounds check (y ==
height would index start_tab[-1]).
Add the fuzzer reproducer as testsuite/sgi/src/broken02.sgi.
Assisted-by: Claude Code / Claude Opus 4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
`merge_deep_pixels()` calls `copy_deep_sample()`, which has an error
status return, so change `merge_deep_pixels()`'s return type from void
to bool.
For OIIO_NODISCARD_ERROR, all internal calls that previously ignored
return values are handled:
- OIIO_CONTRACT_ASSERT where the call can't fail in context, or the
caller is a void function or a constructor.
- `return false` where the caller already returns a bool error state.
- `(void)` cast for the deprecated internal overload.
Follow-up to #5252
Part of #4781
Signed-off-by: Luna Kim <177369799+luna-y-kim@users.noreply.github.com>
Upgrade the embedded/vendored pugixml to 1.16, also the default version
that we use for CI with build_pugixml.bash.
Raise PugiXML minimum to from 1.8 to 1.11 (late 2020, over 5 years ago).
Rely completely on pugixml exported cmake config, eliminate our
Findpugixml.cmake. This is possible because we raised our minimum
pugixml to 1.11, which is the point where its exported cmake configs
were reliable.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Add scripts for openjph auto-build when not found (and
OpenImageIO_BUILD_MISSING_DEPS specifies it, or "all"). Build openjph
0.30.1 when missing.
Since we can build it when missing, take away the various missing
openjph allowances from our CI; auto-build when needed for all variants.
More thorough exception catching for openjph in our jpeg2000input.cpp.
When I compile openjph with clang (including icx), I get some runtime
exceptions. Not when I build it with gcc! I don't have the time or
inclination to investigate why, so for our CI, rig it to build openjph
with gcc even when I'm building OIIO with clang/icx.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Circular IFD chain loop detection (identified by for CVE-2022-3970) was
fixed in libtiff by MR 386
(https://gitlab.com/libtiff/libtiff/-/merge_requests/386), released in
libtiff 4.5.0.
Raise the recommended minimum from 4.2 to 4.5 so users on older
installations get a build-time advisory. The hard minimum stays at 4.0
(for now) to preserve compatibility.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
IOMemReader::pread guarded its memcpy with `size + size_t(offset) >
m_buf.size()`. For a large or negative offset, that addition wraps
around size_t and can compare as in-bounds, skipping both the error path
and the size clamp, so the memcpy reads past (or before) the buffer.
Assisted-by: Claude Code / Claude Opus 4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
When creating an ImageInput, if the initial guess (based on the file
extension) fails to open, we try all the others in turn.
This tries to make that a little cheaper by not doing a full open() but
instead first checking for valid_file, which in theory is a much less
expensive approximate check.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
read_native_scanline computed the scanline's file position as
m_bmp_header.offset + y * m_padded_scanline_size and seeked there
without checking it against the file size. A corrupt header (bogus
pixel-data offset or scanline size) can place that position far outside
the file.
Assisted-by: Claude Code / Claude Opus 4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
- Detect corrupted userbuf size.
- Protection against integer multiply overflow in various places in the
DPX reading code.
- Also fix memory load alignment issue. Didn't seem to be a problem in
practice, but the sanitizers weren't happy on some platforms.
Assisted-by: Claude Code / sonnet-4.6
Signed-off-by: Larry Gritz <lg@larrygritz.com>
IBA::make_texture() only passes the full metadata to the first level of
a MIPmap, and that might include output behavior hints, too! Make this
hint "sticky" in the TIFF writer, so if any subimage's first level's
spec enables half output, all the MIP levels of that subimage will allow
it.
Fixes#5239
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Carelessly written hdr/rgbe files, or correctly written ones that are
checked out by git on windows if you're not careful, might use Windows
CRLF line endings. Be silently tolerant of that.
Assisted-by: Claude Code / opus-4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We were allocating a temporary buffer big enough for the RGBA but
neglecting the z. Only for the 16 bit case.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Guard against corrupted images, which if they said they had more than
the 8 channels allowed by Cineon format, would overrun array bounds
while reading the file. And check for valid bit depths.
Also opportunistically fix a bug I spotted where a red channel was
incrementing the count of greyscale channels rather than the count of
red channels -- which must have been a copy and paste error from
literally 15 years ago. (But those counters are only used for channel
naming, and only if there is more than one greyscale or red channel in
the file. So I doubt anybody has ever hit this bug, or ever would have
in the future.)
Fix possible memory leak by having init() call `m_cin.SetInStream()`
with a null handle to clear out the internals.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
When copy-constructing an ImageCache-backed ImageBuf, the "cache-based
or deep" branch set `m_bufspan = {}`, a default-constructed image_span
whose strides are all zero. retile() addresses a pixel within a cache
tile as `offset = pixelindex * m_bufspan.xstride()`, so a zero xstride
made every pixel resolve to tile offset 0 -- the whole copied image read
back as a constant color (the value of pixel 0,0).
Fixes#5243
Added a regression test (test_copy_of_imagecache_backed) that copies a
cache-backed buffer with distinct per-pixel values and verifies the copy
does not read back a constant.
Assisted-by: Claude Code / Claude Opus 4.8
Signed-off-by: Larry Gritz <lg@larrygritz.com>
To avoid breaking ABI, the old int overload is kept, but its declaration
is hidden behind `#ifdef OIIO_INTERNAL`. Downstream code only sees the
int64_t version, but old binaries still work. The forwarding wrapper
handles it until the next first-digit version changes.
Also, the Python binding is adjusted using a wrapper to resolve the
overload ambiguity. While adding the wrapper, I noticed other bindings
still took `int pixel`, so widened them to `int64_t` for consistency.
Follow-up to #2363Fixes#5242
Signed-off-by: Luna Kim <177369799+luna-y-kim@users.noreply.github.com>
- Fix sort(), incorrectly using the front Z channel in a place where it should have instead used Zback channel.
- Widen pixel indices from int to int64_t (internal only, follow-up to #2363).
- Cleanup unreachable branches in merge_overlaps().
Modified the `DeepData::sort` test to include samples with a tied front
Z, so it verifies the Zback fallback.
Signed-off-by: Luna Kim <177369799+luna-y-kim@users.noreply.github.com>
FITS files are allowed to have multiple "header blocks".
Unfortunately, as originally written, our fits input used *recursion* to
read them. If there is a file with too many header blocks, it can crash
with a stack overflow.
Change to use a loop instead of recursion. And put a maximum iteration
limit of 10000 headers in one file before giving up.
(Note to reviewers: the diff looks bigger than it really is. By
surrounding the contents of read_fits_header with a `for` loop, it
reindented it, and that is the vast majority of the change. I wish
GitHub review diffs more clearly visualized which changed lines are only
whitespace changes.)
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Bump the "bleeding edge" test to C++26 (from 23), and the "latest
releases" Linux test to C++23 (from 20). There are still other variants
that test C++20 including anything based on the VFX2026 platform.
Just one tiny change needed to compile in C++26 mode, related to a
deprecated std type_traits template.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The InterpretingDeepPixels reference link expired. It moved to an
HTML page on the OpenEXR docs site, so update the comments to the
working URL.
Signed-off-by: Luna Kim <177369799+luna-y-kim@users.noreply.github.com>
This suppresses the warnings in the fmt library code which have started
appearing recently when building on a Mac.
I suspect the warnings were triggered by the clang toolchain being
updated to v21, but to be fair I haven't tried to downgrade to confirm
that.
No extra tests. The changes allow OIIO to build with "exit on warning"
on Mac with the latest build tools, whereas without the changes the
build fails.
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
python_tests was from 2010 and either was checked in inadvertently, or
has long, long been supplanted by our real testsuite feature.
The various TRAVIS_SKIP_LSAN file used to mark certain tests as
skippable when running the leak sanitizer... but we long ago removed the
code in runtest.py that recognized its presence and took action. So it
has served no purpose for quite some time.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
OpenImageIO uses the fmt library, but spdlog is also quite common and
uses the fmt library. This fix should allow to use precompiled fmt
library in both spdlog and oiio if they are used in same project.
Added `OIIO_USE_COMPILED_FMT` CMake option to allow linking against a
compiled fmt (fmt::fmt) instead of the header-only variant.
Update externalpackages.cmake to expose the option, select the
appropriate fmt target and include dir, and pass the value into
generated headers via oiioversion.h.
Adjust Config.cmake.in to avoid forcing find_dependency(fmt) when
header-only behavior is overridden. Modify detail/fmt.h to skip defining
`FMT_HEADER_ONLY `when compiled fmt is used, and update libutil
CMakeLists and ustring.cpp to link/use the compiled fmt target and use
fmt::runtime for dynamic format strings.
Assisted-by: Codex 5.5 ExtraHigh
---------
Signed-off-by: Vlad <shaamaan@gmail.com>
Also make our CONTRIBUTING guide link to, and be consistent with,
our SECURITY.md policy file.
Follow OpenEXR's lead by explaining briefly that we only consider
something to be a security issue if it can plausibly be triggered by
ordinary inputs coming from untrusted parties. For example, a
maliciously crafted input image may be a vulnerability, but incorrect
use of an API call is not, because only trusted parties should have
direct access to the APIs.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Some minor fixes and inconsistencies.
- Per the ASWF [repo best practices
guide](https://tac.aswf.io/project_best_practices_guides/repo.html), the
project's technical charter should NOT be redundantly stored in our
repo, but instead we should just link to the canonical copy that the
ASWF maintains.
- Fix incorrect relaive path to CONTRIBUTING.md.
- Fix leftover "interim" designations from TSC members.
- Remove Lewis Wakeland from the TSC roster (he retired some months
back, but somehow we neglected to remove him from the file)
- Add a link to the Google doc with meeting notes.
- Remove stray reference to a 2024 release.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
According to C++26, some newly required std container hardening for
std::span and std::string_view were not reflected in our own analogous
types. In particular, some more precondition checking was required for
span::subspan, first, last, and string_view::remove_prefix and
remove_suffix in order to match the expectations of true parity with
with the std versions.
This is important because some day, we will probably discard our own
string_view and span for the std ones (especially when we are using
compilers in which they are fully hardened).
I found a couple spots where we used remove_prefix and remove_suffix
that assumed the range would be clamped, as it is in our version, rather
than contractually require them to be valid ranges as in std. This would
eventually fail if replaced by std::string_view, so fixing now.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Bump the "bleeding edge" variant to gcc-16, and the "latest releases"
variant to gcc-15.
One minor fix needed to make gcc16 happy -- it is better than earlier
versions at discovering variables whose values are not used.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Mostly for the sake of bots (but also humans) reading plugin.h and
trying to figure out exactly what each function does, just plainly state
which Unix-like dlopen/dlsym/etc function it is the equivalent of.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
It was throwing things off to set CMAKE_CXX_COMPILER in this way,
wrecking the ability to just run as a utility from the command line
without it being set. It was not necessary, as far as I can tell.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Adds agent configuration supporting Claude Code, Cursor, GitHub
Copilot, OpenAI Codex, and Opencode. A single AGENTS.md and contents
of .agents subdirectory serve as the canonical source for
instructions, skills, and other things that can be used across
multiple tools.
A script can be invoked by users to do the local setup of links or
references to allow them to use a particular tool:
`.agents/setup-agent TOOL` Currently supported tools by this script
include Claude Code, Codex, Cursor, GitHub Copilot, and OpenCode.
This initial commit includes two skills that I wrote and have used for
the last couple monthly releases: release-notes-update and
prepare-patch-release. I assume that we will add more over time as we
discover tasks that we want to make standard skill automation for.
Assisted-by: Claude Code / claude-sonnet-4-6
The AGENTS.md, Claude skills, and general scheme are entirely from me.
I used Claude to help me establish the setup script for the other
tools I don't happen to use.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes indexed-color PSD transparency handling when the transparent
palette index is `0`.
Before this change, the PSD reader used `m_transparency_index` both as the
palette index value and as the signal that a transparency-index resource was
present. That made index `0` ambiguous: it is a valid transparent palette index,
but it evaluated as false when deciding whether to add an alpha channel.
As a result, an indexed PSD with transparency index `0` could expose a
3-channel output spec, then later `indexed_to_rgb()` would treat
`m_transparency_index >= 0` as meaning transparency exists and write 4
bytes per pixel. That could write a 4-channel row into a caller buffer sized for
3 channels.
The fix separates those two pieces of state:
- `m_has_transparency_index` records whether the transparency resource exists.
- `m_transparency_index` stores only the numeric palette index.
- Index `0` now correctly adds an alpha channel.
- Transparency indexes outside the valid palette range `0..255` are rejected.
- `indexed_to_rgb()` now checks source and destination spans before writing.
### Tests
Added regression coverage to `testsuite/psd-colormodes`.
The test now generates indexed PSD variants from the existing
`pattern2-8-indexed.psd` fixture with transparency indexes:
- `0`: valid, should report/read as 4-channel PSD.
- `255`: valid, should report/read as 4-channel PSD.
- `256`: invalid, should be rejected.
Validated locally with:
```bash
python3 -m py_compile testsuite/psd-colormodes/run.py \
testsuite/psd-colormodes/src/make-indexed-transparency-psds.py
ninja -C build-psd oiiotool iconvert idiff
```
Also ran the generated PSD cases and a manual full psd-colormodes test;
final out.txt matched the updated reference
Assisted-by: Codex GPT5.5xHigh
---------
Signed-off-by: Vlad (Kuzmin) Erium <libalias@gmail.com>
Fixes#5207
Sometimes the `WxH+X+Y` and `+X+Y` notation (borrowed from X11!) can be
a little awkward. This patch extends the accepted syntax to allow commas
to separate the offset components, for example also accepting `WxY,X,Y`
and `X,Y`.
This affects the commands
--create
--crop
--cut
--fit
--fullsize
--origin
--originoffset
--paste
--pattern
--printstats
--resize
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This patch optionally frees the contents of the ustring table as the
process is exiting. It's triggered one of two ways:
- `OIIO::attribute("ustring:cleanup", 1);`
- Environment variable `OIIO_USTRING_CLEANUP=1`
If neither is used, the teardown won't happen, thus avoiding any
pointless work when it should be exiting the process.
The purpose of this is that when using certain debugging tools, the
ustring table (which only grows and never frees) looks like a memory
leak, even though it's by design. Someimes, it's useful to make it not
appear ot leak, and you are happy for a large table to spend time
pointlessly freeing all the data.
Fixes#3913
Signed-off-by: Larry Gritz <lg@larrygritz.com>
In main (working toward the late-2026/2027 releases) these 2022
containers are both outside the promised 3 year support window, and also
are using CentOS 7-based containers that are essentially irrelevant now
that all the studios have moved to RHEL9 (and Rocky/Alma) compatible OS
distros. So there is really no need to continue CI testing on them for
branches aimed at the future.
Note that we still have Ubuntu-based "oldest gcc" and "oldest clang"
jobs, so dropping this does not actually impact our ability to test
against the oldest gcc, C++, and python that we claim to support.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
New release of libheif, which continues to live up to its habit of
changing pixel values ever so slightly with each release.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Background: this flag was originally added in commit c5494ac06
(2015-11-24) with the message "...now builds linux+gcc...". It was later
refactored in commit 62d7164d3 (2017-01-23). From investigation, it
seems to be preventative rather than reactive because it does not
actively suppress warnings, but `-Werror` is enabled under the condition
NOT MSVC + STOP_ON_WARNING (originally in CMakeLists.txt, now in
compiler.cmake:126).
Compile tests without `-Wno-unused-result` but with `-Wunused-result`
explicitly enabled were conducted on the following commits, and all
compiled successfully without unused result warnings. Also, tests
confirmed that the warning triggers if there is an unused return value.
- 312793af4 (2026-01-01)
- 682825f9d (2025-01-01)
- 48362b26c (2024-01-01)
- 3b8807df8 (2023-01-01)
- 62d7164d3 (2017-01-23): refactored
- c5494ac06 (2015-11-24): originally added
Decision: the flag can be safely removed. Removing it allows gcc to
catch unused return values, which is required for the `OIIO_NODISCARD`
and `OIIO_NODISCARD_ERROR` macros to work as intended.
Related: #5196
Assisted-by: Claude / Opus 4.7
Signed-off-by: Luna Kim <177369799+luna-y-kim@users.noreply.github.com>
This message is triggered by the check that the tag of an auto-build
dependency matches the SHA hash we use as a cross-check. This patch is
just amending the error message to reveal the way to override the check,
so people (like me) don't have to go hunting for it if they are
purposely trying to build from a nonstandard commit of a dependency.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I recently discovered that OCIO's built-in configs' "file rules" that
come into play when using getColorSpaceFromFilepath actually do not
incorporate the search for color space name embedded in the filename
into the logic, and moreover, assume that files ending in ".exr" are
ACES2065-1, of all things. This seems an unfortunate choice because in
the real world, almost no exr files people will encounter actually use
the ap0 primaries.
This PR does the following to shore up this and related edge cases:
* For OCIO built-in configs, fully replace the existing file rules with
a blank slate that just uses the path parsing rule. Everything else is
categorized as "unknown". (Though many of our format readers already do
something better, setting the metadata if they actually know what the
file is.)
* Recognize "unknown" as equivalent to having an empty color space, an
error if you pass such a color space name to IBA color transform
functions.
* For all IBA color conversion, if the input space is not color managed,
be sure NOT TO mark the output color space as the original `to` request!
Input raw means output is also raw.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This is an emergency fix on top of #5201.
PR #5201 passed CI completely. Then I merged it, and now it fails. How
did that happen?
OOOOOOH. The imagespeed_test.cpp is not compiled at all for CI! Only for
local builds. So a build break in it will not be caught in CI. OK,
fixing that, too. It doesn't run in CI, it's not that kind of test, it's
a special benchmarking test meant to run locally and needs to be told a
specific image file to test with. But at least build it for CI so we
can't inadvertently break it again.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
- Annotate all ImageOutput write methods (write_image, write_scanline,
write_scanlines, write_tile, write_tiles, write_rectangle,
write_deep_scanlines, write_deep_tiles, write_deep_image), as well as
open, close, and copy_image, with OIIO_NODISCARD_ERROR.
- Also fix the handful of internal callsites (imagebuf.cpp,
maketexture.cpp, py_imagebuf.cpp, iconvert.cpp, imagebufalgo_test.cpp,
and imagespeed_test.cpp) that were previously ignoring the return value
of close(), either by propagating the error or by casting to void where
the error path is already being handled.
- Reference: #4781
### Tests
- tested against unit_imageinout, unit_imagebuf, unit_imagebufalgo
Assisted-by: Claude Sonnet 4.6
---------
Signed-off-by: rose413 <116857309+rose413@users.noreply.github.com>
##### PNM
The PNM reader did not provide an optimized `valid_file` implementation.
While `PNMInput::open` would perform basic validation of the PNM header,
it would only do this after having read the entire contents of the file
into memory. Besides the obvious risks/issues with having `open`
unconditionally read arbitrarily large files into memory, this also
meant that any call to `valid_file` would necessarily do the same (as
the default implementation delegates to `open` when ioproxy support is
present).
To fix these issues, the following changes were made:
* Provide an efficient implementation of `valid_file` for `PNMInput`
that only loads the header, and then validates magic number/dimensions.
* In `PNMInput::open`, first load only the header to memory, and
subsequently, only load the rest of the image data if the header is
parsed successfully, and the data contained within is valid.
* Added a rough limit to header read size of 1KiB.
* Added a rough limit to full file read size of 1GiB.
Note that, for now, if the file size exceeds the 1GiB limit, the read is
simply truncated to 1GiB, rather than failing altogether.
##### JPEGXL
The JXL reader already provided an efficient `valid_file` override,
implemented by validating the 128 byte JXL signature. The signature,
however, was not being validated in `JxlInput::open` before attempting
to decode the file.
The process of JXL decoding appears to read the entire contents of the
file to memory (at least for some non-JXL inputs). Taken in combination,
this means that when `JXLInput::open` was called on arbitrarily large
non-JXL inputs, the entire file was being read into memory before `open`
would fail.
As a simple fix for this, `JxlInput::open` now checks the result of
`valid_file` before attempting any decoding.
### Tests
I did not add a new testsuite case for this, but did test on my end that
for both PNM and JPEGXL, large invalid files are no longer being read
into memory, while valid images (of their respective types) are still
read/opened successfully.
---------
Signed-off-by: maxwelliverson <maxwelliverson@gmail.com>
Adds `OIIO_NODISCARD` and `OIIO_NODISCARD_ERROR` to `color.h`.
Deprecated functions are left unannotated.
Part of #4781
Assisted-by: Claude / Opus 4.7
---------
Signed-off-by: Luna Kim <177369799+luna-y-kim@users.noreply.github.com>
Our CI job that builds against VFX Platform 2023 via docker container
aswf/ci-osl:2023-clang15 broke after some dependency changes in a recent
aswf docker container update. It's not worth trying to "fix" an old
year, just lock down to the previous version of the container that is
known to work.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This PR adds a `--nchannels` flag to oiiotool, providing parity with the
existing flag in maketx.
### Problem:
When processing images from varied sources in automated pipelines, using
name-based channel selection (e.g., --ch R,G,B) is fragile. If an input
image uses non-standard names (e.g., X,Y,Z or 1,2,3), the command may
output black channels or issue warnings that are easily missed in large
logs.
### Solution:
While you can solve this by specifiying the indices of the channnels,
e.g --ch 0,1,2, the `--nchannels` flag offers a more intuitive and
robust "shortcut" to force an image to a specific channel count. maketx
has had one for years and I found that it works really well and often
wished oiiotool would have one too.
### Implementation:
The implementation acts as a wrapper around action_channels,
programmatically generating the index string. It uses ustring to intern
the generated string, ensuring memory safety even if the action
execution is deferred/postponed.
Assisted-by: Google Gemini (advice on ustring interning and
architectural structure).
### Tests
I added some tests to showcase what this is trying to solve
Signed-off-by: Danny Greenstein <danny.gr@gmail.com>
Background: OCIO 1.x configs were LUT based and so tended to have
different color space names for different data types/bit depths. We
capitalized on this to let `oiiotool --autocc` also select type/bits,
but this trick doesn't work so well for OCIO 2.x configs with procedural
(non-LUT) transformations that all look like "float". We don't want to
promote to float just because of this!
Solution: When using color space data type (nearly always float for new
configs and certainly the built-in configs) to infer output, DO NOT use
"float" as a real request. Treat it as not being a specific type, don't
let it override the intended output type for --autocc. (But a color
space still marked as a particular integer bit depth is still an
--autocc inferred request.)
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Add `ColorConfig::isData(name)` method to C++ and Python APIs.
* Fix `ColorConfig.isColorSpaceLinear()` that was missing from the
Python bindings. (Tests added as well.)
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
In PR #5164, we changed to not be quite as aggressive in treating OCIO
exceptions as real errors. All still good with that, but sometimes it is
helpful to know exactly what OCIO wants to tell us, for debugging.
In this PR, we allow our color debug mode (triggered when environment
variable OIIO_DEBUG_COLOR is set to nonzero) to print debugging messages
that reveal the exception messages. It's a debugging aid for developers,
no users will want this. In ordinary operations, we really do want to
just ignore those exceptions. But if something is misbehaving and we
don't know why, it may be helpful to know which of the exceptions is
being caught and what the message was.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We test on 3 Windows runner configurations currently -- Windows 2022
with VS2022, Windows 2025 with VS2022, and Windows 2025 with VS2026.
GitHub says the middle one will go away imminently (actually, it won't
"break", but it will redirect to Windows 2025 with VS2026, which we
already test). So take it out of our lineup, it's just waste.
The one thing the middle one did differently than the others is test
nanobind, so just move that to the VS2026 one.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
iconvert was the very first command line utility for OIIO, and its
syntax was just `iconvert [options] infile outfile`. YEARS later,
oiiotool came along, superceding iconvert with 100x the functionality,
and also differed by requiring `-o` to specify output file.
iconvert still exists for testing and examples -- iconvert uses just
plain old ImageInput + ImageOutput straightforwardly, whereas oiiotool
is a very complex app that's ImageBuf powered. It's often useful for
debugging to test the ImageInput/ImageOutput layer without any extra
complication, or to directly see how going through an ImageBuf might
differ.
The fact that oiiotool requires `-o` to specify an output file, whereas
iconvert considers that an error, is a minor annoyance when switching
back and forth while debugging, and it's really bugging me.
So here it is, a tiny patch to let iconvert silently accept -o, or not,
and do the obviously right thing. A little quality of life gift to
myself.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Preserve RGB of transparent color index pixels in GIF files.
When reading GIF files, pixels of transparent color index were skipped.
This left their RGB channels zeroed, discarding possible valid colormap
data. To preserve the RGB values, we change to always write the
colormap's RGB and set alpha conditionally.
According to [GIF spec](https://www.w3.org/Graphics/GIF/spec-gif89a.txt):
>viii) Transparency Index - The Transparency Index is such that when
encountered, the corresponding pixel of the display device is not
modified and processing goes on to the next pixel. The index is present
if and only if the Transparency Flag is set to 1.
So basically, it just defines a display compositing behavior but says
nothing about the color table entry for that index being invalid or
empty. In practice, encoders may still assign meaningful RGB values to
the transparent index, and those values are perfectly well-defined in
the colormap. For example, GIMP reads the valid RGB values of
transparent pixels (see attachment below).
### Tests
A new test image and test case is added:
https://github.com/AcademySoftwareFoundation/OpenImageIO-images/pull/11
<img width="847" height="695" alt="image"
src="https://github.com/user-attachments/assets/484c8b53-5a1b-4732-b95c-55a2f52021b5"
/>
Signed-off-by: Lumina Wang <lumina.wang@autodesk.com>
We used new/delete for some small allocations, but there was a possible
leak because early returns would skip the delete. But even when not
leaking, this was wasteful. Replace by a stack allocation using
OIIO_ALLOCA outside the loop -- both problems solved.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes bounds checks for malformed JPEG APP1 Exif and APP2 ICC metadata
markers.
Before this change, APP1 markers were treated as Exif when they started
with
`"Exif"` and had at least 4 bytes. The reader then skipped 6 bytes for
the
`Exif\0\0` prefix, so short payloads such as `Exif` or `Exif\0` could
advance
past the saved marker buffer and underflow the remaining length passed
to the
Exif parser.
The APP2 ICC path also used `strcmp()` on raw marker bytes before
proving the
marker was long enough or NUL-terminated. Short APP2 markers could
therefore be
read past the saved marker buffer, and the code later accessed sequence
bytes
and subtracted `ICC_HEADER_SIZE` without first checking the marker
length.
This is reachable by normal OpenImageIO image-open paths, including
tools such
as `iinfo`, `iconvert`, and `oiiotool`, when opening a malformed JPEG.
The fix adds bounded marker-prefix helpers:
- APP1 Exif is accepted only when the full `Exif\0\0` prefix is present.
- APP2 ICC is accepted only when the marker is at least
`ICC_HEADER_SIZE` bytes
and matches `ICC_PROFILE\0` using bounded comparison.
### Tests
Added coverage to `testsuite/jpeg-corrupt` for short APP1/APP2 metadata
markers:
- APP1 Exif payload lengths 4 and 5
- APP2 ICC payload lengths 11, 12, and 13
Validated locally with:
`ctest --test-dir build-jpeg-only -R '^jpeg-corrupt$'
--output-on-failure`
Assisted-by: Codex GPT 5.5xHigh
Signed-off-by: Vlad (Kuzmin) Erium <libalias@gmail.com>
This just avoids needing to zero out the allocated buffer, since we know
we are allocating it merely to write into it. Uninitialized contents is
ok and faster.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
These are all items fixed and documented in the latest releases. I had
hoped that the CVE reference numbers would be ready before I pushed
those releases out. But let's just say that GitHub's "3 business days to
assign a CVE" claim is aspirational.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes bounds handling for WebP EXIF metadata chunks.
Before this change, the WebP reader always skipped the first 6 bytes of
an EXIF chunk before calling `decode_exif()`:
```cpp
bytes + 6
size - 6
```
That assumed every WebP EXIF chunk was at least 6 bytes long and always
started with the optional Exif\0\0 prefix. A malformed WebP file with a shorter EXIF
chunk could therefore advance the pointer past the chunk payload and
underflow the remaining size passed to the EXIF parser.
This is reachable through normal OpenImageIO image-open paths, including
tools such as iinfo, iconvert, and oiiotool, when opening a malformed WebP
file.
The fix changes the WebP reader to treat the EXIF chunk as a bounded byte span.
Before calling decode_exif(), it now verifies that the payload contains enough
bytes for a TIFF header, either at the start of the chunk or after an optional
Exif\0\0 prefix. The WebP reader no longer performs an unconditional 6-byte
skip; the shared EXIF decoder already handles the optional prefix safely.
### Tests
Added WebP regression coverage for malformed short EXIF chunks.
The testsuite/webp test now generates WebP files with EXIF chunk payload
lengths:
- 0 bytes
- 4 bytes: Exif
- 5 bytes: Exif\0
- 6 bytes: Exif\0\0
- 13 bytes: Exif\0\0 plus a truncated TIFF header
Each generated file is opened through oiiotool to verify that malformed EXIF
metadata does not crash or read past the chunk buffer.
Validated locally with:
python3 -m py_compile testsuite/webp/src/make-short-exif-webp.py
git diff --check
cmake --build build-webp --target oiiotool -j 8
Also ran direct oiiotool smoke tests for all generated malformed WebP
inputs.
Assisted-by: Codex GPT5.5xHigh
---------
Signed-off-by: Vlad (Kuzmin) Erium <libalias@gmail.com>
* SWAPRGBABytes: convert to from raw pointers to span-based.
* A variety of overflow safety fixes, replace int (32 bit) with size_t
or int64_t and use safe_mult64.
* Use ImageInput::check_open() for dpx files to check for
reasonable/legal resolutions and channel counts
* Comment out more function declarations not used by OIIO.
Aside:
I should say, this code was originally imported/vendored from another
project by Patrick Palmer: https://github.com/PatrickPalmer/dpx
We imagined it might be an ongoing/improving work, so we made some very
minor changes here and there but endeavoured to make as little change as
possible -- even excluding it from our clang-format rules! -- so that we
could diff against the changing dpx project and pull in any changes or
even use it as an external dependency.
But as you can see if you go there, it hasn't had any modifications for
17 years! So we never needed the "feature" of minimizing divergence from
the original. And now I think with the rate of discovery and reporting
of vulnerabilities and bugs accelerating, the pressure is on to make
this code "safer," for example with these changes in this PR.
I think it's time to give up the pretense entirely and just allow
ourselves to fully absorb this code as our own, be unconcerned about
divergence. So after this PR is merged, I expect follow-ons to:
- Once and for all, fully remove the "dead code" parts that we commented
out because they aren't used in OIIO.
- Allow clang-format to process these files and bring them into
formatting unity with the rest of OIIO.
- Convert all the raw pointer use to spans
- Remove redundant code -- functions in the original dpx project that
were functionality equivalent to things already in OIIO -- where we kept
the originals in place for the sake of minimizing divergence. Let's just
use the OIIO ones we use everywhere else, in cases where they already
exist.
- Root out all remaining overflow and bounds issues, some of the new LLM
based tools are really good at finding those.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This will get us through the great Canonical attack of May 2026, where
it seems that `add-apt-repository ppa:strukturag` is still unreliable
even as we are now multiple days overdue to get our releases out.
We will revert this after everything is fixed. I don't like it silently
passing when we think it's testing heif files but it's really not, any
longer than we have to be in that state.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This is mostly to get it to pass when the GHA runners can't
add-apt-repository ppa:strukturag when it's glitchy down. It can fall
back to an older libheif 1.17 that's already on those runners, but needs
a matching reference output for that version. This is it.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Fix bugs when channel order isn't strictly R,G,B,A by using a
`m_channel_map`.
* Detect malformed file with too many channel packets.
* Gracefully handle duplicate channel bits across packets by not
thinking that there are duplicate channels.
* Guard against channel packets that contain no channels.
Assisted-by: Claude Code / Opus 4.6
(mostly for analysis/identification of the potential problems; touching
the code only for the out-of-order channels fix)
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Fix incorrect return check for fread in read_pixels_pure_run_length.
* Detect and report error for unsupported packet type.
* Fix possible infinite loop from malformed RLE packet with a zero
run-count.
* Check for valid y range in read_native_scanline.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Oh dear, somehow our heif reader never correctly overloaded the
current_subimage method, instead inheriting the one from the base class
that always returns 0. That's very bad for a file type that supports
multiple subimages.
Along the way, simplified how we deal with the fact that the "primary"
subimage doesn't need to be first in the file, but we always want to
present it as if it were first. There was a point where I thought that
was related to the bug so I was rewriting it; it turned out to not be
the problem, but I'd still like to keep the cleanup I had done.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
First PR on the way to a migration from pybind11 to nanobind.
New env flag introduced `OIIO_PYTHON_BINDINGS_BACKEND` = `pybind11` or
`nanobind` or `both`
> Select which Python binding backend(s) to configure. `both` keeps the
existing pybind11 module and also builds the experimental nanobind
module.
When it is nanobine one is build it is in
`PyOpenImageIONanobindExperimental` target.
```
cmake --fresh -S . -B build \
-DUSE_CCACHE=OFF \
-Dfmt_DIR=/opt/homebrew/lib/cmake/fmt \
-DOpenColorIO_DIR=/opt/homebrew/lib/cmake/OpenColorIO \
-DOIIO_INTERNALIZE_FMT=ON \
-DOIIO_PYTHON_BINDINGS_BACKEND=both
```
### Checklist:
<!-- Put an 'x' in the boxes as you complete the checklist items -->
- [x] **I have read the guidelines** on
[contributions](https://github.com/AcademySoftwareFoundation/OpenImageIO/blob/main/CONTRIBUTING.md)
and [code review
procedures](https://github.com/AcademySoftwareFoundation/OpenImageIO/blob/main/docs/dev/CodeReview.md).
- [x] **I have updated the documentation** if my PR adds features or
changes
behavior.
- [x] **I am sure that this PR's changes are tested somewhere in the
testsuite**.
- [x] **I have run and passed the testsuite in CI** *before* submitting
the
PR, by pushing the changes to my fork and seeing that the automated CI
passed there. (Exceptions: If most tests pass and you can't figure out
why
the remaining ones fail, it's ok to submit the PR and ask for help. Or
if
any failures seem entirely unrelated to your change; sometimes things
break
on the GitHub runners.)
- [x] **My code follows the prevailing code style of this project** and
I
fixed any problems reported by the clang-format CI test.
- [x] If I added or modified a public C++ API call, I have also amended
the
corresponding Python bindings. If altering ImageBufAlgo functions, I
also
exposed the new functionality as oiiotool options.
This code contribution was assisted by Cursor/Composer-2
---------
Signed-off-by: Aleksandr Motsjonov <soswow@gmail.com>
PR #5154 crossed streams with another that turned on more warnings for
unchecked errors. The check isn't necessary here, though.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
1. Harden against buffer overruns from malformed RLE blocks. The simple
fix is the `if (count > ...)` clauses on lines 503 & 510, but I also
took the opportunity to rewrite decode_rle_span using spans rather than
raw pointers.
Fixes#5152
2. Clean up seeking methodology to be more robust.
Fixes#5159
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Implements the FLIP algorithm from Andersson et al. 2020, rewritten with
OIIO idioms (no source code from the BSD-licensed reference FLIP.h is
directly incorporated). Included are C++ and Python API for
ImageBufAlgo::FLIP_diff(), and `oiiotool --flipdiff` command. The basic
operation is to compare two images and produce a per-pixel error map
that conveys perceptual difference to human observers.
I won't explain it all here; see the extensive comments in
imagebufalgo.h, imagebufalgo_flip.cpp, oiiotool.cpp, imagebufalgo.rst,
oiiotool.rst.
There are some important changes (especially for how we expose this via
oiiotool) versus the way the original NVIDIA reference implementation's
command line tool worked. Please read the extensive comments at the top
of imagebufalgo_flip.cpp for details.
This is a preliminary, experimental implementation. It's hidden behind
an `experimental` namespace (and the oiiotool command requires use of
the `--experimental` argument) to emphasize that it may change and is
not yet considered part of OIIO's public API, and thus is exempt from
our usual strict rules about breaking backward compatibility. Try it out
and give feedback, but do not rely on this yet!
Assisted-by: Claude Code / sonnet-4.6 + opus-4.6
I used Claude Code for the inital stab at transforming the NVIDIA
reference implementation into OIIO idiomatic equivalents. But to be
honest, that got me over the hump of the blank page, but I rewrote most
of it bit by bit as I continued to refine my design ideas for how it
should work and be exposed to users.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Move this from internal-only in imageio_pvt.h to be public in memory.h.
It's useful and we've used it in limited ways for a while without
needing any changes. It seems ready for prime-time (for example,
allowing it to be used in reader/writer plugins). None of the logic
changes, it's just moving from a private header to a public header and
out of the 'pvt' sub-namespace.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Try to bubble up errors, stop processing as soon as an error has
occurred, correct return code if a file could not be read properly.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
In PR #5114, just a few weeks ago, I was trying to catch every possible
OCIO exception. That was a good idea, but I was over-eager and treated
every OCIO exception as if something really bad happened. I should have
recognized that OCIO enjoys throwing exceptions for things where I would
have just used an empty return value to indicate a simple failure to do
what was asked.
So this patch is revising the approach as follows:
* Exceptions thrown because it couldn't read a config properly (it was
missing or broken), or couldn't produce a ColorProcessor when asked --
those are real problems that we should treat as serious errors.
* Exceptions thrown because it can't satisfy a simple request (like
"what is the name of the color space for this role" in a case where the
role you passed is not known by the config) -- we should catch those
exceptions, but it's not an "error" for OIIO, we just return an empty
string because there is no color space for that role.
Also beefed up some of the documentation of our ColorConfig class
methods to describe what happens with certain kinds of failures.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Fix setting and response of OIIO_CI -- it was less functional before
than I had thought because we didn't actually define the symbol for
compilation.
* More aggressive cost reduction when combining CI + Debug builds,
reduce resolution of some benchmarks, etc., in addition to reducing
iterations as we had done before.
* Mark the most expensive unit tests with a cost, so CTest knows to
start them first. (more explanation below)
* Add Testing/Temporary/CTestCostData.txt to the cache-save paths --
that preserves CTest's understanding of how much each test costs so it
can schedule it more intellegently for the next run.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Guard against corrupt XMP blocks with TIFF related names: if the XMP
item starts with "tiff:", just ignore it if it's not one of the few
TIFF-related XMP items we recognize of if it's marked redundant, and
don't *replace* a TIFF related item that already had a value in the
spec.
Fixes#5157
Signed-off-by: Larry Gritz <lg@larrygritz.com>
decode_xmp() only searched for `</rdf:Description>` end tags, so
self-closing `<rdf:Description ... />` elements (valid XML) were
silently skipped. Now detect both `/>` and `</rdf:Description>` endings,
plus simplify quite a bit generally, by rewriting decode_xmp to use
pugixml more thoroughly to do the whole parse job instead of trying to
hunt for tags ourselves.
Fixes#5021
Assisted-by: Claude Code / Opus 4.6
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Check that the palette isn't senselessly bigger than the bpp calls
for.
* Protection against integer overflow when addressing the palette.
* Fix misunderstanding of the role of the palette "start" index. Pixel
values are absolute, not relative to the start index. The bug would have
produced incorrect images (or incorrectly reported image corruptions) if
we ever came across an tga file with a non-zero palette start index.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This fixes a longstanding issue where saying
make test TEST="foo|bar"
would be malformed, instead of running tests that matched either foo or
bar (misinterpreting the `|` as a pipe). The problem was just not
quoting it properly.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
According to the TIFF spec, TIFF files with a missing rowsperstrip
should assume the whole image is one strip.
Somewhat related (found in debugging): add a DASSERT to help debug if
fmath.h's round_to_multiple() is given an invalid denominator.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This method is advertised as being able to convert any type of the
attribute into a string, but actually, it would only *search* for
attributes that were entered as strings, so there was no conversion
happening.
Beef up testing of paramlist and imagespec to cover the holes that let
this mistake happen.
Fixed a couple typos in imageio.h comments as well.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Rearrange the automatic setting of `OIIO_NODISCARD_ERROR_ENABLE` so that
for external (not OIIO itself) builds, we default to enabling for OIIO
>= 3.3, and disabling for older and current versions. But now, when
building OIIO itself, it is ENABLED regardless of version.
So this means that as we add OIIO_NODISCARD_ERROR annotations to
functions, we MUST fix any calls to those functions where we ourselves
don't check their return values, and can't accidentally add new breaking
code.
Also, had to fix a bunch of spots that were missed in the original PR,
places where we didn't patch up our sloppy uses of what was annotated in
that patch. I'm not sure I understand how these could have been missed
in that last PR, since that PR did fix other uses of the annotated
functions.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The security@openimageio.org is still fine, but we prefer that true
vulnerability reports come via the GitHub security advisory mechanism.
(That makes it easy for us to turn them into CVEs when needed, among
other administrative niceties.)
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Introducing this as a way to allow us to have experimental oiiotool
features or behaviors that are enabled only if this flag is also
present.
It's a way of allowing features to be turned on for feedback before
we're sure it's in a form that can be a permanent, supported feature,
thus allowing it to undergo further change that is exempt from our usual
rules about what may change in which versions. By forcing users to also
employ this flag to enable the trial behavior, there's no way that they
can inadvertently come to depend on a temporary behavior and be
surprised if it changes later.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
A few years ago, in PR #3301, we made a big effort to remove Imath types
as much as possible from the public APIs. Somehow, as part of that
refactor, we ended up with just the half-varieties of fmath.h's
convert_types() templates actually being implemented in fmath.cpp and
only declared (by default) in fmath.h. I'm honestly not sure why I did
that now, and the PR doesn't explain that particular point.
But that sets up a potential link incompatibility if OIIO used a
different release of Imath than the use is using when calling that
utility function.
So this PR makes it fully inline again, so the Imath version the
downstream package uses just doesn't need to match what was found/used
at the time that OIIO was built.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Change DASSERTs (which were correct, but did not enforce anything in
release builds) into real checks that will return error messages and
flags if the corruptions are found.
* Switch to using spans for extra checks if anything goes wrong.
* Use check_open.
* Check that various header items are valid enum values.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We didn't have a complete enough test for when it was safe to hwy on a
buffer. Encapsulate in new function HwySupports so that all the logic
lives in one place and doesn't have to be replicated everywhere. The
more complete test now checks for:
- contiguous scanlines
- right number of channels (only indirectly checked)
- 2D image, not 3D volume (not previously checked)
- pixel data window fully contains the ROI (not previously checked)
- correct data type (should always be true, but was not checked)
This eliminates some test failures, which were specifically cases
where the ROI being operated on did not overlap with the data area of
the ImageBufs, and the Highway related code did not handle those in
the way that the old ImageBuf::Iterator transparently handled running
off the edge and various wrap modes.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* imageio.h: Document 'enable_hwy' global option.
* imagebufalgo_hwy_pvt.h: Remove RoiNChannels -- it's the same as
roi.nchannels()
* Macros: we can simply test OIIO_USE_HWY, since it's always defined;
there is no need to also test defined(OIIO_USE_HWY).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
1. Some tests are repeated, run once with hwy off and once with hwy on.
The "on" versions append ".hwy" to the test name and set the
OPENIMAGEIO_ENABLE_HWY=1 environment variable to force it on while
running the test. The advantage to this approach is that the test itself
doesn't need to change at all, we just run it separately in each mode.
In this PR, I do this with oiiotool, oiiotool-xform, and
docs-examples-cpp.
For now, while hwy is not enabled by default, it's doing these extra
things to enable hwy. Eventually, if/when hwy is enabled by default,
we'll flip the sense and use the env variable to turn it off.
2. Some tests do certain operations both ways when hwy support was
enabled at build time. This is more convenient for certain unit test
executables, where perhaps we want to benchmark both ways and have those
show up right next to each other within the run.
In this PR, I do this with the imagebufalgo_test unit test.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Add a new ImageBuf method: `contiguous_scanline()`, which reveals if
each scanline is contiguous in the ImageBuf's internal memory (no
padding or unusual spacing between channels or pixels). It's analogous
to the existing `contiguous()`, but the latter also tests contiguity in
y and z.
Additional internal fixes:
* Always and only set m_bufspan via calls to `set_bufspan()`, so it's
consistent. In particular, it ensures that we correctly set the channel
size, which some call sites previously did not do correctly for untyped
"byte" buffers.
* Re-evaluate the contiguous flags whenever resetting m_bufspan.
* contiguous() calls need to call validate_pixels().
An IB from a file that hasn't been read yet into memory yet isn't
considered contiguous. But once it's read, it will be (if it's got the
strides right, etc.). So when somebody asks if it's contiguous, what
does that mean? They are asking because they are about to do something
that requires memory contiguity (meaning, it needs to be IN memory),
just like if they call localpixels(). So just like localpixels(), we
need to call validate_pixels() to assure that if the pixels are destined
for the local buffer, it's done now if not yet.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Recent PR #5081 added more detection of corruptions in (among other
places) the IPTC blocks in a JPEG file.
But we maybe got a little too aggressive, since there appear to be some
jpg files out there that have strange (maybe technically incorrect?)
IPTC blocks but are otherwise fine.
Anyway, I noticed that for ICC profiles in JPEG files, we use the OIIO
attribute "imageinput:strict" to decide whether a localized corruption
in the ICC profile ONLY causes the ICC profile to be skipped (reading
the pixels just fine), or a stricter mode where anything suspicious is
grounds to stop reading the file entirely and declare it corrupt.
So in this patch, I'm using the same control for the analogous purpose
when reading the IPTC blocks. By default, just skip the IPTC block if we
don't know how to decode it. But if in strict mode, make a full error
that will cause us to stop reading the file.
With this change, I needed to update the jpeg-corrupt test to force
strict node, and update some some ref outputs.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
## Summary
Adds `OIIO_NODISCARD` to `ImageInput::create()`, `open()`,
`read_image()`, and `read_scanlines()` methods. Callers ignoring return
values from these methods now get a compiler warning.
## Why this matters
As noted in #4781, it's easy to accidentally ignore the bool return from
`read_image()` or `open()`. These methods return false on failure, and
silently proceeding with bad data leads to hard-to-debug issues.
`create()` returns a unique_ptr that should never be discarded.
## Changes
- `src/include/OpenImageIO/imageio.h`: Added `OIIO_NODISCARD` to 14
method declarations in the `ImageInput` class, covering `create()` (2
overloads), `open()` (4 overloads), `read_image()` (4 overloads), and
`read_scanlines()` (4 overloads).
Scoped to `ImageInput` only per @lgritz's guidance to start with one
class and iterate.
## Testing
- Header compiles (syntax validated)
- No behavioral changes - annotation only
Partial fix for #4781
Assisted-by: Claude Code / Claude Opus 4.6 (1M context)
Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Co-authored-by: Larry Gritz <lg@larrygritz.com>
* Have build_dependency_with_cmake print the error message if the git
repo can't be cloned.
* Fix mis-quoted env variable in status message.
* Improve grammar of status message.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I think forcing fmt to disable exceptions is probably doing more harm
than good, probably does not work as expected for apps that use fmt
elsewhere (when we first started using fmt, it was not super popular, so
we weren't really concerned about how we might conflict with other
libraries in the same app), and disabling exceptions makes it behave
differently than std::format/print.
So let's stop doing this and see what happens! (In main.)
Also, declutter by removing the FMT_USE_GRISU definition, which was only
meaningful for very old fmt versions that are older than our minimum
supported version now.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
For the sake of Windows' need for wstring filenames, we do some
utf8<->utf16 conversion in strutil.cpp. That has always used the
std::codecvt set of functions, but they have been deprecated beginning
in C++17 and will be removed entirely in C++26, so their days are
numbered and we already have to suppress warnings to get the compiler to
shut up about it.
Assisted-by: Claude Code / Opus 4.6
Signed-off-by: Larry Gritz <lg@larrygritz.com>
It must have been among the homebrew packages in the base Mac
installation on GHA, until just a couple days ago?
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#5132.
Reading DDS volumetric files used to work, but was broken by PR #4669
last year, the one where we added all the span-oriented API calls.
At that time, I did not realize/remember that DDS files could be
volumetric, observed that all the (other) volumetric files only
supported tiled access, and so in the course of that API refactor, I
removed the z parameter from the new read_scanlines API.
I don't want to break ABI compatibility again for this, so the solution
I came up with in this PR is just to make DDS volume files report
themselves as "tiled", with each tile being (width x 1 x 1), i.e. what
used to be one scanline, and it all just works out.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fix warnings for gcc15:
* Suppress warnings about deprecated utf conversion (already suppressed
for clang, needed to do so for gcc >= 15 as well).
* Rename some internal methods that gcc 15 was warning were shadowing
the names of virtual methods.
Also took the opportunity to update INSTALL.md to reflect the latest
dependency versions that we have tested against.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Optional SIMD optimizations for selected ImageBufAlgo operations using
the Google Highway library:
* add/sub
* mul/div
* mad
* resample
Adds CMake and build system support, new implementation helpers, and
developer documentation.
This is currently off by default, needing to be enabled at build time both by
setting CMake variable `OIIO_USE_HWY` to ON *and* having the Highway
library installed and able to be found (we do not yet auto-build it). And even
if enabled at build time, currently it is only enabled at runtime by setting the
environment variable `OPENIMAGEIO_ENABLE_HWY=1` *or* setting the
global attribute `OIIO::attribute("enable_hwy", 1)`. We hope that the defaults
might be changed to fully enabled by the time we release OIIO 3.2.
Assisted by: Opus4.5 and Codex GPT5.2
_Code mostly wrote using frontier Opus4.5 and Codex GPT5.2 High models
with a strict rules._
---------
Signed-off-by: Vlad (Kuzmin) Erium <libalias@gmail.com>
We have enough spurious failures that I don't want all of the variants
in a job matrix to fail just because one goes wrong.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This test did more than was really necessary. It resized a small image
(twice) to get big images, but the contents didn't matter, so just using
a gradient is good enough. Also, we can check results just by using the
hash of the image contents, don't need to do extra idiff calls.
It's barely noticeable on a regular release build, since the whole test
was only a couple seconds on my laptop and maybe 15 seconds on the GH
runners. But for the CI "sanitize" test, this cuts about 30 seconds from
this test! Every little bit helps, the sanitize tests are usually the
last to complete in a CI run.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This PR fixes an inconsistency in OIIO's XMP write path that prevents
some metadata fields from being emitted into XMP, even when they are
present in `ImageSpec::extra_attribs`.
**Problem**
`encode_xmp()` gathers attributes through `gather_xmp_attribs()`, which
currently does:
```cpp
const XMPtag* tag = xmp_tagmap_ref().find(p.name());
if (tag) {
if (!Strutil::iequals(p.name(), tag->oiioname))
continue;
...
}
```
At the same time, `xmp_tagmap_ref()` builds its lookup map using
`xmpname`, not `oiioname`.
This becomes self-contradictory for mappings where the XMP name and the
OIIO-facing attribute name differ.
A concrete example from `xmp.cpp` is:
```cpp
{ "xmp:Rating", "IPTC:Rating", TypeDesc::INT, 0 },
```
With the current implementation:
- If the spec contains `IPTC:Rating`, lookup fails because the map is
keyed by `xmpname`.
- If the spec contains `Xmp:Rating` / `xmp:Rating`, lookup succeeds, but
the attribute is then rejected because it does not match `oiioname`.
As a result, `encode_xmp()` does not emit the rating field, and format
writers that rely on `encode_xmp()` (for example JPEG and TIFF) may fail
to write an XMP packet for such metadata.
**Fix**
This patch makes the XMP tag lookup accept both spellings:
- the XMP name (`xmpname`)
- the OIIO metadata name (`oiioname`)
and allows `gather_xmp_attribs()` to accept either as a valid match.
This preserves existing behavior for fields where `xmpname == oiioname`,
while fixing aliased mappings such as rating.
**Why this is correct**
OIIO already models these fields as explicit mappings between an XMP
name and an OIIO attribute name. The write-side gather logic should
therefore be able to consume metadata stored under either spelling.
Without this, some entries in the XMP mapping table are not actually
encodable.
**Observed impact**
In a local reproduction using JPEG output:
- `IPTC:Rating` was present in `ImageSpec::extra_attribs`
- the JPEG writer called `encode_xmp()`
- no XMP APP1 segment was written to the final file
This was traced back to the lookup/match mismatch described above.
**Testing**
I recommend adding tests that cover both direct XMP naming and aliased
OIIO naming.
1. Unit test: `IPTC:Rating` is encoded as XMP rating
```cpp
ImageSpec spec(1, 1, 3, TypeDesc::UINT8);
spec.attribute("IPTC:Rating", 4);
std::string xmp = encode_xmp(spec, true);
CHECK(xmp.find("xmp:Rating=\"4\"") != std::string::npos);
```
2. Unit test: `xmp:Rating` is encoded as XMP rating
```cpp
ImageSpec spec(1, 1, 3, TypeDesc::UINT8);
spec.attribute("xmp:Rating", 4);
std::string xmp = encode_xmp(spec, true);
CHECK(xmp.find("xmp:Rating=\"4\"") != std::string::npos);
```
3. Optional integration test: JPEG writer emits an XMP APP1 segment when
rating is present
- Create a small image
- Add `IPTC:Rating` or `xmp:Rating`
- Write JPEG
- Inspect the output bytes for the standard XMP APP1 namespace header:
`http://ns.adobe.com/xap/1.0/\0`
This verifies the full `encode_xmp()` -> JPEG writer path, not just the
XML generator.
**Notes**
This patch is intentionally narrow:
- it does not change the XMP mapping table itself
- it only fixes how mapped attributes are discovered during XMP
generation
Signed-off-by: Qi-fly <72260719+Qi-fly@users.noreply.github.com>
Fixes#5124
`fancy_add_executable()` currently adds the Windows manifest resource
file for all
`WIN32` builds, but it also unconditionally passes `/MANIFEST:NO` to the
linker.
That switch is MSVC-specific, so MinGW ends up forwarding it to `ld` as
a path
and the executable link fails.
This change keeps embedding `src/windows/oiio_exe.rc` for all Windows
toolchains,
but only adds `/MANIFEST:NO` when the build is using MSVC.
That preserves the intended long-path manifest behavior from #5066 while
letting
MinGW link the Windows executables again.
Assisted-by: Codex / GPT-5
Tests:
- Ran a targeted CMake smoke test that configures a dummy executable
with
`WIN32=TRUE,MSVC=TRUE` and verified that `/MANIFEST:NO` is present in
`LINK_OPTIONS` and the `.rc` file is attached as a source.
- Ran the same smoke test with `WIN32=TRUE,MSVC=FALSE` and verified that
the
`.rc` file is still attached but `/MANIFEST:NO` is not added.
- I could not run an actual MinGW build locally in this environment
because a
MinGW toolchain is not installed here.
Signed-off-by: Asish Kumar <officialasishkumar@gmail.com>
fmtlib renamed its "master" branch to "main" yesterday. So now our
"bleeding edge" test that requests fmt "master" fails.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Some fuzzing stumbled across a very interesting edge case in the
rehashing logic of ustring internals:
If the initial hash location of a newly ustring-ized string is already
occupied, it rehashes. But we never considered what happens if it ends
up rehashing to the same slot! And in fact, that can happen if the true
hash is 0 (I'm not sure if it can/does for any other value). And we may
have thought that the only string that would ever hash to 0 is the empty
string (which is 0 hash by design, and doesn't need to rehash because
the table is seeded with that entry already in its correct slot).
So anyway, if there was some other string that happened to hash to 0, it
would get caught in an infinite loop of rehashing but never finding a
free slot, because it keeps rehashing to the very same occupied slot.
And would you believe that there IS such a string, and it is this one:
"\t\n\n\t\301\350`O\217c85?\202\200\251r|}\304\351\2517\337K'\217C\240"
(unprintable chars expressed as escaped octal values)
The crux of this patch is to notice when the old and new hash bit
pattern has not changed, and bump it by 7 (a prime number, so it should
not land on the same location again until every slot in the bin has been
exhausted).
Solution entirely from my own human brain, in case you're wondering.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I was reminded by Eric Enderton that we should mention and cross
reference the recently-added AI policy docs in other places, since it
can easily be overlooked now, buried in the docs directory.
* Mention it in the pull request template's checklist.
* In the CONTRIBUTING guide.
* Briefly mention in the main README in the list of important
documentation for developers.
* A few very minor touch-ups and wordsmithing in other areas, no
substantive changes.
Also, responding to comments at the OSL TSC meeting this week, I added
a brief section explaining that we don't have any specific guidance for
how AI tools should or shouldn't be used for "artistically creative"
work, since this project repo doesn't really house such things. This
is mostly for the benefit of other projects that may use our policy
document as inspiration, to warn them that they may have issues that
need policy guidance but that we do not address.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
TL;DR:
* Bump ASWF containers to the latest versions for their years.
* Move the "oldest" and "hobbled" tests off the 2022 containers
entirely, do as close as possible to equivalent on Unbuntu.
Detailed explanation follows:
First of all, bump to the latest ASWF docker containers for each year.
We had been locking down, but hadn't bumped to the latest for a while.
(Exception: leave the sanitizer test where it is -- the newer containers
have additional checks that are catching UB that I am tracking down
separately.)
Also, we are trying to reduce the surface area of the CentOS 7-based
2022 aswf-docker containers, which are getting long in the tooth, cannot
upgrade to the latest "actions" (because of a tricky interplay between
runner configuration, node version the actions require, and glibc on the
old containers), and may have a limited time that they will continue
working.
In most film studios, CentOS is long gone or taking its last breaths, so
being able to support and test on 2022 era dependencies and OS distros
is of dwindling relevance. We do, however, want to keep testing versus
the oldest supported versions we can muster for many major dependencies.
So to this end, this PR also does the following:
* Move three tests from running in the 2022 containers to just running
on bare Ubutu and by hook or by crook using the right dependencies:
- the "oldest" versions we claim to support for as much of the toolchain
toolchain and dependencies as is easily achievable, built with gcc
- and the same, built with clang
- the "hobbled" test, which is similar to the "oldest" test but with as
many of the optional dependencies missing or disabled, to make sure we
really do build with only required dependencies (this also is the test
build for no SIMD use)
* Keep two last tests running on 2022 containers, these are only for the
purpose of testing "pure" CentOS VFX Platform 2022 compliance for just a
bit longer.
* A few minor related touch-ups necessary to force some older
dependencies on the bare Ubuntu runner.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#4942
IBA::fillholes_pushpull() produced incorrect results when the source
image had a non-zero data window origin (such as exr files with
overscan).
Two interrelated bugs:
1. The paste() call double-applied the data window offset. paste() maps
source pixel (0,0) to destination (xbegin,ybegin), but since the source
data starts at (x,y), passing (x,y) as the offset shifted every pixel by
an extra (x,y). Pixels in the negative-coordinate region landed outside
the destination buffer and were silently lost.
2. The top pyramid level preserved the original display/full window
while all smaller levels had full=data at (0,0). Since resize() uses the
full window for coordinate mapping, it only sampled the display window
portion of the top level, losing all overscan data.
Fix by shifting the top pyramid level to origin (0,0) with full=data,
making all levels coordinate-consistent. The initial paste translates
source pixels into the origin-based pyramid, and the final paste (which
was already using src origin offsets) correctly translates back.
Add test cases in testsuite/oiiotool for fillholes with an offset data
window, and an offset display window, to ensure that both cases work.
Also, for the existing fillholes related test, move the source image
from ref (never should have been there) to src.
Assisted-by: Claude Code / claude-opus-4-6
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I've heard occasional reports of uncaught exceptions in print
statements. I haven't caught it in the act yet, but one set of reports
involved the following output:
OpenImageIO/detail/fmt/format-inl.h:40: assertion failed: string pointer
is null
So I'm proactively trying to find places that might be passing a null as
a string. I'm just guessing here, but these are plausible candidates.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
PR #5113 didn't fix all the problems after all. Revert it, because it's
not needed after the real solution.
The real solution is to mark tiff_datatype_to_typedesc() with
OIIO_NO_SANITIZE_UNDEFINED to have the sanitizer skip it. What's
happening is that corrupted files end up with a value in the data type
field that is not a valid value of the TIFFDataType enum, and ubsan is
flagging that. BUT... `tiff_datatype_to_typedesc()` itself is actually
safe in that circumstance, because it's got a 'default` clause that
handles the unknown enum values just fine.
In contrast, I've been running around in circles trying to find
something to do *within* that function to make it "safe" (by the
sanitizer's reckoning), and trying to check for valid values prior to
converting to a TIFFDataType is just too hard, partly because there are
places where that conversion happens unchecked inside libtiff (C
language, just does a cast), so it comes to us as a TIFFDataType already
with an invalid value. It's easier to just mark the whole function as
being ignored by ubsan, given that we can see by inspection that it has
totally deterministic and desired behavior for the illegal values.
Making this spurious ubsan error disappear allows us to upgrade the
container version we are using for the "sanitizer" CI test variant, so
we also push that all the way up to 2026.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Corrupted exif data could put a value in a "tiff data type" field that
is not one of the valid enum values. That's UB. Identified by running
the sanitizer with a newer clang than we usually do.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
When reading corrupt files, sometimes there are a lot of different
potential error messages reported by libtiff depending on the exact
version and platform, and which corruption it encounters or notices
first. So we tend to have to add new refs frequently as libtiff changes.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
https://opencolorio.readthedocs.io/en/latest/api/exceptions.html
"All functions in the Config class can potentially throw this
exception."
Oh dear, that is not something I previously fully understood the
implications of. This patch makes extra sure that there's try/catch
around every call to OCIO::Config methods. Some previously did, but some
that were less obvious that they might throw did not try/catch. No
matter how benign they look, I'm taking no chances now.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#5063
In that issue, it was reported that the printing of any unretrieved
error messages upon destruction of an ImageBuf could itself have an
exception (including if it, for some reason, could not correctly fwrite
the output). And if that is the case, it could terminate the app.
So for safety, this patch encloses this particular print statement in a
try/catch. And also found another in ErrorHolder in the internals of
Strutil, which is used for "global" errors issued to OIIO::errorfmt().
Assisted-by: Claude Code / Opus 4.6
I did the fix in ImageBuf, then Claude found the other spot in
ErrorHolder that also needed the fix.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Rather than use old actions across the board for a handful of job
variants that need old containers, use the obsolete actions only where
really needed. This involves duplicating the some steps within the
workflow, and choosing whether to use old or new based on job. We'll get
rid of the "old" versions when we retire the 2022 containers that
require them.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Summary of our core values:
- Human always in the loop and is the responsible party.
- You're still on the hook for fully understanding and standing behind what you submit.
- Interact with the project and community yourself, not by agent.
- Disclose what tools you used and how. At the very least, have an `Assisted-by TOOL / MODEL` line.
- Don't waste maintainer time with low quality PRs.
- Try to reserve the curated Dev Days and "Good First Issues" tags for human learning.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The jph_infile class itself did not change, it just moved up in the file
to be defined before it was used. It's the use of the persistent
unique_ptr that fixes the leak.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Validity-check resolution of RLA files with check_open. RLA file
headers contain int16_t values for left & right (and top/bottom) window
coordinate, leading to a maximum resolution of 2^16-1.
* Fix potential bug with sign extension in RLE decoding -- if a signed
char is -128, negating it can't make signed char 128 (no such thing), so
must widen the var to an int.
* Fix potential bug by detecting when the number of matte or auxiliary
bits is 0, but the number of matte or aux channels, respectively, is
not.
* Better bounds checking in decode_channel_group. We did the checks
before, but after some accesses that would have been out of bounds! Move
the checks earlier than all the accesses. It actually looks like was the
result of a cut and paste error long ago.
* More care in read_native_scanline for checking valid scanline numbers,
offset into m_sot, and check whether ioseek succeeded (i.e. whether the
offsets loaded from the file are within the range of the size of the
file).
Code and fixes all are from my own brain, but some of the analysis of
which spots have bounds issues were identified in part by conversation
with Claude Code Opus 4.6.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
If certain camera metadata was mangled and or otherwise wasn't one of
the known metadata names, it could run off the end of the list and
dereference a null pointer.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
A visual inspection shows that it works fine and the images are a visual
match. But some numeric changes in ffmpeg 8.1 make it need new hashes.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
1. Validity check the resolution/channels.
2. Also, in png_pvt.h's `read_info()`, found a spot where errors were
being sent to the global error reporter instead of to the PNG reader's
own error store. (It is relevant to this PR because that is called by
the ICO code for certain ico files that are really using PNG encoding.)
3. Handle scanline width that's not 8 pixel multiple without running
over buffer limits.
4. Guard against possible palette array overrun for 4bpp case. The 1-
and 8-bpp palette cases already had computed the palette index and
checked for being in range before accessing the array. But the 4-bpp
case did not!
Items 1 & 2 were entirely LG. Items 3 & 4 were identified and fixed with
the assistance of Claude Code / Opus 4.6 -- it was really helpful! I'm
honestly not sure if it saved me 5 hours or closer to 5 minutes, but
even if short, it would not have been at all fun to trace through by
hand to find a subtle case that we previously forgot to check.
Assisted-by: Claude Opus 4.6
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This had been biting our wheel building for a couple weeks before it hit
the regular CI jobs, I guess? And I didn't even know it or fully
understand the cause. But it's the same solution as the other fix from
yesterday: explicitly specify the path to the xcode version of clang, so
it doesn't accidentally pick up the homebrew copy of a different clang.
(Either is ok, but mixing and matching is no good.)
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Update action version pins to their latest releases, referenced by full
commit hash with version tag as a comment. Except for a couple, as noted
below.
| Action | Old | New |
|-------------------------------------|------------|------------|
| actions/checkout | v4.2.2 | v4.3.0 |
| actions/upload-artifact | v4.5.0 | v7.0.0 |
| actions/download-artifact | v4.1.8 | v8.0.0 |
| actions/setup-python | v5.2.0 | v6.2.0 |
| pypa/cibuildwheel | v3.2.1 | v3.4.0 |
| pypa/gh-action-pypi-publish | release/v1 | v1.13.0 |
| sonarsource/sonarqube-scan-action | v6.0.0 | v7.0.0 |
| ossf/scorecard-action | v2.3.1 | v2.4.3 |
| github/codeql-action/upload-sarif | v2.3.6 | v3.32.6 |
| sigstore/gh-action-sigstore-python | v3.0.0 | v3.2.0 |
| jmertic/slack-release-notifier | main | 0.1 |
Note that all are locked down to hashes now, including a couple which
hadn't been previously.
We wish we could upgrade actions/checkout all the way to the latest
versions. But in both cases, they require nodejs 24, which in turn needs
a glibc that's too new for our ASWF VFX Platform 2022 containers. So
we're going to have to keep them locked to old versions until we're
ready to retire the 2022 containers, which we hope is soon. The checkout
action is upgraded from 4.2.2 to 4.3.0 only, intead of the latest 6.0.2,
and the cache actions had to remain on 4.3.0 as it was before, instead
of upgrading to the latest, 5.0.3.
The initial gruntwork of tracking down the latest versions and upgrading
was done by Claude Code using the Sonnet 4.6 model using the following
prompt:
```
For all the workflows in @.github/workflows/ , check every action
version reference to see if it is using the latest release
version. Upgrade any that are not the latest. Be sure to refer to the
action by full hash, not version number (but put the corresponding
version number as a comment).
```
That upgraded everything possible without me needing to track down each
action myself, though it was only a partial solution: the newest actions
failed on the old containers (as noted above) and it took me a while to
sort out by hand which ones needed to be held back to run on the old
containers. So Claude helped with the first 80% and then I did the
remaining 80% of the work.
Assisted-By: Claude Sonnet 4.6
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Don't copy from memory using the stride, as this can get padded and read
past the end of the input pixel data. The 10 and 12 bit paths were
correct.
Test added, this only failed with address sanitizer.
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
1. Validate channel count against color mode to prevent heap buffer
overflow.
2. Corrupted PSD files can declare a color mode (e.g. RGB, needing 3
channels) with transparency (needing +1) but report fewer channels
in the header. This caused an out-of-bounds read in
read_native_scanline when setup() built channel pointer arrays
using mode_channel_count, which exceeded the actual
m_image_data.channel_info size.
3. Additional buffer overflow protection, error checking and
propagation.
4. Avoid leak by properly freeing z_stream resources in decompress_zip.
Used Claude Code / Opus 4.6 to help narrow down and confirm the bugs 1
and 2.
Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Reading the TIFF 6.0 spec, on p. 23, it sure looks like palette images
can only be 4 or 8 bits per sample. But the libtiff set of test images
indeed has 2, 4, 8, and yes, 16 bps palette images among the examples.
So I think we should support up to 16, since we might encounter it in
the wild, but no higher.
We didn't previously check for this, though, and that could lead to
accepting invalid TIFF files, and then potentially have problems with
`(1 << m_bitspersample)` which appears several times. That of course is
safe if m_bitspersample can't be > 16.
Also check and proper error for a file that has a bitspersample that is
not one of the valid choices (1, 2, 4, 6, 8, 10, 12, 14, 16, 24, 32,
64), and proper errors for if the sampleformat was an unknown value or
was not a valid value for the number of bits per sample.
Note that this changes the reference output of some tests because the
new checks can alter the first error discovered in a corrupt file.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Also change the x and y counters to uint64_t to prevent any possible
overflow as it's computing offsets.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Be even more aggressive for the runs that are both debug mode (including
sanitizers) and a CI run. Speeds up the slowest GHA test runs. Should
not affect ordinary builds.
This shaves about 8 minutes off the long "sanitizer" run, which is often
the last job to finish in our CI workflows.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The files that match the pattern "src/cmake/build_<dependency>.cmake"
are all similar and each one is called by our build system if a
dependency is not found, to download and built it locally to be
incorporated into our build. One example is src/cmake/build_TIFF.cmake.
Within these files, the version to be retrieved is determined by a pair
of lines such as
```
set_cache (TIFF_BUILD_VERSION 4.7.1 "TIFF version for local builds")
set_cache (TIFF_GIT_TAG "v${TIFF_BUILD_VERSION}" "Git branch or tag")
```
The tag is passed to `build_dependency_with_cmake` with the `GIT_TAG`
argument. There is also an optional `GIT_COMMIT` argument, that we
usually do not use (previously, only needed if we wanted to build from a
commit that didn't have a version tag).
But it turns out that it's probably not safe to blindly retrieve a tag,
in case the dependency's repo has been hijacked and the tag reset to
point to a malicious commit.
So here is how to address this:
- `build_dependency_with_cmake` modifies its internal logic:
- If only the GIT_COMMIT is passed and GIT_TAG is blank, use the commit.
- If both GIT_COMMIT and GIT_TAG are passed, check out the tag but
verify that it is the commit specified, and if not, then consider it an
error and do not proceed with building the dependency.
- If only GIT_TAG is passed and not GIT_COMMIT, warn about the potential
safety problem.
- But as a special option for dangerous tasks, an override called
OpenImageIO_DEPENDENCY_BUILD_ALLOW_UNVERIFIED_TAGS allows an unverified
git tag to be checked out. This is how we make the "bleeding edge" test
still build against dependecy main/masters.
- For each of the `build_<dependency>.cmake` files, change it as
follows:
- Determine the commit hash of the specified version by checking the
repository.
- Add a line setting `<dependency>_GIT_COMMIT` to the corresponding
hash.
- Pass both arguments to build_dependency_with_cmake.
This patch was made with the help of Claude Code using Opus 4.6. I
started the dialog with a prompt that was substantially similar to the
description above, detailing my proposed plan.
I first suggested that if only the TAG is present, to allow it as long
as it turned out to be an immutable git release tag, but it pointed out
that (a) this requires a build-time dependency on the `gh` command line
util, (b) not all our dependencies are on GitHub (such as libtiff on
GitLab), so we modified the plan to warn in that case rather than try to
determine if it's immutable. The rest proceeded more or less as
expected, with a few rounds of back and forth and corrections.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Certain CMYK files (maybe only corrupt ones?) hit an edge case that
caused us to read the original 4 channels but into space for only the 3
channels we ultimately report back. Fix that logic.
Also noticed several spots where we didn't correctly report back error
messages for newer versions of libtiff. This was some kind of oversight
from long ago.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* `decide_iptc_iim()` (semi-public) : change from taking a pointer &
length to a string_view, and internal safety checks.
* `jpeg_decode_iptc()` (non-public): change from taking an unlengthed
pointer (!) to a string_view, and internal safety checks.
* JpgInput::open(): a number of extra checks for memory bounds validity.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* fix check_open used m_spec instead of parameter spec!
The function advertised that its doing checks on the parameter passed
to it, but -- oops -- it was really doing it on this->m_spec. They
were almost always (or always always?) the same, but still. Bug.
* Check zero-sized images separately from nonsensical negative sizes.
By being togther, there was some confusion about whether "noimage"
feature should negate the error (that feature should allow
zero-sized images, but not negative dimensions).
* Add new checks for the full_width/etc to also be sensible.
* Some other minor refactoring of check_open.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The docs for TexturesSystem::attribute() had a summary of ImageCache
options that were accepted and passed along. Over the years, a couple
had been added to IC without being reflected in the TS docs.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The m_legacy_texture_blur flag is set in TextureSystemImpl::Init, which
overrides the default initialisation depending on the OIIO version.
In other words, the texture blur fix was always enabled, regardless of
the OIIO version.
I also updated the help description for this flag in testtex.
This fixes minor shortcomings of #5071
---------
Signed-off-by: Pascal Lecocq <pascal.lecocq@gmail.com>
Encountered at work: when using certain libheif dependency builds that
can read, but not write, avif files, we needed an additional reference
output.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Background: oiiotool is pretty good at remembering the data type of the
first file it reads in a command, and ensuring that the output preserves
the same types, despite the fact that internally, it tends to keep
everything in float. (As a further reminder: -d can override the
heuristics that make this work.)
But there was an exception: when `-i` was used with the optinal modifier
`-i:ch=...` to select a subset of channels or reorder channels upon
read, the underlying call to IBA::channels would cause it to not have
that information in the "nativespec" at the right time that it was
collected.
This PR primarily ensures that the `-i:ch=...` preserves the information
about which channels had which data types, despite the channel shuffle.
Some other bits and pieces along the way:
* A new `TypeDesc::basetype_merge()` variety that take a
`cspan<TypeDesc>`.
* A new utility `TypeDesc::all_types_equal(cspan<TypeDesc>)`
* Cleanup in IBA::channels() to more correctly compute the common type
of the channel subset that you actually specify.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Adds an embedded manifest to OIIO's generated executables on Windows,
which enables "long path" awareness.
This is done via a "resource" file, with the manifest resource ID
specified. From what I've read, this is more broadly compatible with
e.g. non-MSVC build toolchains like MinGW.
The manifest file was created by building OIIO from `main`, extracting
the manifest from the resulting `oiiotool` (using the `mt` tool), and
manually adding in the XML snippet from
https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry#application-manifest-updates-to-declare-long-path-capability.
Fixes#5064
Added a new Windows-only test to validate that `oiiotool` can correctly
read long image paths.
The one executable that I have not tested is `iv`. The only reason I'm
more curious about it than others is that it is a GUI executable, so I'm
not sure if its default
---------
Signed-off-by: Nathan Rusch <nrusch@users.noreply.github.com>
It's not in any pybind11 tagged release yet, but in its master, a change
has been introduced that causes it to allow functions with float
paramters to automatically match int arguments. This is fatal for us, as
we have set up several cases of methods that overload on int vs float,
and with this change, the float one gets called when an int is passed.
Pybind11 docs say that the `.noconvert()` modifier on the argument will
do the trick, but I'm finding that it doesn't help for overloaded
function names.
The solution is to find these cases, change our declaration to a generic
py::object, and then sort out what type was passed with py::isinstance.
It seems to mainly affect `attribute()` style calls.
In the process, minor changes to the pybind11 auto-builder allow it to
take a git commit hash directly. Also, I took the opportunity to remove
a few reference outputs that are no longer needed because they were
python2-specific.
Meanwhile, just in case anybody had been paying attention, the problem
in pybind11 master that had caused us to temporarily stop CI testing
against master seems to have been resolved. So re-enable the bleeding
edge test to use pybind11 master (which will fail if not for this PR
here), and bump the "latest version" tests to the latest pybind11 3.0.2.
I diagnosed the problem myself, wrote the solution for one class and
verified that it solved the issues with that class. Then I used Claude
Code to make the analogous set of changes to the other classes that
needed them and I reviewed that it was correct.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#5069
This PR fixes the texture blur overstimate reported in #5069
We provide the following changes:
* Use the mathematically correct Pythagorean form to properly adjust the
ellispse footprint with st-blur parameters.
* Add a new legacy_texture_blur attribute in TextureSystem to opt into
the fix without breaking existing renders.
* Add a new --fix-texture-blur option in testtex to enable the fix.
* Add a new texture-blurfix test.
Note: This PR has been partially edited using the Claude coding
assistant.
---------
Signed-off-by: Pascal Lecocq <pascal.lecocq@gmail.com>
The comments that explain how to use the OIIO_MAKE_VERSION and related
macros actually were using formatting that would have been rejected by
our own clang-format settings. Change those examples so that they use
the correct formatting that woule be accepted by the project.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I don't know why, maybe some slightly different version of libtiff
rolled out to the GHA runners, but despite no changes to our TIFF
handling in the recent past, just today we started intermittently
getting different output for this test on some of the Linux runners.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Clarify in the documentation that `openexr:dwaCompressionLevel` is a
read-only attribute and cannot be used to set the compression level on
write.
Not sure if this is an obvious change or not, but it wasn't immediately
clear if the attribute is meant to be writable.
Signed-off-by: Aamir Raza <aamir_raza@outlook.com>
Lately, some Mac Intel CI runs have taken forever or failed because of
timeout. It appears to be because some packages are maybe not cached
(bottled) for Intel anymore? It only happens sometimes, so maybe it
fails when the package version has just been bumped, but eventually
somebody published a pre-built bottle and then it's ok?
openvdb and its dependencies seem to be a frequent culprit. Let's just
skip them for this one soon-to-be-obosolete variant.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Most visible change: The TIFF reader and writer now correctly handle GPS
tags.
Fixes#5049
But along the way, and to get it completely right, a fairly extensive
refactoring of our TIFF tag handling was needed. The libtiff behavior
around different tags is extremely convoluted, and there were a bunch of
cases we either didn't handle correctly, or were dropping on the floor.
So there are also some other non-GPS tags that we missed all along but
now read properly.
Note that the support for separate IFDs for GPS tags is only supported
via the API in libtiff 4.2 and newer, so this is disabled when bulding
against older libtiff versions. (Our support of old libtiff versions
extends extraordinarily far back in time.)
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
`oiiotool --compression XXX` just used the argument as a literal, and
wasn't able to use expression substitution for the value. That made it
behave differently than the supposedly synonymous `--attrib compression
XXX` in the case that the argument was an expression rather than a
literal value.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
PR #5036 incidentally added a new TypeURational alias, but I neglected
to make the corresponding change on the Python bindings, to update some
of the docs, or to fully test it. This PR completes those items.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The docs described a syntax where you could use `-d` to change the data
type of all channels in one subimage of a multi-subimage image. But it
didn't work, as revealed in #5047.
Fixes#5047
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Problem identified by @irieger Ingmar Rieger, solves a problem that pops
up when Conan build provides the targets.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I tested out the JPEG XL CICP support and noticed that color primaries
12 was not supported. This pull request is looking to extend P3 support
for color primaries 12.
Note: color primaries 11 uses the DCI white point and color primaries 12
uses the D65 white point.
The JxlPrimaries enum only covers P3 primaries as value 11 and not 12.
See,
https://github.com/libjxl/libjxl/blob/main/lib/include/jxl/color_encoding.h#L55-L75
Further code is therefore required to account for this on read and
write.
Tests for read and write of color primaries 11 and 12 were added.
Signed-off-by: Shane Smith <shane.smith@dreamworks.com>
I was seeing warnings with instantiation of the ispow2 function template
for unsigned type, where the `x >= 0` clause is meaningless. Use a
constexpr if to eliminate that pointless test for unsigned types.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The CI stub generation has been broken for a few days, failing CI every
time. The checked-in stub files seem fine. Just turn off this check
until we can figure out why it is broken.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The previous minimum, 7.0, dated from mid-2020.
We are raising now (in main / future 3.2 only) to 9.0, which dates from
mid-2022, so we're still supporting several versions and/or years back.
Because this changes minimum dependency versions, it will NOT be
backported to release branches (3.1 or earlier).
I had to remove the CI test variant for icc, because ancient icc can't
correctly build newer versions of fmt, it seems. There is a separate PR
to simply drop icc from our list of supported compilers.
If anybody wants to argue for pulling the minimum up even farther (say,
to fmt 10.0, released in 2023, so still supporting 3 years back), which
would simplify even more places, I would consider it.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Intel icc is deprecated and hasn't had a release for a few years. It's
holding us back, both by making us work around an ever growing number of
icc bugs and limitation that will never be fixed, as well as not
allowing us to upgrade minimum versions of certain dependencies, because
icc can't correctly compile newer versions (as an example, it cannot use
a 'fmt' library newer than the oldest we support, 7.0).
So it's time to thank icc for its service and put it on the ice floe for
the polar bears to eat. This is of course in main (future 3.2), and will
not be backported to release branches, since we never stop support of a
dependency or toolchain of existing releases. People requiring icc for
whatever reason may keep using OIIO 3.1 or older.
We will continue to support and test icx, the fully supported Intel
LLVM-based compiler.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* cmake utility build_dependency_with_cmake was unconditionally doing a
shallow clone and using `clone -b`, but that only works if it's got a
branch or tag name, not if it has a commit hash. So change the logic so
it does a shallow clone only if GIT_TAG is specified but GIT_COMMIT is
not.
* pybind11 self-builder is modified to allow a git commit override.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#5044
Oops, the logic was a little mixed up when there were exactly two
images. One reason that this was a special case is that conceptually,
there is just a stack, but the implementation is that there is a
separate variable for the top item, and then the actual stack is all the
other items.
Also add more thorough testing of TOP/BOTTOM, including what happens for
2, 1, and also 0 items on the image stack (errors in that last case).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This allows us to correctly read the ExifVersion and FlashPixVersion
metadata in an EXIF block of a TIFF file.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Implement support for reading and writing monochrome. Reading requires
libheif 1.17+ for heif_image_handle_get_preferred_decoding_colorspace.
Previously writing a single channel image would cause an exception due
to
wrong parameters, but close() would continue writing the image and
crash.
Destroy m_ctx on exception to prevent that for other potential errors.
Test added for monochrome read and write.
---------
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
This is a PR proposing to keep the gamma precision.
In some cases, we need more precise gamma values, while the existing
rounding operation loses most of the precision. This change will
continue to use rounded values to calculate and store color space
information, but retain the original value in the "Gamma" parameter. In
addition, it can also tidy up existing code.
I've verified with png/exif.png & python-colorconfig tests. No
regression is introduced.
Signed-off-by: Lumina Wang <lumina.wang@autodesk.com>
This is a more comprehensive fix for issues discovered in PR #5035.
The original problem reported in Issue #5023 was a crash when writing
TIFF information that was supposed to be arrays of more than one
rational, it was reading past the end of a memory array.
#5035 is a minimal, immediate fix to address the crashes. But in the
process, I saw a number of ways in which we were dropping metadata on
the floor when the types didn't exactly match, but that we *could*
handle with automatic conversion.
The new cases that we handle with this PR are:
* Exif RESOLUTIONUNIT tag is a short, but by convention we store it by
the name as a string in OIIO metadata, so we need to convert back to a
code (we did so for the main TIFF metadata, but not for Exif in TIFF).
* Handle Exif "version" and "flashpixversion" metadata which have
unusual encoding in TIFF files (they are 4-character strings, but must
be stored in a TIFF tag of type BYTES, not as the usual type ASCII that
most strings use.
* Handle things that TIFF insists are ASCII but that come to us as
metadata that's strings. Easy -- our `ParamValue.get_string()`
automatically converts ther things like ints or floats into string
representation.
* Much more flexibility in automatically converting among the signed and
unsigned, 16 and 32 bit, integer types when the metadata in our
ImageSpec is integer but not the specific type of integer that TIFF/Exif
thinks it should be.
This doesn't appear to change the results of anything in our testsuite,
but it's possible that some non-TIFF-to-TIFF image conversions that
contain Exif data may now do certain type conversions properly instead
of just silently dropping the metadata that had non-matching (but
reasonably valid) types.
Additionally, to do this nicely, I ended up adding a new TypeURational
alias in typedesc.h (similar to TypeRational, but the case where both
numerator and denominator are unsigned ints).
And also fixed a random comment typo I noticed in tiffinput.cpp.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Bump the version of 'fmt' library that we download and build (if not
found) from 10.2 to 12.1.
Some other touch-ups in build_fmt.cmake.
Also, we have seen that recent fmt versions will fail to compile on MSVC
unless using the `/utf-8` compiler flag, so ensure that is used and also
passed on to other clients of libOpenImageIO_Util (which expose
templates using those headers).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#5023
This was crashing when writing TIFF information that was supposed to be
arrays of more than one rational, but in fact was provided as a single
value, it was reading past the end of a memory array.
I noticed that this whole region needs a cleanup, this is not the only
problem. But a full overhaul seems too risky to backport, so my strategy
is as follows:
* THIS fix first, which I will backport right away to 3.0 and 3.1.
* I will then submit a separate PR (already implemented and tested) that
is a much more complete fix and overhaul of this portion of the code
(and other places). That will get merged into main when approved.
* After the second PR is merged, I'll hold it in main for a while to
test its safety, and then decide if it seems ok to backport to 3.1 (but
definitely not 3.0).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Extra protections for corrupted BMP files that claim to be palette
images, but have a BPP that doesn't support palette images. Also an
extra guard around accessing the palette array if it is empty.
Add an extra test case for this kind of corruption.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Mac Intel is getting long in the tooth, and quite often the Homebrew
packages for Intel are found to be uncached and will try to build from
source. When it's OpenCV, that's disastrous for our CI build times, it
can get stalled for hours building all of OpenCV and its dependencies.
So disable it for that one build variant.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Review: we have long had two assertion macros: OIIO_ASSERT which aborts
upon failure in Debug builds and prints but continues in Release builds,
and OIIO_DASSERT which aborts in Debug builds and is completely inactive
for Relase builds.
Inspired by C++26 contracts, and increasingly available "hardening
modes" in major compilers (especially with the LLVM/clang project's
libc++), I'm introducing some new verification helpers.
New macro `OIIO_CONTRACT_ASSERT` more closely mimics C++26
contract_assert in many ways, and perhaps will simply wrap C++
contract_assert when C++26 is on our menu.
Important ways that OIIO_CONTRACT_ASSERT differs from OIIO_ASSERT and
OIIO_DASSERT:
* Keeping in line with C++ contracts, there are 4 possible responses to
a failed contract assertion: Ignore, Observe (print only), Enforce
(print and abort) and Quick-Enforce (just abort).
* Also define hardening levels: None, Fast, Extensive, and Debug,
mimicking the levels of libc++. The idea is that maybe there will be
some CONTRACT_ASSERT checks you only want to do for certain hardening
levels.
* By default, the contract failure response is Enforce, unless it's both
a release build and the hardening level is set to None (in which case
the response will be Ignore). But it's also overrideable optionally on a
per-translation-unit basis by setting OIIO_ASSERTION_RESPONSE_DEFAULT
before any OIIO headers are included (though obviously that only applies
to inline functions or templates, not to any already-compiled code in
the library).
* Macros for explicit hardening levels: OIIO_HARDENING_ASSERT_FAST(),
EXTENSIVE(), and DEBUG(), which call CONTRACT_ASSERT only when the
hardening level is what's required or stricter.
I also changed the bounds checking in operator[] of string_view, span,
and image_span to use the contract assertions. Note that this adds a
tiny bit of overhead, since the default is "enforce" for release builds
(previously, using OIIO_DASSERT, it did no checks for release builds).
But the benchmarks seem to idicate that the perf difference is barely
measurable.
I added some benchmarking that proves that the bounds check adds a
minute overhead to an element access for a trivial `span<float>`, maybe
even indescernable. Here are benchmarks comparing raw pointer access,
std::array access, span access with the new checks, span access
carefully bypassing the tests.
Linux workstation, gcc-11, on my work computer:
pointer operator[]: 647.8 ns (+/- 0.1ns)
std::array operator[]: 647.8 ns (+/- 0.1ns)
span operator[] : 657.6 ns (+/- 0.5ns)
span unsafe indexing: 648.2 ns (+/- 0.2ns)
span range : 648.1 ns (+/- 0.1ns)
These are the most stable tests I have, with the least trial-to-trial
variation, and show about a 1.5% speed hit on the bounds-checked span
access itself, which I think will be truly un-measurable in the context
of being interleaved with any other operations that you do with the data
you pull from the span.
Mac Intel, Apple Clang 17, on my (old) personal laptop: (much more
variable timing, probably from MacOS scheduler quirks)
pointer operator[]: 929.2 ns (+/- 6.7ns)
std::array operator[]: 913.1 ns (+/- 20.6ns)
span operator[] : 905.8 ns (+/- 13.3ns)
span unsafe indexing: 913.9 ns (+/- 16.6ns)
span range : 916.4 ns (+/- 20.3ns)
You can see that here there is no obvious penalty, in fact it appears a
little faster, but all within the timing uncertainty of the multiple
trials, so statistically it's hard to discern any penalty.
And a couple more for good measure from our CI, but note that because
these are uncontrolled machines somewhere on the GitHub cloud, the
timings might not be as reliable:
Windows, MSVS 2022:
pointer operator[]: 3716.3 ns (+/- 6.3ns)
std::array operator[]: 3715.5 ns (+/- 3.4ns)
span operator[] : 3715.6 ns (+/- 2.6ns)
span unsafe indexing: 3712.1 ns (+/- 0.7ns)
span range : 3714.2 ns (+/- 2.9ns)
Linux, gcc-14, C++20:
pointer operator[]: 1130.9 ns (+/- 0.2ns), 884.2 k/s
std::array operator[]: 1132.0 ns (+/- 0.4ns), 883.4 k/s
span operator[] : 1133.7 ns (+/- 0.4ns), 882.1 k/s
span unsafe indexing: 1134.2 ns (+/- 1.6ns), 881.7 k/s
span range : 1133.9 ns (+/- 0.7ns), 881.9 k/s
MacOS ARM:
pointer operator[]: 3456.6 ns (+/- 7.5ns)
std::array operator[]: 3466.8 ns (+/- 12.2ns)
span operator[] : 3610.9 ns (+/- 11.0ns)
span unsafe indexing: 3607.4 ns (+/- 4.9ns)
span range : 3612.4 ns (+/- 12.2ns)
Windows with MSVS and Linux with newer g++ don't appear to show any
penalty, and the bracketing of trial times indicates that maybe it's
consistent enough to be meaningful? I can't think of anything I'm doing
wrong here that would throw off the timing or disable the range checking
on these tests.
For MacOS ARM, the span looks like it has about a 4% penalty versus raw
pointers? But OTOH, span bounds-checked vs non-checked vs range-for are
all the same, so maybe the speed vs raw pointer is something else
entirely?
Also please note that a preferred way to avoid these extra bounds checks
entirely is to change an index-oriented loop like
span s;
for (size_t i = 0; i < s.size(); ++i)
foo(s[i]); // maybe bounds check on each iteration?
to a range based loop:
span s;
for (auto& v : s)
foo(v);
which should be inherently safe and require no in-loop checks at all.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Since the OpenImageIO 2.5 series, when calls to `check_open` were added,
any format that did not declare support for "tiles" would immediately
fail to open. But many of the formats which attempted to emulate tiles,
by buffering the contents and writing it all as scanlines at the end,
were not updated. All of the tile emulation code for these formats is
effectively dead-code and untested.
Remove the tile emulation code from these formats.
An example of what the failure currently looks like:
```python
>>> out = oiio.ImageOutput.create("test.png")
>>> spec = oiio.ImageSpec(64, 64, 3, 'uint8')
>>> spec.tile_width = 64
>>> out.open("test.png", spec)
False
>>> out.geterror()
'png does not support tiled images'
```
No tests were impacted.
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
Need to test some MSVS-specific macros to determine what architecture to
report.
And especially, if it doesn't know the processor architecture, it still
should be *appending* that to the platform, not replacing it! This
caused MSVS-compiled OIIO on Windows to report "unknown arch?"
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I think it was basically harmless, since we do all the metadata name
comparisons using case-insensitive comparisons. But we use "Exif:" as
our prefix for Exif data throughout OIIO by convention, and there was
this tiny handful of places where we said "exif:".
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Even though we have CI testing on Mac with ARM CPU that were passing,
after getting a new laptop, I saw some test failures that were due to
just a few pixels on a few tests needing a higher comparision threshold.
Results are correct, just different due to the math. I guess this
machine (CPU? build flags? specific compiler or library versions?) is
ever so slightly different than the CI Macs, so I caught a few more
instances that needed to be adjusted.
I tried to increase the thresholds as little as possible to fix the
problem.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#4715
- Added save functionality
- Added flipping and rotation capabilities via metadata.
- Hotkeys: Ctrl-Shift-R and L to rotate the image left and right.
Note that the actual pixel data remains unchanged, so only image formats
that support the rotation metadata will reflect the changes.
I thought that it made sense for the flip and rotate to be under the
tool section, similar to where the macbook Preview app places it.
Keyboard shortcuts also follow the shortcuts that Preview uses.
Tests
Used a .jpeg photo to test:
- saving
- making changes with/without saving and re-opening the image
- flipping and rotation from different initial rotation/mirrored states
Otherwise there doesn't seem to be a test suite for iv?
Signed-off-by: valery <71804886+vangeliq@users.noreply.github.com>
There's something in pybind11 master at the moment that is crashing in
its destructors. It's been causing our "bleeding edge" test to fail for
over a week now. I'm tired of our test failing, so I'm locking down to
the last known working version. Will check back periodically and return
to testing against pybind11 master after they have fixed it.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Makes OpenColorIO 2.5.1 the default version when building OCIO locally.
This will cause the released python wheels to build against the
currently-latest version of OCIO.
Signed-off-by: Zach Lewis <zachcanbereached@gmail.com>
Like other file formats, the returned ImageSpec should indicate if the
image contains unassociated alpha.
This was an oversight in #4770.
Test added.
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
Add IOProxy support similar to other file formats.
MyHeifWriter was renamed for consistency with other code.
All input and output now goes through the proxy, so this is covered by
existing tests.
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
Utilities to ask the IB for the local pixels as an untyped span of
bytes. This is a "safe" alternative to `localpixels()`, which just
returned a single pointer, instead returning an image_span that
understands the sizes and strides of the buffer.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Nearly all IBA functions take an optional parameter controlling the
threading. But make_texture() did not, so there was no way to control
its thread usage (it would always use the default number of threads).
Without changing the call signature or ABI, this patch merely makes
make_texture honor any "maketx:threads" hint passed in the config
ImageSpec that it already takes to convey all sorts of controls over the
texture creation process. Then this value is passed to any IBA
functions, use of parallel_image, and anything else in the
implementation of make_texture that would end up using the thread pool.
Fixes#4254
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The WebP format is very limited in the maximum resolution it supports.
We need to change the values we allow to be much smaller.
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
Initializing the encoder to `heif_compression_HEVC` by default throws an
exception when libheif was built without HEVC support, which prevents it
from being used entirely even if there is AVIF support.
So delay that initialization until we are sure which encoding we want.
Not really any practical way to automatically test this.
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
We previously merely *marked* tests as "broken" in this case, and
depended on ctest being launched with `-E broken` to ensure those were
skipped. Not everybody did. Let's just truly skip them.
Fixes#4979
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Needed to change the Libheif::Libheif target we set up ages ago to the
"heif" name that the package's exported config uses.
Also, need to give the warning about static libraries occur any time we
are using static libheif, even if we didn't set LIBSTATIC to try to
force all static libraries.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* ImageBuf internal buffer span lacked correct chansize. The internal
`m_bufspan` is an `image_span<byte>`, and as such, it needs to remember
the size of the original data type. Otherwise, there's a cascade of
potential errors when it thinks that the individual values are byte
sized.
* In both ImageInput and ImageOutput, several sanity checks of
image_span size versus expectations were incorrect. They were only
checking if the total byte sizes matched expectations, but they are
allowed to disagree when you consider type conversions (in which case,
it's the total number of values that need to match, not the total byte
sizes.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Yay, I love how these things just break with no notice.
Fixes CI that broke a couple days agi.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Needed for some systems after the changes of #4963.
Ever so slightly different LSB somewhere makes the hashes not match, but
it's a correct visual result.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Due to typos in the option name in compiler.cmake (HARDENING vs
OIIO_HARDENING, oops), I think we were never really setting the intended
compiler flags. Fix that all up, and repair the other problems that it
revealed -- some compiler version and option combinations weren't happy
with each other, etc.
One notable change is in encode_iptc_iim_one_tag: I think I made it
safer by taking an early out for 0-sized data, and also it needed a
warning suppression when certain gcc hardening levels were used.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The Intel MacOS 15 CI testing is getting dicier... lots of times,
Homebrew doesn't have cached versions of updated packages, so it tries
to build from source, which takes forever. The big culprit today is Qt.
So, basically, just on this one CI job variant, don't ask it to install
Qt. If it's there, it's there. If not, just skip it. It's tested plenty
in other variants.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* CI test vs the latest freetype 2.14.1
* Bump the version of freetype that we auto-build to the latest (from
2.13.2)
* Simplify BZip2 finding logic, switch to using targets
Signed-off-by: Larry Gritz <lg@larrygritz.com>
For IBA::resample() when bilinear interpolation is used, almost all of
the expense was due to its relying on ImageBuf::interppixel which is
simple but constructs a new ImageBuf::ConstIterator EVERY TIME, which is
very expensive.
Reimplement in a way that reuses a single iterator. This speeds up
IBA::resample by 20x or more typicaly.
Also refactor resample to pull the handling of deep images into a
separate helper function and out of the main inner loop. And add some
benchmarking.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Starting with 1.21, libheif seems to change behavior: When no CICP
metadata is present, libheif now returns 2,2,2 (all unspecified) on
read. OIIO convention, though, is to not set the attribute if valid CICP
data is not in the file.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Implement RLE compression support for the SGI output plugin. Reading RLE
encoded images was already supported, but writing was never done up
until this point.
The existing sgi test seems sufficient to catch issues and it covers
input/output of both 1 byte-per-pixel and 2 byte-per-pixel files.
The documentation for the image plugins are sometimes not very clear
about which attributes are relevant for input vs. output. There's
usually 3 sections: Attributes, Attributes for Input, and Attributes for
Output.
Before this PR, SGI mentioned the "compression" attribute in the
"general" Attributes section (rather than say just the Input section),
which caused a bit of grief as the only way to discover that RLE was not
implemented for Output was to glance at the file size of the resulting
file... I had assumed that compression was supported for output too but
discovered that it was not.
Now that this PR implements the attribute for output I've left the
documentation as-is in the "general" Attributes section since it applies
to both read/writing now. But I'm open for suggestions here.
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
Rearrangements in 3.1 dropped the list of recognized attributes from
the visible online docs and failed to document the span varieties. We
fix and also reword a lot of the descriptions for clarity and uniformity.
The previous organization was that there were several varieties of attribute(). In the header, the first one had the overall long explanation, including the list of all the recognized attributes. The other ones had short explanations of how they differed. In the docs, each one was referenced explicitly, pulling in its attendant bit of documentation.
What really happened is that in the header, I made the new span-based version the "flagship" one with the full explanation, but I neglected to reference it in the docs, so the long description disappeared.
I could have fixed by just adding refs to the new functions to the docs, as I originally meant to. But while I was there, I took the opportunity to surround the whole collection with a group marker, and then include the lot of them with a single reference to the group, rather than need to refer to each function variant individually. And while I was at it, I also reworded (and hopefully improved) some of those explanations.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The JPEG XL color encoding metadata only supports a subset of CICP. So
for
example `srgb_p3d65_display` and `pq_rec2020_display` are supported, but
`g26_xyzd65_display` is not.
Custom primaries, custom white point and arbitrary gamma could be used
to
support more, but I didn't implement that.
Tests for read and write were added.
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
If the colorspace exists and has an interop ID in an OCIO 2.5 config,
use that.
Otherwise check if the colorspace is equivalent to a known color interop
ID.
Tests were added.
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
Periodic revisiting of the phrasing in the hopes of improving PR and
Issue quality / actionability. Not that I think they are bad now, just
always looking to guide people to improvements.
Will it work? 🤷 Let's try and see if we think PRs improve. If not, we
can revise again later.
Also make the issue title prefixes less yelly and shorter (makes it
easier to see the actual title on phone or other limited width display
of messages).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Each file writer that is capable of writing arbitrary metadata is
supposed to tace care to suppress hints meant for other writers, or for
OIIO in general, not interpret them as literal metadata to write to the
file. PNG was not doing so, and ended up writing hints as literal
metadata. This brings it in line with our behavior when writing OpenEXR
and other formats.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The GaussianPyramid::value() method used by IBA::compare_Yee() was
incorrectly accessing channel 1 instead of channel 0. This fix corrects the
channel access to properly retrieve the luminance value from the
single-channel pyramid levels.
One oddity this led to is that `idiff -p` would appear to PASS for
some clearly different images if they had different data windows.
Update reference outputs to reflect the corrected perceptual diff results
(Max error changes from 1 to 10 for the test images).
Fixes#4948
Signed-off-by: pmady <pavan4devops@gmail.com>
The default Ubuntu images keep shifing around. Last week, some packages
that used to be installed disappeared. Need to adjust what we install in
gh-installdeps.bash, and slightly adjust the optional deps in ci.yml to
allow some things that seem to be only on some runners to be missing.
Also bump the clang test to clang18 on an ubuntu-24 runner (from clang
15 on ubuntu 22).
Gets us back to all-passing CI after a week of all sorts of failures in
tests that used to work.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
If we don't specify anything, it seems that Ptex will think it's using
C++98, which it is in fact not capable of doing.
Meanwhile, since OIIO's own minimum requirement is C++17, it certainly
meand that any compiler we're using can support 17 as a minimum when
building Ptex as well.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
There seems to be a couple seconds overhead for each "brew install".
Refactor install_homebrew_deps.bash to first assemble the list of all
packages to install, and then do a single `brew install` for the whole
list.
Also remove mention of packages that are pre-installed on the Mac GHA
runners or that we don't need.
In the process, by using the `OIIO_BREW_INSTALL_PACKAGES` variable and
only setting it if empty, we now have the ability to fully override the
list of packages that brew will install on a per-job basis.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
On Windows, `Filesystem::exists` forward the call to
`std::filesystem::exists`, and finally calls `_std_fs_get_stats` .
`_std_fs_get_stats` is slow if the argument is a netwok path.
Signed-off-by: Jackson Sun <jackson.sun@autodesk.com>
When reading image files with CICP metadata, automatically set the
corresponding "oiio:ColorSpace". When writing files that support CICP
and no other colorspace metadata can represent "oiio:ColorSpace",
automatically write CICP metadata.
Setting "oiio:ColorSpace" on read prefers scene referred over display
referred color spaces, changing existing behavior as little as possible.
The alternative would have been to interpret the presence of CICP
metadata as an indication that the image is likely display referred,
which might be reasonable too. Either way it's a guess.
There is no automatic mapping from `g22_rec709_display` to CICP
currently to keep behavior unchanged, as this is often not what you want
and further discussion is needed to decided on the right behavior.
Also add new ColorConfig `get_cicp` and `get_color_interop_id` API
functions to share logic between file formats.
## Tests
Tests were updated, the auto detected color space name appears in the
output.
Commands like these should also do the right thing automatically.
```
oiiotool in.exr --ociodisplay "Rec.2100-PQ - Display" "ACES 2.0 - HDR 1000 nits (P3 D65)" -o out.avif
oiiotool out.avif --autocc -o out.exr
```
---------
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
Move write_complete_data() to close() for proper scanline order support
The WebP output plugin incorrectly assumed that scanlines would be
written in sequential order, triggering the final write when y == height
- 1. This violated the write_scanline API contract which allows
scanlines to be written in any order (the plugin even advertises
'random_access' support).
Changes:
- Remove premature write_complete_data() call from write_scanline()
- Move final encoding/writing to close() where it belongs
- Add m_image_complete flag to track write state
- Move buffer cleanup to after write_complete_data() in close()
Fixes the issue where writing scanlines out of order would result in
incomplete or corrupted WebP files.
Signed-off-by: pmady <pavan4devops@gmail.com>
- Mount the host's root under /host/root so it's visible in the
containers.
- Remove stuff we don't need to free lots of disk space.
- While we're at it, arrange for the commands (copied from OSL) that
allow the containers to see a CPU, when available.
This is freeing 20+ GB!
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I think something changed with Homebrew on the runners, this worked
until last week.
Emergency fix, will merge immediately if CI finally passes.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The EXR reading code was correct. These attribute names are case
sensitive in OpenEXR.
Test output was updated.
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
Was just wrong. Presumably broken recently.
In the process, switched internal cmyk->rgb conversion to be span based.
Fixes#4962
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We were supporting Exif 2.3, but in 2024, Exif 3.0 was published and it
adds some more tags for us to recognize.
The only part I am not sure about here is that the ExifVersion we output
is still marked as 2.3, not 3.0, because I'm worried that if we mark it
as 3.0, older software reading the images will reject it. Let's see what
happens. If downstream apps care, maybe we can bump to 3.0, or even bump
to 3.0 only if any of the 3.0 tags are used (that is more involved).
Fixes#4960
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Result of our annual license scan.
* Add license notices some places we had missed (mostly documentation).
* Double check some of the code files that are still marked as a mix of
Apache 2.0 + BSD, see if there are any remaining lines that are from
authors that hadn't relicensed their contributions -- there were a few
that had none of those old lines left, so they can be changed to
advertise that they are purely Apache 2.0.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
A few days ago, the icc and icx CI tests started failing, saying they
were out of disk space while installing dependencies (specifically, the
intel compilers themselves).
Now, the VFX Platform 2023 asf docker image I'm using hasn't changed,
and neither has the years-old Intel rpms with the compilers. So maybe
the runners themselves just changed again, in a way that adds some
pre-installs and leaving us not quite enough disk space for everything
else we need?
Asking for a few less things to get installed seems to free up just
enough space. And for the icx one, I also swtiched to the 2025
container, which let me use a slightly newer icx version.
Also: Don't try to install giflib needlessly, which speeds up the setup
time. And remove some dead code in my dependency setup script.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Recently, when building OIIO using `OpenImageIO_BUILD_LOCAL_DEPS=all` cmake option, I noticed a reference to `lerc` that fails during the linker stage, when compiling the simd tests:
```
[ 71%] Building CXX object src/libutil/CMakeFiles/simd_test.dir/simd_test.cpp.o
[ 71%] Linking CXX executable ../../bin/simd_test
/usr/bin/ld: ../../lib/libOpenImageIO.so.3.1.7: undefined reference to `lerc_decode'
/usr/bin/ld: ../../lib/libOpenImageIO.so.3.1.7: undefined reference to `lerc_encodeForVersion'
/usr/bin/ld: ../../lib/libOpenImageIO.so.3.1.7: undefined reference to `lerc_getBlobInfo'
collect2: error: ld returned 1 exit status
gmake[2]: *** [src/libutil/CMakeFiles/simd_test.dir/build.make💯 bin/simd_test] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:2158: src/libutil/CMakeFiles/simd_test.dir/all] Error 2
gmake: *** [Makefile:166: all] Error 2
```
`lerc` is an image compression library from ESRI: https://github.com/Esri/lerc
A deeper dive of this reference to lerc shows that it is referred in `libtiff`. `libtiff`, however, did not seem to correctly export this dependency to OIIO when being built locally.
If someone was building OIIO with a system-installed `libtiff`, this error is unlikely to happen.
By disabling `lerc` in `libTiff`, the local build scripts ( `OpenImageIO_BUILD_LOCAL_DEPS=all` ) runs without a problem.
Tested build OIIO with `OpenImageIO_BUILD_LOCAL_DEPS=all` with this patch and was successful in finishing the build.
In exr output, we were doing attribute name substitutions using
case-insensitive comparisions, whereas on the exr input side, we were
using case-sensitive comparisons. This was leading to some strange
behavior where certain attributes were not properly "round tripping"
when reading and writing exr files, particularly when it involved
attribute names that differed only in case from certain
reserved/standard names.
Also be a little more flexible in adjusting exr attributes that have the
wrong type, by being able to convert int or float to a value that's
required to be a string.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Two Ubuntu-based cases (debug build and static build) started failing
this past week, unrelated to us. I think the runners changed yet again,
it seems to be having particular trouble because those tests were trying
to use gcc-9. But there's no particular need for that anyway, it just
reflects how long ago we last modified those tests.
So this spurred a grab bag of CI fixes:
* Move the debug and static linkage tests off of raw ubuntu runners and
to using the ASWF docker VFX Platform 2025 containers. This makes them
more modern and also more stable.
* Turns out the "static" test hadn't been working properly lately -- it
was accidentally building dynamic libs, not static at all! Fix that, but
then it uncovered some problems that are out of scope to fix in this PR.
I will look into it separately, but in the mean time I have commented
out the static build test to suppress the failures.
* Bump a few of the "latest" dependencies to the most recent.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Almost all the other unit tests do this, just adding it to
image_span_test: automatic reduction in test load for CI and debug
renders by doing fewer time trials. This cuts down on runtime for CI,
but stil exercises the test code. Command line arguments allow you to
override, in cases where you need really good timing fidelity even for a
debug run.
Also conform simd_test to the usual standard `-iters` used elsewhere
instead of being the one place that uses `-iterations`.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I think that in a recent checkin, we accidentally made some ref outputs
for this test be identical, that were supposed to be for when libheif is
built without avif support.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Ugh, used the wrong option name. Also, it's sufficient to just do it
with the env variable, don't need extra logic in ci-build.bash.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Closes#4791
This PR introduces the `oiio:ACESContainer` hint for OpenEXR outputs.
The hint can take one of the following values: `none` (default),
`strict`, or `relaxed`. If not `none`, the spec will be checked to see
if it is compliant with the ACES Container format defined in [ST
2065-4](https://pub.smpte.org/pub/st2065-4/st2065-4-2023.pdf). If it is,
chromaticities will be set to the ACES AP0 ones, and the
acesImageContainerFlag attribute will be set to 1. In `strict` mode, if
the spec is non-compliant, the output will throw an error and avoid
writing the image. While in `relaxed` mode, if the spec in
non-compliant, only a warning will be printed and the attributes
mentioned above will not be written to the spec.
I've added several tests under `openexr-suite`. These use oiiotool to
attempt writing several EXRs with the `oiio:ACESContainer` hint.
---------
Signed-off-by: glowies <self@oktaycomu.com>
Signed-off-by: Oktay Comu <self@oktaycomu.com>
## Description
This PR fixes and improves a handful of wheels-related items:
- Fix an issue where wheels produced with v3.1 releases were not getting
uploaded to pypi
- Update cibuildwheel to latest version (v3.2.1)
- Remove the manylinux2014 builds, replace with manylinux_2_28 builds
where applicable
- Add CPython 3.14 wheels
- Update macos-x86 runner to `macos-15-intel`
- Update Python typestubs
## Notes
- This PR ends support for pre-built wheels for glibc < 2.27 (CentOS-7
etc). However, the module can still be pip-installed locally, provided
devtoolset-7+ (e.g., gcc-9+) is available.
- Unless we arrive at a more elegant solution, we must remember to
change the conditional in the "upload_pypi" step for every major OIIO
release.
---------
Signed-off-by: Zach Lewis <zachcanbereached@gmail.com>
Due to apparent limitations or bugs in swscale, there is a problem
reading the 16 bit green channel that is not 4 byte aligned like red and
blue. Reading each channel into its own plane solves the issue.
This also addresses an existing comment about 32 bit float being read as
16 bit. This needed the same mechanism of reading separate planes, so
the same code is reused to implement that.
Resolves#4901
I added a 10 bit test file encoded with VP9 and Matroska, same as the
existing 8 bit test file to maximize compatibility. I verified this now
converts correctly to a HDR PNG file.
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
A small portion of time -- maybe 10%, just enough for many CI workflow
runs to have a single job fail -- our attempt to `dnf install ffmpeg`
fails.
I thought maybe it was an intermittent, transient failure. But looping
to try multiple times with delays in between does not help. A job that
fails once either succeeds initially or fails over and over. I'm now
leaning toward the belief that rather than a transient failures, there
are just a subset of GHA runners that will always fail, and the
semi-random nature of our job failures is just random luck of the draw
in runner selection.
Anyway, so I'm not going to merge the try-in-a-loop approach. Instead,
this PR just makes ffmpeg an optional dependency, so the failures to get
the dependency won't fail the OIIO build and therefore the CI. It seems
like most of the time we really do get the dependency, but I'm out of
better ideas at the moment.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Support for retrieving and setting the idManifest for deep image files
with object ID channels. This is an OpenEXR feature where the image
header can contian metadata that maps all the object ID channel integer
codes to strings of the object names.
We report this in the ImageSpec as the metadata
"openexr:compressedIDManifest", whose type is an array of uint8 (byte)
values. It is encoded the same way it appears in the exr file itself:
the first 8 bytes are a little endian uint64_t giving the *uncompressed*
size of a serialized OpenEXR IDManifest object, and then starting at
byte 8, the zip-compressed seriailized IDManifest. Therefore, the
compressed block size is the size of the metadata blob, minus 8 bytes
for the length. It is up to the caller to use the OpenEXR APIs to turn
this compressed serialized IDManifest into a fully expanded IDManifest,
if that's what they want to do.
It works for output, too, so it should copy the manifests correctly
between files, even if the app doesn't know what to do with the binary
data it contains.
Also added support for deepImageState, which we previously had ignored.
We report this as the metadata "openexr:deepImageState", a string, which
if present has one of the values "messy", "sorted", "non_overlapping",
or "tidy".
Signed-off-by: Larry Gritz <lg@larrygritz.com>
IffInput::read_native_tile was simply incorrect for images with nonzero
data window origin. Fix!
Also switch the forumulation to use spancpy rather than memcpy, to be a
little more careful that we are't overwriting the presumed sizes of the
buffers.
And while I'm in there, I realized we can hold the mutex for less time.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The CMake (or env) variable OpenImageIO_BUILD_MISSING_DEPS enumerates
dependencies to try to locally auto-build if not found. If set to "all",
it will build them all.
This patch also allows you to set it to "required", which will try to
auto-build any missing dependences that are required, but not bother
with non-required missing packages.
It also works to use required and also have list items naming additional
optional dependencies, such as
-DOpenImageIO_BUILD_MISSING_DEPS="required;WebP"
build all required dependecies, and also WebP (which is optional), if
they are no found already installed.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Two places where the docs and header for ImageBuf::set_pixels did not
match.
In one case, the docs were right, the declaration was supposed to have a
parameter that was a `const image_span<T>&` but the `&` was left out.
Now, if this was an actual linkable function, we wouldn't be able to fix
it until a compatibility-breaking boundary. But because it's a template
and inline, I believe it is actually safe to fix it in situ without
breaking ABI compatibility (and it seems to pass our ABI check in CI).
In the second case, the header was correct but the docs used the wrong
declaration (leaving out a `const`) and made the docs fail to pull in
the right explanation.
Fixes#4923
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#4641
Has something to do with mixing Windows compiler versions, there's a
subtle ABI compatibility issue that this sidesteps.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Building the python wheels takes a long time! I really hate waiting for
that when testing PRs, there must be a way to speed it up.
* For the wheel workflow, add cache actions to save and restore the
CCACHE_DIR. Hey, it's no easy feat to figure out what the path should be
to that directory, especially on Linux where the wheels are built in a
container, so the paths inside the container (where the wheel is built
and the C++ compilation happens) don't match the paths outside the
container (where the cache restore and save actions execute).
* I had a heck of a time on Linux trying to get a pre-built ccache
installed and had to resort to writing a bash script to build ccache
itself from scratch, which is much more expensive than I'd like, but
we'll have to come back to fix that separately.
* Changed our "auto-build" utility build_dependency_with_cmake to print
the amount of time it takes to build each dependency.
* When auto-building, pass along CMAKE_CXX_COMPILER_LAUNCHER so that the
dependenencies also for sure use ccache.
* Use CMAKE_BUILD_PARALLEL_LEVEL on the wheel run to use all the cores
and compile in parallel. (We did that on the regular CI but I think not
for the wheel building.)
* Fixes to the logic in our compiler.cmake where it tries to use ccache
even if the magic CMAKE_CXX_COMPILER_LAUNCHER isn't set -- I have come
to believe we were doing it wrong before, it was having no effect, and
all along we only got ccache working on CI because we *also* set the env
variable.
* For CI, set CCACHE_COMPRESSION=1 to make the caches take less space
against the precious limit of how much cache we can use total on GHA.
So, the result of all this:
**Previous times (typical), and first wheel run for any git branch**
| platform | total | compile OIIO + deps |
| ----------- | ----- | ------------------- |
| Linux Intel | 10:35 | 500s |
| Linux ARM | 7:20 | 294s |
| Mac Intel | 20:18 | 1146s |
| Mac ARM | 7:19 | 388s |
| Windows | 14:00 | 759s |
**With ccache active, 2nd or more wheel run for a git branch**
| platform | total | compile OIIO + deps |
| ----------- | ----- | ------------------- |
| Linux Intel | 3:33 | 98s |
| Linux ARM | 3:34 | 83s |
| Mac Intel | 5:01 | 212s |
| Mac ARM | 2:30 | 95s |
| Windows | N/A | not using ccache |
The "compile OIIO + deps" column is the isolated time to build OIIO and
also any auto-building of dependencies from source that we do. But it
does not include any other setup, such as 40-60s of container setup on
Linux, 20-40s setting up Python on Mac, or -- ick! -- 45-60 seconds to
build cmake itself from scratch.
So this is considerably better, speeding up the full wheel workflow by
2-4x on all platforms but Windows. Remaining room for improvement
(possibly in subsequent PRs, hopefully not necessarily by me):
* Is there a way to use ccache on Windows? I'm not sure if there is,
when using MSVS.
* Find a way to install pre-built binaries on Linux rather than building
ccache from scratch, which would save almost a whole minute per job.
* Organize each platform to build all of its wheels (i.e. all of the
python versions we are building for on that platform) in a single job
rather than as completely independent jobs, allowing (a) the fixed
per-job overhead such as container initialization, and installing
python, and building of certain dependencies to happen ONCE per platform
instead of separately for each wheel, and (b) the magic of ccache to
speed up the builds *across* those wheels, since only a tiny amount of
OIIO source codes depend on python at all.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#4387
Added build_GIF.cmake module for GIF library dependency management.
Since GIFLIB does not provide a CMakeLists.txt, I created a custom one
that gets added to the cloned repository.
The flow is similar to other libraries with cmake support, so I added a
conditional statement in the build_dependency_with_cmake macro that, if
a cmakelists.txt path is provided, will add/replace existing
CMakeLists.txt files on the cloned repository.
Signed-off-by: Valery Angelique <71804886+vangeliq@users.noreply.github.com>
Version 3.1 removed this definition, thinking it was safe because we
didn't use it anywhere, didn't document it, and what it expands to
(`using namespace OIIO`) isn't any longer than the macro name, so is
pointless to use.
But downstream projects did use it. Oops.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
A test needed updating because the set of color spaces in the built-in
config are different than for 2.4.
Fixes#4915
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Raise the version we "autobuild" for some packages:
* OpenJPEG 2.5.2 -> 2.5.4
* pybind11 3.0.0 -> 3.0.1
* libtiff 4.6.0 -> 4.7.1
* webp 1.4.0 -> 1.6.0
* libjpeg-turbo 3.0.4 -> 3.1.2
And some minor debugging and other aids along the way:
* After a build, on linux, do a `ldd` to verify what libraries are
linked dynamically for the logs.
* Enhance build_dependency_with_cmake macro to allow optional commands:
GIT_COMMIT, GIT_SHALLOW, QUIET
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Two small fixes to compile on musl libc systems.
The first commit is about the use of `__WORDSIZE`, which is not a
standard define and is defined another header in musl libc. I decided to
fix this by following what was done in `libdnf` in [this
commit](https://github.com/rpm-software-management/libdnf/pull/1159/commits/82a07dbe6e53fccfe3a8f8e0d951618e0aa72999).
However, there are other ways, such as including `bits/reg.h` or
`bits/user.h`, which define `__WORDSIZE` in musl libc, or using
`__LP64__` to check for 64 bit instead (inspired by [Alpine Linux's
patch](https://gitlab.alpinelinux.org/alpine/aports/-/blob/0f71e9504b59049e0ec1abb486ba455ab2650c83/community/openimageio/0001-fix-compile-error.patch)):
```c
#if defined(__GNUC__) && (__WORDSIZE == 64 || (defined(__LP64__) && __LP64__)) && !(defined(__APPLE__) && defined(__MACH__)) || defined(__NetBSD__)
```
The second commit is about the use of the `feenableexcept` function.
This function is only defined by glibc. The original code assumed all
Linux systems use glibc (checking `__linux__` define), so I just changed
it to check for glibc directly (by checking the `__GLIBC__` define).
I have only tested it on my musl libc system. I think just compiling it
on different systems is enough of a test in this case...
---------
Signed-off-by: omcaif <103504667+omcaif@users.noreply.github.com>
Fixes#4799
When trimming an image that contains an empty subimage, the resulting
ROI unexpectedly included the top-left pixel even if zero in all
subimages.
To avoid this, empty ROIs are ignored when computing the overall
non-zero region.
If all subimages are empty, the first subimage ROI is used and doctored
to be 1 pixel.
Two tests have been added to the oiiotool testsuite:
- 1 to check the trimming of an empty image
- 1 to check the trimming of a non-empty image with an empty subimage
Signed-off-by: Carine Touraille <carine@raynault.com>
Closes https://github.com/AcademySoftwareFoundation/OpenImageIO/issues/4752
Add Python type hints into documentation, in places where example code
defines a function. That ended up being just
`src/doc/pythonbindings.rst`. I also updated `testsuite/runtest.py` and
the functions in `testsuite/docs-examples-python/src/docs-examples-*.py`.
While I was in `pythonbindings.rst`, I noticed a couple places with
`py::method` (note the double colon). These lines didn't render in the
docs, so I removed the extra colon to match other usage.
I locally built the docs to make sure html looks good. I ran the tests
with `ctest` and the tests ran, so I'm taking that as a sign that
`runtest.py` still works.
Signed-off-by: Connie Chang <connie.chang@dreamworks.com>
We recently had a problem where our use of a dependency was broken, and
it had gone unnoticed because none of our CI jobs actually had the
dependency present and tested it. This can easily happen because so many
of our dependencies are "optional" -- our build will just give a warning
and silently disable the functionality that would have been supported by
the missing dependency.
We have long had build-time options `OpenImageIO_REQUIRED_DEPS`, to
consider specific dependencies (or "all") required even if they would
ordinarily be optional, and `OpenImageIO_OPTIONAL_DEPS`, to make
exceptions. But we didn't use these in CI.
So this PR makes sets `OpenImageIO_REQUIRED_DEPS=all` to make all
dependencies ostensibly required, and then list all exceptions
explicitly. This should make it much more difficult in the future to
make a mistake where use of a dependency is completely untested in our
CI without our being aware of it. And it gives us a visible "hit list"
of untested or under-tested dependencies to slowly whittle down.
Some changes that came long for the ride:
* checked_find_package: explicitly disabled packages are treated as
optional.
* cuda_macros.cmake: Only look for CUDA on platforms that might
conceivably have it (i.e., don't even look on Mac).
* Separate linux-aswf from linux-ubuntu into separate job groups,
because that makes it easier to have shared commonalities of which
dependencies they test.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
On Windows, using a wildcard input path containing `\` back slashes
would lead to failures because it would be compared with file paths
containing `/` forward slashes. Use the existing
`Filesystem::generic_filepath` API (which calls `path::generic_string`)
to get a normalized path containing consistent directory separators.
An example of the existing failure:
```
.\oiiotool.exe -v t:\render\foo_#.png --resize 160x90 -o t:\smaller_#.png
oiiotool WARNING : No frame number or views matched the wildcards
oiiotool ERROR: read : File does not exist: "t:\render\foo_#.png"
Full command line was:
> oiiotool.exe -v t:\\render\\foo_#.png --resize 160x90 -o t:\\smaller_#.png
```
Workaround - without this PR you would have to use an input path
containing `/`:
```
.\oiiotool.exe -v t:/render/foo_#.png --resize 160x90 -o t:\smaller_#.png
Reading t:/render/foo_0001.png
Writing t:\smaller_0001.png
Reading t:/render/foo_0002.png
Writing t:\smaller_0002.png
```
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
These additional things need not trigger a CI test build of the online
documentation:
* Build scripts
* Markdown docs that aren't part of the documentation we're generating
(like the main README.md, INSTALL.md, etc.)
Doing so just makes CI take longer and wastes resources.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We recreated the SonarCloud project to clear the 404s. This commit fixes
the workflow condition by comparing to the string '1', so the Sonar scan
step runs and uploads results.
Updates the GHA workflow guard so the Sonar scan actually executes:
- Change `if: inputs.sonar == 1` → `if: inputs.sonar == '1'` because
`inputs.sonar` is a string (from `workflow_call` inputs).
No unit tests added (CI-only change). Validation: run the “Analysis”
workflow and confirm SonarCloud receives results.
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>
PR #4875, switched to using openjph's exported cmake config instead of
our own FindOpenJPH.cmake, and in the process also renamed OpenJPH ->
openjph to follow their convention.
But I botched it, still using the old OPENJPH_LIBRARIES (etc.)
variables instead of fully switching to the correct targets exported
from their config.
This PR minimally fixes that error, re-enabling use of openjph.
Fixes#4893
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The version of open that takes an array of subimage specs didn't use
them properly for finding the FramesPerSecond hint (or the ioproxy!).
Fixes#3716
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This adds thumbnail support to the raw input plugin.
Fixes#4107
This supports 3 of the 4 formats of thumbnails Libraw provides: BMP,
JPEG, JPEGXL(untested). H265 is not supported, I don't know of any
camera using it, and expect the code being significantly different from
the other formats, given that the H265 is a movie container.
One thing I don't like about this implementation - I understand there is
no way to query the number of thumbnails available in the file? We are
expected to iterate, until `get_thumbnail()` returns false? If
`get_thumbnail()` returns false not because there is no thumbnail for
the requested index, but any other reason, we are not going to try
reading past that index.
## Tests
I have tested manually with a bunch of raw images having either JPEG or
BMP thumbnails.
I have not tested the JPEGXL branch, as I'm not aware of any cameras
using that for thumbnails. The code is identical to the JPEG branch
though, so I'm not expecting any issues with that.
I couldn't see any unit tests around the thumbnail functionality at all.
Should I add some? I assume we can't use `oiiotool` for this?
---------
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
PR #4880 checked in some output reference that contained version
numbers, which passed in main but failed when backported to 3.1. This
repairs it and makes the output suitable for any version that contains
the patch.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fix SonarCloud analysis failures by switching the C/C++ configuration
from the
deprecated `sonar.cfamily.build-wrapper-output` property to the current
`sonar.cfamily.compile-commands` property.
**Changes**
- Remove `sonar.cfamily.build-wrapper-output` from
`sonar-project.properties`.
- In `.github/workflows/build-steps.yml`, pass
`--define
sonar.cfamily.compile-commands="$BUILD_WRAPPER_OUT_DIR/compile_commands.json"`.
**Root cause**
The SonarCloud config still used the deprecated
`sonar.cfamily.build-wrapper-output`
property. The current CFamily analyzer expects a `compile_commands.json`
(produced by build-wrapper). Using the old property caused scanner
failures and
blocked analysis.
**Validation**
Regenerated `bw_output/compile_commands.json`, ran `sonar-scanner`, and
confirmed the scan completed and uploaded successfully.
**Tests**
CI-only change. Verified locally by running the scanner against a fresh
build-wrapper capture.
---------
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>
There are still a few historical authors who never signed on to the
relicensing of OpenImageIO from BSD-3-Clause to Apache-2.0 two years
ago, usually because we could find no current way to contact them.
The few (< 25) files that still had their extant code were therefore
marked as using both licenses. They they almost always are down to just
a few lines in the whole file that are still original to those authors
(and therefore BSD), so every once in a while, natural code churn and
rewrites of what they touched remove the very last of these lines from a
file, leaving that file entirely Apache-2.0. I noticed that this was the
case for these three files, and so removed the reference to BSD license,
which no longer applies.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Many codecs don't provide `bits_per_raw_sample`. The bit depth of the
luma channel is the closest equivalent to a single bit depth then, while
the chroma channels may be subsampled.
Tested on basically all the video files on my computer, they all give a
non-zero bit depth that looks correct now.
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
* The "oldest" didn't really test against the oldest we claim to
support.
* The "latest versions" didn't really test against the latest versions.
* The "bleeding edge" didn't really test against libpng master.
* The auto-build was unnecessarily a few releases behind.
* Documentation touch-ups
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Read and write 10 and 12 bit images
* Read and write CICP
This makes it possible to write HDR images that are much smaller than
PNGs, in a file format that is supported in all modern web browsers.
By setting the matrix coefficients, the library will perform RGB to YUV
conversion on write. The YUV to RBG conversion was already happening on
read automatically.
A future improvement would be a better default choice of matrix
coefficients for writing. When no CICP is specified or when CICP is read
from PNG, then no conversion to YUV will be performed and compression
will not work as well. This is the same behavior as before. When there
is support for conversion between CICP and display interop ID, a good
default choice for matrix coefficients could be made as part of that.
Tests for CICP and 10 bit were added.
---------
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
This patch implements the new scheme that we hope will preserve ABI
compatibility across minor (yearly) releases.
Brief summary:
* We will strive to have annual minor releases (e.g., 3.1.x -> 3.2.x)
be ABI/link backwards-compatible, as strongly as monthly patch
releases (3.1.5 -> 3.1.6) always have been. Every-several-years
major releases (3.x -> 4.x) are still a potential full compatibility
break.
* Consumers of OpenImageIO don't need to change their code, and can
just continue to refer to `OIIO::Foo` or `OIIO::Bar()` as always.
* Internally, all items in the public OIIO APIs will be in a versioned
inner namespace, and will continue to live in the versioned
namespace where they were first introduced, forever. (Well, until the
next first-digit-changing major release, at which point we will clean
up the old cruft and version everything up.) If anything needs to
change in an ABI-breaking way, it will be *duplicated* in the newer
namespaces, so the old version continues to be available in the old
namespace.
* The new header nsversions.h contains a long and detailed comment
explaining how all the new declarations work.
---
Detailed explanation:
"Major" or "first digit" releases, which only happen once every
several years for us, are full ABI+API breaks. Previously, "minor" or
"second digit" releases, which are annual, have fully incompatible
ABIs, with separate namespaces to enforce it. Here, we are striving to
allow our annual/minor releases to no longer break ABI, to make
upgrading
easier for downstream users.
The basics are that once a symbol is introduced, it will forever live
in the namespace of that release. Subsequent release years will have
their own namespaces, but will either alias or duplicate the original
symbols. This is enabled by our recent (in the lead-up to 3.1)
introduction of a split 2-part namespacing scheme. So 3.2 is the first
release that has the potential to perserve compatibility with 3.1 in
this way.
In this PR, I have done this for the whole codebase. It works! -- as
measured by being able to have our automated ABI checker show 100%
back compatibility with 3.1. So currently, even though this is the
nascent 3.2 tree, everything in the public APIs live in the 3.1 ABI
namespace. And so they will remain, except for changes that cannot be
done without breaking ABI back-compatibility, which will then end up
with multiple versions in different versioned namespaces.
Let's look this over and decide if we like it. It does involve our
developers to do some hoop jumping and take extra care as we develop,
to keep things associated with the right namespaces.
Users of OpenImageIO shouldn't need to change anything on their end,
nor to be aware of this at all.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Something must have changed with the containers, it was failing on the
build step, before ever getting to the actual sonar analysis.
Switch to the same container and OCIO version that we use for the
regular 2024 era container CI.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The actual CICP reading code is trivial.
Also recognize mkv (Matroska) extension for the test.
Add test using Matroska + VP9, to maximize the chance of an ffmpeg build
supporting them. They are royalty free, built into ffmpeg and relatively
old.
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
Enable ICCProfile attribute support for WEBP for both input and output.
Like other formats this support is passive; pixel data will not be
altered by the mere presence of the ICCProfile attribute (i.e. it
functions like other informational metadata only).
The WebP API is a bit odd when it comes to writing the ICCProfile. We
must use the "mux" API set; the "advanced" API is apparently not
advanced enough to add the metadata to the file. This wouldn't be too
bad except that the mux API is at odds with how we write out the data.
Before: When it comes time to do the file write, we would pass our own
file writer into webp. The writes would go straight to disk. Now: If ICC
profile data is necessary to write, we let webp use its own writer to
write to memory. We then assemble the in-memory file and the ICC profile
data together and pass that entire package to our writer as a final
step.
When no ICC profile data is used, the old write behavior is kept.
Added `oiiotool` test variations which uses ICC profiles with WEBP.
---------
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
This PR addresses the primary concern of #4678 -- implement support for
reading and writing the `cICP` chunk for PNGs.
CICP (Coding Independent Code Points) is a means for using a tuple of
integers to communicate characteristics of a variety of, traditionally,
video encodings. The tuple is a series of four values that map to
integer enumerations (codified in [ITU-T
H.273](https://www.itu.int/rec/T-REC-H.273-202407-I/en)) representing a
certain ColourPrimaries, TransferCharacteristics, and
MatrixCoefficients, as well as a VideoFullRangeFlag.
In particular, CICP is used for describing HDR encodings to browsers and
image viewers capable of displaying the image as intended. For example,
if one were to write P3-D65 PQ-encoded RGB values to a PNG, it will only
look "correct" if there's an appropriate `cICP` chunk describing the
primaries (14) and transfer function (16); without such metadata,
PQ-encoded images will appear significantly darker and lower in
contrast.
Internally, CICP metadata is stored in a `int[4]` type `CICP` ImageSpec
attribute.
This PR adds the following:
- An oiiotool `--cicp` flag for setting, modifying, or removing CICP
metadata for the top image
- Methods for reading and writing PNG `cICP` chunk metadata <-->
ImageSpec `CICP`
- Tests
I've included tests. I've also embedded CICP metadata in the existing
16-bit test png in the testsuite.
But to see what this is all about with your own eyes, you can quickly
convert a scene-linear AP0-encoded EXR to a PQ-encoded P3D65 HDR PNG
with the following command:
`$ oiiotool -i input_linap0.exr --ociodisplay:from=ACES2065-1
"ST2084-P3-D65 - Display" "ACES 1.1 - HDR Video (1000 nits & P3 lim)"
--cicp "12,16,0,1" -o output_p3d65pq.png`
If you compare in a capable image viewer on a capable display the image
produced with the above command compared to another one that lacks CICP
metadata, you should see a pretty significant difference:
`$ oiiotool -i output_p3d65pq.png --cicp "" -o output_no_cicp.png`
("Preview" on a ~5 year old Macbook should suffice).
---------
Signed-off-by: Zach Lewis <zachcanbereached@gmail.com>
Freetype 2.14 was recently released and just became the default for
homebrew.
It seems that no changes are needed on our end and no output references
even need to change!
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Original changes and goal:
* Bump our "latest releases" CI tests to OpenEXR 3.4.0
* cleanup: dead guard removal for openexr -- our minimum OpenEXR release
supports float vectors, so we don't need any `#if` guards for it
anymore.
* Fix warnings related to mixing Core and C++ library constants for
H2J2K compression enums within a switch statement (only noticed on ARM
Linux with Clang 18).
But it seems that OpenEXR 3.4's auto-building of OpenJPH (to support
exr's new h2j2k compression methods) revealed that our CI wasn't
building and testing against OpenJPG as well as we thought. So:
* Clean up some openjph API use and type mismatches
* Suppress warnings on clang and recent gcc versions had with the
OpenJPH headers and our use of them.
* Fix our test scripts and reference output for the htj2k tests; I think
maybe they weren't running at all before, and nad some incorrect file
paths.
Finally,
* When no OpenEXR is found, update the auto-build version to 3.3.5. I
wish I could go all the way to 3.4.0, but the OpenEXR 3.4.0 currently
auto-builds OpenJPH in a way that (I think incorrectly) installs the
OpenJPH components in ways that can overwrite or break existing
installations.
* Remove FindOpenJPH.cmake, the nomenclature clashes with the lowercase
"openjph" used by the new OpenJPH exported cmake configs.
* But those are only generated for OpenJPH >= 0.21.2, so make that our
new minimum. (This only affects OIIO 3.1 and higher, so it's safe to do
this.)
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Say you need to take a multi-part exr file and split it into separate
files for each part. (Note: what OpenEXR calls "parts", OIIO calls
"subimages.") You could do this
oiiotool multipart.exr --sisplit -o:all=1 "part.%04d.exr"
But this would name the parts part.0001.exr, part.0002.exr, etc. Maybe
you would prefer to split them into files that contain the subimage name
rather than just the inscrutible index number (which may also not be
consistent across files)? You could try this dance to loop over the
subimages and write each one with a name extracted from its metadata:
oiiotool multipart.exr --for i {TOP.SUBIMAGES} --dup --subimage {i} -o
"subimage_{TOP.'oiio:subimagename'}.exr
That works, but that's a little awkward and hard to remember off the top
of your head. Perhaps instead we would prefer a simpler version:
oiiotool multipart.exr --sisplit -o:all=1
"part.{TOP.'oiio:subimagename'}.exr"
This PR makes that last command work. It actually only required a small
change, which is that the function that handles the `-o` command needs
to defer the evaluation of the expression subsitution in the filename to
occur separately for each subimage being output, rather than just once
for the whole `-o:all=1` (which would have baked the FIRST subimage name
into all the output file names).
Docs are updated to give examples of how to do this.
Also fixed a silly typo in the test script.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Try a build against ffmpeg 8 -- seems to work out of the box, so
document that we support that version in INSTALL.md.
* Augment FindFFmpeg.cmake to correctly identify 7.1 and 8.0 by the very
non-intuitive versioning scheme of ffmpeg's headers.
* Add a FFMPEG_EXTRA_LIBRARIES variable to FindFFmpeg.cmake to allow us
to easily inject extra library dependencies (coincidentally, I need this
at work to force it to use static libraries).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Enforce gcc, clang, and apple clang minimum versions.
* Raise advertised minimum clang version to 10.0. Error for clang < 5,
which we know is the true minimum because earlier than that won't
support C++17 which we require. But in truth, we haven't tested
versions older than 10.0 for a long time, so we should not be
implying we know older versions will work or that we will support
them.
* Simplify cases no longer relevant for older gcc and clang.
* Remove unneeded gcc guard in simd.h. I think it was important only
for gcc 5! But it was also preventing the slightly faster construct
from activating for clang. Seems to pass all our CI jobs with this
simplified.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This doesn't really change any of the color management algorithms per
se, or change how we use OCIO. That will come later.
But this tries as best as we can to shift to CIF recommended interop
names for color spaces.
* "srgb_rec709_scene" as the canonical name for what we used to refer to
as "sRGB".
* "lin_rec709_scene" as the canonical name for what we used to refer to
as "lin_rec709" and several other names.
* "lin_ap1_scene" as the canonical name for ACEScg.
* Stop setting "oiio:Gamma" metadata (though we still accept it for back
compatibility), instead using space names like "g22_rec709_scene" and so
on. We never came across files with truly arbitrary gamma.
If I've done it right, the old names will continue to be accepted as
aliases.
I recognize that changing the output (when printing metadata) to the
canonical names might be an inconvenience at first, but in the long run,
I think it's better to comply with the conventions that will be used for
USD, MaterialX, and OpenEXR.
I'm fine with the new wordy and pedantic names as what is stored
underneath as the "oiio:ColorSpace" alias or stored in an OpenEXR file
in the colorInteropID metadata. But I gotta say, from a UI/UX
perspective, I do not feel good about replacing "srgb" and "acescg" as
things that are easy to type and are known/understood by the vast
majority of OIIO users with a change to these wonky, error-prone,
hard-to-type names. So I'm very uncertain about changing oiiotool
documentation, say, away from the short and widely understood aliases.
Seeking feedback on what to do here!
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Co-authored-by: Doug Walker <doug.walker@autodesk.com>
* Test against alpha of openexr 3.4.
* Document that we build against several things we know work but hadn't
updated INSTALL.md in a while.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
And a bunch of other minor updates:
* Change from 'testing' to regular containers
* Add forgotten 'export'
* Update default self-build pybind11 to 3.0.0
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* CI build as C++23 for bleeding edge test
* Fixes to disambiguate OIIO::print from std::print. Apparently, at
least for gcc14, std::print ends up in the global namespace and really
messes with things.
* Change to the library that contains the implementation of C++23
stacktrace in gcc14 vs 12/13.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Remove the CI jobs that tested VFXPlatform 2021 era, which we only were
still doing for the sake of python 3.7.
Also stop building Python wheels for any python versions older than 3.9
(we never built 3.7 wheels, but we did build 3.8).
This is just in main, a.k.a. OIIO 3.1, to be released in fall 2025 and
expected to be the main release series people are using in 2026.
VFX Platform has been 3.9+ since 2022, that's more than the "3 years
back" rule dictates. And in the real world, it hasn't been "current"
since 2020.
Just to be extra safe, I'm documenting 3.9 as the minimum and removing
all 3.7 tests, but I'm not having the build affirmatively reject 3.7 for
now, just in case somebody needs it. It should continue to work if
someone tries it, with no guarantees or support. But probably by the
time we do a 3.1 release in September, we will make it actually reject
3.7.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
ABI changed slightly after #4827. (Allowed in main but requires an
update of the reference.)
Also, bump to using the 2025 container for the ABI check.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
PRs #4840 and #4841 crossed in flight with their tests passing
individually, but they needed one small adjustment to reference output
in order to to account for their interaction in main.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* span-ify ParamValue (including ctr and init)
* add span versions of attribute() and getattribute() to the classes
that have them: ParamValueList, ImageSpec, ImageCache, TextureSystem.
Much like the previous span/image_span additions we've made elsewhere,
now each of these calls has 3 flavors:
1. The previously existing -- but acknowledged "unsafe" -- version that
takes a TypeDesc and a raw pointer (which had better match the TypeDesc
in type and size, but who knows?)
2. A version that takes a TypeDesc and a `span<std::byte>` delineating
the raw memory to use, that must match the size implied by the type, and
is implemented underneath (for now) by calling the pointer version after
checking that the size is adequate.
3. A template version that takes a TypeDesc and a `span<T>` that must
match both in total size and appropriate type and asserts in debug
builds if they don't. It's implemented by calling the raw byte version
after the type check.
The third variety is the preferred one for apps to call, whenever
possible, but the others are still exposed for special cases.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Allow CMAKE_MSVC_RUNTIME_LIBRARY to be initialized from an environment
variable of the same name. (CMake doesn't do that on its own.)
* If set, propagate CMAKE_MSVC_RUNTIME_LIBRARY to any packages we
auto-build within our build system.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
New logic for reading OpenEXR files:
1. Recognize "acesImageContainerFlag" as always being lin_ap0_scene.
2. If the new colorInteropID is present in the metadata, that's the single source of truth about color space.
3. Otherwise, leave it blank, don't make any assumptions about color space.
It was a mistake for OIIO 3.0 to assume that completely unlabeled openexr
files are lin_rec709.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Inside ImageInput::create(), there's a section where the created
ImageInput tries to do a quick call to its valid_file() method to do an
inexpensive check that the file appears to be the right file format
corresponding to the ImageInput subclass. However, it always called the
variety of valid_file() that takes a string filename, ignoring the fact
that create() itself may have been passed an IOProxy, which should
always take precedence over trying to open a real file.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
When passing numpy arrays for pixel blocks (say, to
ImageOutput.write_image), the sorting out of what the numpy arrays meant
had an error for the 2D case. A typo (or maybe copy pasta) got the
stride wrong.
This took a long time to be discovered because if there is more than one
channel, it gets passed as a 3D numpy array (height, width, chans). So
only for 1-channel images would it stumble across the bad code if it
were passed as a 2D (height, width) numpy array with no dimension for
channels.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Submitting this from my work as an intern on the RenderMan development
team.
- Bug fix with prman tile size for bumpslopes textures.
- `--slopefilter`: maketx has used sobel, txmake uses centraldiff
because sobel sometimes has ringing artifacts
- `--bumpinverts`: invert slopes in the s dir, txmake default behavior
has this on
- `--bumpinvertt`: invert slopes in the t dir, txmake default behavior
has this on
- `--bumpscale`: scale the strength of the resulting map
- `--bumprange`: support normal map values on either [-1,1] or [0,1]
- `--keepaspect`: sets PixelAspectRatio metadata to match txmake `up`
resize behavior.
`txmake -bumprough 1 0 0 0 0 0`: `maketx --bumpslopes --bumpinverts
--bumpinvertt --slopefilter centraldiff`
`txmake -bumprough 2.5 0 0 1 0 0`: `maketx --bumpslopes --bumpscale 2.5
--bumpinvertt --slopefilter centraldiff`
---------
Signed-off-by: Scott Milner <smilner@pixar.com>
Recent OpenEXR internals changes make header reads via the C++ library
dramatically slower if a Imf::IStream subclass does not implement the
size() and stateless-read method.
See https://github.com/AcademySoftwareFoundation/openexr/issues/1984
On my laptop, when forcing OIIO to avoid the "core" library and use the
C++ API to OpenEXR, this speeds up the reading of small exr files by
about 6x.
These changes were introduced with OpenEXR 3.3 and we never changed our
side.
(Also opportunistically added a couple DASSERT's to the existing
functions.)
Signed-off-by: Larry Gritz <lg@larrygritz.com>
avcodec_close (AVCodecContext *avctx) removing in ffmpeg 7.2 and was
deprecated for couple years
Global avcodec_close (AVCodecContext *avctx)
Do not use this function. Use avcodec_free_context() to destroy a codec
context (either open or closed). Opening and closing a codec context
multiple times is not supported anymore – use multiple codec contexts
instead.
In ffmpeg master, it has already been removed. But still work in v7.1
We checked the ffmpeg changelog, and avcodec_free_context was added 7 years ago and should be present in ffmpeg 4.0, the oldest version we support.
Signed-off-by: Vlad (Kuzmin) Erium <libalias@gmail.com>
The main thing is no longer using a custom namespace prefix for
auto-built libpng. We're already building it statically by default and
hiding the symbols, so the custom prefix only serves to make occasional
build problems by having different symbols than typically found in the
library. We still keep it as an option that can be enabled.
Also, be a little more careful in clearing unneeded variable that may
have been set by an earier attempt to find the package but that failed
to turn up a version we were happy with.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Add missing C++ header file
On the F41 and F42 compilation using LLVM based Clang++ failed.
```
[ 23%] Building CXX object
src/libOpenImageIO/CMakeFiles/OpenImageIO.dir/__/jpeg2000.imageio/jpeg2000input.cpp.o
/usr/src/github.com/1div0/OpenImageIO/src/jpeg2000.imageio/jpeg2000input.cpp:462:9:
error: use of undeclared identifier 'va_start'
462 | va_start(args, fmt);
| ^
/usr/src/github.com/1div0/OpenImageIO/src/jpeg2000.imageio/jpeg2000input.cpp:464:9:
error: use of undeclared identifier 'va_end'
464 | va_end(args);
| ^
2 errors generated.
```
Signed-off-by: Peter Kovář <peter.kovar@reflexion.tv>
This changeset has bunch of improvements for pixel closeup window
feature:
- Min/max/avg values
- RGB channels are colored with respective color
- Pixel colors in preview window are showing actual colors (previously
there was semi-dimming overlay on top of them)
- Center pixel is now marked including corner cases when close to the
edge
- During follow cursor mode window switches the the left from the cursor
when too close to right edge, and at the top when too close to bottom
edge
- Number of previeable pixels is configurable
- Number of pixels in a preview used for averaging is configurable
-- When avg subset is smaller then main closeup window there is an
indicator showing are that being averaged
---------
Signed-off-by: Aleksandr Motsjonov <soswow@gmail.com>
Also found one stray test that was using a pybind11 version older than
we claim to use, so bump that up to the advertised minimum.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Switch to posix_spawnp for macOS background launch to enable PATH
lookup, this is a fix for:
https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/4640
Description:
Previously, the code used posix_spawn with argv[0] as the executable
name. This failed when argv[0] was not an absolute or relative path
(e.g., just "iv"), because posix_spawn does not perform PATH lookups.
Switched to posix_spawnp, which behaves like execvp and searches PATH
when argv[0] does not contain a '/' character. This allows the program
to re-spawn in the background using the original command name, without
requiring _NSGetExecutablePath() to resolve the full path.
Signed-off-by: Mikael Sundell <mikael.sundell@gmail.com>
Wow, my fingers are so used to typing "exr", and I don't use the OS that
would make me ever type "exe", so...
Signed-off-by: Larry Gritz <lg@larrygritz.com>
pybind11 3.0 changed the generated stubs, causing CI to fail, so lock
down to no later than the last version before it broke.
We think the pybind11 3.0 generated stubs are mostly better, so at some
point soon, when we have more time to look at it carefully, we will come back
to this topic and bump our wheel workflow to use pybind11 3.0 and update
the stubs accordintly.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
[As reported on Slack](https://academysoftwarefdn.slack.com/archives/C05782U3806/p1751642730171029),
between OIIO 2.5 and 3.0, we had a big slowdown when doing a
multithreaded `maketx --envlatl`, which was traced almost entirely to
the new mutex lock in ImageBuf::IteratorBase::init_ib(), which were all
happening inside the call stack of `resize_block_<float>()`.
The reason was that it was calling ImageBuf::interppixel_NDC for every
pixel, which in turn works by creating (internally) an
ImageBuf::ConstIterator to sample the 4 nearby pixels. But every one of
those ConstIterator constructors called init_ib, which briefly grabbed
the mutex for the IB, which in addition to being wasteful on its own,
caused the threads to block on each other.
The solution is straightforward: there is no need to construct a new
Iterator for every pixel. We can create the iterator once (a single call
to the init_ib for each thread region of the image) and then for each
pixel, just call its `rerange()` method to reset the set of pixels to
loop over for the samples needed for that pixel.
I also opportunistically eliminated a few redundant spec() calls in
various routines in imagebuf.cpp.
On my Mac laptop, when doing a maketx --latlong on a 16k image with 16
threads, it was previously taking 170.5s (including 117.8s for the
initial resize and 42.8s for the MIP computations). With this change,
the same operation takes 12.4s (including 3.7s for the initial resize
and 1.6s for he MIP computations). That's almost a 14x speedup. YMMV,
depending on platform, compiler, image size, and number of threads.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
It was implemented as int, documented as float. Change the docs to match
what was always indended (int), and also switch the implementation so
that either int or float will be transparently accepted.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Just my periodic updating of the CHANGES doc in main.
Will merge immediately, as it's administrative docs.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#4811
This patch makes oiiotool's initial cursory scan of the command line
notice a `--threads` and carfully honor it when parallelizing the frame
loop.
Explanatory background info:
The cited issue shows something subtle happening with oiiotool's
`--parallel-frames` argument. The symptom was that it was using too many
threads and running out of memory, and seemed impervious to the use of
`--threads` to properly control it.
The reason is that oiiotool first scans the command line args in a
cursory way looking for signs of a "sequence", like a `--frames` or any
filename containing frame wildcards (like `linear.%04d.exr`) and also
noting if there is a `--parallel-frames`, but not actually taking any
actions for any of the commands yet.
Having seen that there are frame wildcards, what it then does is execute
the whole command line, once for each frame number that matched the
pattern. And since it noted that a `--parallel-frames` argument was
present somewhere, it does it with a parallel for loop. But how many
threads does it use? Well, enough for all the cores, since in some sense
it has not yet noticed the `--threads` argument. Moving the `--threads`
to be placed prior to the `--parallel-frames` doesn't help, either.
The suggested workaround was to set the default number of threads with
the environment variable `OPENIMAGEIO_THREADS` prior to launching
oiiotool. But this patch is a more robust solution.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Seeing build failures when building against libheif 1.20:
```
/private/tmp/openimageio-20250701-18561-96sl0/OpenImageIO-3.0.8.0/src/heif.imageio/heifinput.cpp:390:37: error: no matching member function for call to 'get_plane'
390 | const uint8_t* hdata = m_himage.get_plane(heif_channel_interleaved,
| ~~~~~~~~~^~~~~~~~~
/opt/homebrew/include/libheif/heif_cxx.h:909:32: note: candidate function not viable: no known conversion from 'int *' to 'size_t *' (aka 'unsigned long *') for 2nd argument
909 | inline const uint8_t* Image::get_plane(enum heif_channel channel, size_t* out_stride) const noexcept
| ^ ~~~~~~~~~~~~~~~~~~
/opt/homebrew/include/libheif/heif_cxx.h:914:26: note: candidate function not viable: no known conversion from 'int *' to 'size_t *' (aka 'unsigned long *') for 2nd argument
914 | inline uint8_t* Image::get_plane(enum heif_channel channel, size_t* out_stride) noexcept
| ^ ~~~~~~~~~~~~~~~~~~
1 error generated.
```
relates to https://github.com/Homebrew/homebrew-core/pull/228740
Starting with 1.20, libheif changed the type of a stride parameter.
---------
Signed-off-by: Rui Chen <rui@chenrui.dev>
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Co-authored-by: Larry Gritz <lg@larrygritz.com>
We've already done ImageInput, ImageOutput, ImageCache, TextureSystem.
This applies the analogous set of changes to the public APIs of ImageBuf
-- adding span and image_span versions of all the API calls that
previously required raw pointers and strides. Many of the internals are
still using the old versions, but at least this takes care of the
interface.
Specifically, this does constructors, reset(), and get_pixels().
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#4584
A minor edit with large consequences.
Previously the setting would encode uncompressed files due to a bug,
causing the much faster encoding as stated in the old description.
In the upcoming v0.12 libjxl release, [my
PR](https://github.com/libjxl/libjxl/pull/4201) overhauls the setting,
to correctly scale decode speed at the cost of density.
---------
Signed-off-by: Jonathan Brown <jonathanbr30@gmail.com>
Some actual bumps in CI, others just bringing the INSTALL documentation
up to date with what we have been testing and know works.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The upload_pypi step only happened for tags, not for other events like
PRs. But it would TRY (and fail) if you pushed a v3.0.* tag to your own
fork. This patch makes it not even attempt it, except when the tag is
pushed to the official repo.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We have been having intermittent CI failures when the auto-build of
OpenColorIO triggers the auto-build of Pystring, and then sometimes
immediately seems to think the version it installed is not correct.
I'm jiggling some things here to see if I can get it to stop
occasionally failing. I've run the testsuite on this patch several times
without seeing any failures that in recent days had been happening
typically in 1-2 jobs per CI run. So maybe this fixes it? I'd be lying
if I claimed to have a coherent and full explanation for exactly why
these changes help.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Delete some reference test output for older toolchains and dependency
versions that seem no longer needed.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We had a couple documents that were temporary checklists and status
reports from back when the project was first coming into the ASWF,
reminding us of tasks that needed to be performed.
That was years ago! All done, and OIIO is even a fully
"adopted/graduated" project. Remove the files, which are misleadingly
out of date and no longer needed.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Among other simplifications, remove all vcpkg builds from windows CI,
don't need them. Just rely on our auto-build. Shortens the build time on
Windows!
Signed-off-by: Larry Gritz <lg@larrygritz.com>
An important clarification: ImageInput::seek_subimage() returns false
but does not set an error message if the only thing wrong is
requesting a non-existent subimage or miplevel, and similarly,
ImageInput::spec() and spec_dimensions() set an empty spec (easy to
tell by spec.format == TypeUnknown) in that case. All three of these
only set an error message if the reason it couldn't perform the task
because there was actually something wrong with the file or the
I/O. (Put another way, it's common to use these to simply probe which
subimages or mip levels exist, and returning false for "no it doesn't"
isn't really an error, it's expected behavior.)
OK, then.
We weren't always following these rules, sometimes setting
error messages when we shouldn't.
Other times (in different routines) when we didn't set error messages
but should have. And also, there are some downstream places such as
ImageBuf::init_spec() that assumed that seek_subimage / spec /
spec_dimensions set an error message (but they didn't, by design) and
should have raised their own errors when discovering that a requested
subimage or miplevel didn't exist.
Also, there were a couple spots in ImageBuf internals where we tried
to do the right thing, but instead of calling ImageBufImpl::error(),
we called errorfmt(), which didn't exist in ImageBufImpl, and so was
calling the global OIIO::errorfmt(), which therefore was setting the
error on the wrong object entirely. (Fix that by adding an errorfmt
method in ImageBufImpl so that mistake is not possible. Should have
been like that all along.)
All this leads to more consistent error reporting (and not reporting,
in the cases we weren't supposed to), especially for people calling
ImageBuf::read() and ImageBuf::init_spec().
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I'm unsure how this could have worked correctly. There were three
different variables used before, now there is only one.
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
- The parameter 'pattern' now supports a special value 'auto', which is
also default.
- The parameter 'layout' now supports a special value 'auto', which is
also default.
- The parameter 'algorithm' now supports a special value 'auto', which
is also default.
- New parameter 'white_balance_mode.
- The raw input plugin now adds an attribute 'raw:WhiteBalance' in
addition to the libraw-specific 'raw:cam_mul', carrying the same values.
This changes the current behaviour, please read below.
Previously, the parameter 'pattern' defaulted to 'bayer'. The new default
value of 'auto' makes OIIO look for the input ImageBuf's attribute
'raw:FilterPattern', and deduct the pattern from its value. If the
attribute is missing, 'bayer' is assumed.
Previously, the parameter 'layout' defaulted to 'RGGB' for Bayer pattern,
or 'GRBGBR BGGRGG RGGBGG GBRGRB RGGBGG BGGRGG' for X-Trans. The new
default value of 'auto' makes OIIO look for the input ImageBuf's
attribute 'raw:FilterPattern', and deduct the layout from its value. If
the attribute is missing, the old default values are assumed.
Technically the 2 changes above the current behaviour may change, but
only in a very rare use case: when the user relies on the old default
values to decode the image, and there is metadata present in the
ImageBuf saying that the pattern and/or layout is different from the
default.
Previously, the parameter 'algorithm' defaulted to 'linear'. The new
default value 'auto' maps to 'MHC' for the Bayer pattern, and 'linear'
for the X-Trans.
The newly added parameter 'white_balance_mode' can have one of the
following three values:
- 'auto' (default) - the white-balancing weights are read from the input
ImageBuf's attribute 'raw:WhiteBalance', defaulting to (1, 1, 1, 1).
- 'manual' - the white-balancing weights are read from the parameter
'white_balance'.
- 'none' - same as 'manual' with the weights of (1, 1, 1, 1).
This changes the current behaviour where the white-balancing weights
from the parameter 'white_balance' get applied if present. Now, in
addition to providing them, you need to explicitly set
'white_balance_mode' to 'manual'.
I've updated the unit tests to cover the new 'auto' mode.
---------
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
ImageOutput::supports() recognized queries for "multiimage" and
"mipmap". But ImageInput::supports() only recognized "multiimage". Add
"mipmap" to make it easier for client software to know if the image file
format they're reading from allows the possibility of MIP-mapping.
As far as I can tell, only OpenEXR, TIFF, Ptex, and DDS do, among the
file types we currently support.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
On Windows ARM64, when building with clang-cl, LLVM can sometimes use it's own built-in intrin.h file, located in `<LLVM INSTALL PATH>\lib\clang\20\include`.
This file, as opposed to the built-in one that ships with the Windows SDK that MSVC uses, doesn't have an inclusion of arm_neon.h. This PR adds a small check to the simd.h file to include this header.
I didn't add an extra check to specifically only do this for clang, as it seemed a little messy - I can do this, but it's relatively harmless for MSVC given intrin.h includes it anyway for that compiler.
This change has already been in place for some time in Blender, this is just an upstreaming PR for that change: https://projects.blender.org/blender/blender/src/commit/ea604b6b80e49adb51d12df4e4edd5af1c51f446/build_files/build_environment/patches/oiio_windows_arm64.diff
Fix some `oiiotool --missingfile` behavioral corner cases:
When substituting, warn rather than making a true error. This didn't
seem like a big deal for a single command line, but when doing a loop
over frames, it would prevent correct behavior from pressing on to the
next iteration.
How big is the substitute image? It uses the rez and channels of the
first image successfully read (if available). But it when in a frame
loop, it resets this on every iteration, so any missing frame looks like
it doesn't have a good size guess -- but it does, as long as it's not
the first iteration of the loop. This involves safely gettingthe first
image dimensions from the *parent* (each iteration is a child Oiiotool
object).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
When doing oiiotool command line expression substitution, asking for
this metadata evaluates to the number of subimages in the referenced
image.
The purpose is that asking to retrieve, say, `{TOP.'oiio:subimages'}`
might look right, but generally it's only added to images (or format
readers) that are potentially multi-image. Whereas SUBIMAGES is not
actual image metadata, it's just an evaluation token that will always be
the number of subimages.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Improvements to the way our CI utilizes ccache to speed up build. Ported
from OSL:
- Split caching into separate restore/save, and rearrange the step order
so we save cache before tests run. This allows us to save the ccache
even for jobs with failed tests (where the all-in-one unsplit approach
would skip the cache save step for a failed job), or for jobs that we
purposely end early (maybe because we see failing tests), not only when
all tests finish and succeed. This makes subsequent builds, that we do
on the same branch while attempting to fix failures, go faster by using
the cache.
- Make subsequent pushes to the same PR or REF to cancel any previous
jobs, so if we push updates before the last CI run of the same branch
completes, it won't keep those going as useless zombie jobs.
- Change the cache key names we use to remove the github ref to
encourage cache reuse across different commits or branches (I think),
and to fix a couple errors where we had mis-named things and were
sharing caches between jobs when we shouldn't or vice versa.
- Be more careful about the location of the cache, making sure we were
for sure telling the GHA cache action to be saving/restoring the same
directory location that we told ccache to use as its cache location.
- Use CCACHE_COMPRESSION env var to be extra sure we're using compressed
caches to be sure we get the best bang for buck on the limited cache
storage we're allowed.
- Set CMAKE_CXX_COMPILER_LAUNCHER and CMAKE_C_COMPILER_LAUNCHER in the
shell for CI in the hopes of making more dependencies we build also use
ccache.
- Alternate launcher structure -- use LANG_COMPILER_LAUNCHER, not
RULE_LAUNCH.
A sampling of typical results:
test build deps build oiio
----------------------- ---------- ----------
VP 2024 (cold) N/A 6:22
VP 2024 (cached) N/A 0:55 (7x)
latest rels (cold) 8:29 5:30
latest rels (cached) 2:24 (3.5x) 0:45 (7.3x)
(The N/A is because that run gets all its dependencies from the
container and doesn't have to build any from scratch.)
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Just on the "bleeding edge" CI job, we were refraining from doing the
apt-get update. It was breaking something last year. Now NOT doing it is
breaking something in this CI test, just since this week (presumably
some default package on the GH runners changed, I haven't been able to
isolate any other reason). But bringing all the apt packages up to date
makes it go away.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Sorry for not doing it in a first PR. I thought to see if I can make
VSCode / Cursor to actually syntax highlight that raw string literal
content. And I found a way. Seems like if you change the delimiter using
language of content syntax name you make VSCode to syntax highlight it:
I don't think those clang exceptions are needed now.
Signed-off-by: Aleksandr Motsjonov <soswow@gmail.com>
This PR fixes and improves a tiny constellation of OCIO-related bits,
directly or indirectly pertaining to OCIO Viewing Rules and File Rules,
and the propagation of `oiio:ColorSpace`.
## Description
The focus of this PR:
- feat: Permit the `ociodisplay` ImageBufAlgo to dynamically select a
default View for the `fromspace` (or current `oiio:ColorSpace` value),
as directed by the OCIO config's Viewing Rules (if present)
- feat: Have the `ociodisplay` IBA appropriately set the destination
ImageSpec's color space.
As part of this effort, a handful of additional changes and improvements
are addressed:
- fix: Re-enable the "looks" override functionality, matching the
previous OCIO-1-style behavior.
- fix: Improve `ColorConfig::getDisplayViewColorSpaceName(...)`'s
handling of Shared Views
- feat: Overload `ColorConfig::getDefaultViewName(...)` with function
that takes an additional `inputColorSpace` argument
- change: Alter the `ociofiletransform` IBA's behavior pertaining to
setting the destination spec's color space: instead of storing the
filepath as the `oiio:ColorSpace` value, attempt to get the color space
from the filepath; else, leave the `oiio:ColorSpace` attribute alone.
This isn't 100% foolproof, obviously, but I'd prefer to reserve the
`oiio:ColorSpace` attribute for color space names, unless there's a
reason not to. In a future PR, we can additionally attempt to divine a
valid color space string from specific CLF and CTF metadata, if present.
- feat: Implement a `ColorConfig::filepathOnlyMatchesDefaultRule(...)`
function, which wraps the OCIO::Config function of the same name.
- change: Alter `ColorConfig::getColorSpaceFromFilepath(...)` to behave
more like the OCIO::Config function of the same name. In particular,
return the Config's default color space name (instead of failing over to
OCIO-1 string-parsing heuristics, which, in turn, fail over to an empty
string); and provide an overloaded variant for implementing the legacy
(i.e., current) behavior. Update all usages in the codebase (i.e., in
oiiotool, w.r.t. autocc) to use the overloaded variant, so there's no
apparent change in behavior to how autocc functions.
## Background
This PR was inspired by #4753, which adds to the Python ImageBuf
bindings a `_repr_png_` magic method which returns a PNG bytestring
(which Jupyter notebooks can display inline), similar in behavior to
other libraries, like Pillow. This is great for previewing gamma-encoded
output-referred buffer data, but less great for previewing "linear"
encodings. Ideally, the `_repr_png_` method would apply an `ociodisplay`
ImageBufAlgo just before writing the PNG; but there would need to be
some means of controlling for the buffer "image state" when determining
how / whether to apply an `ociodisplay` IBA. For example, it would be
appropriate to apply the default ColorConfig's "ACES 1.0 - SDR Video"
View to an image buffer that contains scene-referred data; but very
evidently inappropriate to apply that same View to an image buffer that
holds display-referred data, yielding a "double-LUTted" appearance.
Fortunately, OCIO-2 `Viewing Rules` provides a solution. Viewing Rules
allow OCIO Config authors to selectively filter the set of Views
available for a given Display, on a per-input-colorspace basis.
Ostensibly, applications use the OCIO API's DisplayViewHelper class for
building View menu UX widgets that dynamically update as a function of
both the selected Display and Input Color Space; but there are
implications for headless applications, as well -- crucially, the Config
author may order Active Views such that different Input Color Spaces
elicit different default Views.
## Tests
I've included a test OCIO config with some Viewing Rules and File Rules
in testsuite/python-colorconfig; I'm testing that the new and updated
python bindings behave as expected; and I'm testing that the
`ociodisplay` IBA is OCIO-Viewing-Rules-aware and is properly
propagating `oiio:ColorSpace` to the dst buffer's spec.
Notably:
```python
import os
import OpenImageIO as oiio
import numpy as np
os.environ["OCIO"] = "/path/to/testsuite/python-colorconfig/src/oiio_test_v0.9.2.ocio"
a = oiio.ImageBuf(np.array([[[0.1, 0.2, 0.3]]]))
# The first time the ociodisplay IBA is applied, the lack of an `oiio:ColorSpace`
# attribute will be interpreted as the OCIO config's default color space -- in this case,
# "lin_rec709", which corresponds to a scene-referred default view (ACES 1.0 - SDR Video).
b = oiio.ImageBufAlgo.ociodisplay(a, "", "")
# The second time around, the `oiio:ColorSpace` attribute is now "sRGB (~2.22) - Display",
# a display-referred color space, thanks to the previous operation; as such, the IBA uses
# a different colorimetric "pass-thru" View as its default ("Colorimetry"), instead of the
# ACES View. Upshot is, values remain unchanged this time.
assert b.spec()['oiio:ColorSpace'], 'sRGB (~2.22) - Display'
c = oiio.ImageBufAlgo.ociodisplay(b, "", "")
assert c.spec()['oiio:ColorSpace'], 'sRGB (~2.22) - Display'
# Subsequent applications will be noops as well.
d = oiio.ImageBufAlgo.ociodisplay(c, "", "")
assert np.allclose(b.get_pixels(), c.get_pixels())
assert np.allclose(b.get_pixels(), d.get_pixels())
assert not np.allclose(b.get_pixels(), a.get_pixels())
```
## TL;DR
This PR enables OCIO config authors to dynamically control `ociodisplay`
IBA behavior w.r.t. the `oiio:ColorSpace` attribute, such that
`ImageBufAlgo.ociodisplay(buf, "", "")` always "does the right thing",
regardless of whether `buf` holds scene-referred or display-referred
data.
---------
Signed-off-by: Zach Lewis <zachcanbereached@gmail.com>
This flavor of runner will be retired at the end of the month and is
already experiencing intentional brown-outs to warn us.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Oops, my bad. Looks like #4664 (merged about a week ago) and #4783
(merged today, but authored before #4664 was merged) interacted in a way
that produced a build break. This tiny fix is essential to unbreak it.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I think this is all the span-izing that the public interface of
ImageCache and TextureSystem needs. And it doesn't break ABI
compatibility (unlike the ImageInput and ImageOutput changes recently)
because these aren't virtual methods.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#4714
This PR adds support for selecting a rectangular region within an image
to compute and display per-channel minimum, maximum, and average values.
The feature can be activated through the Tools menu or by using a
keyboard shortcut (Ctrl-A / Cmd-A) . Once active, the user can click and
drag with the left mouse button to define a rectangular area, which is
visually indicated by a blue outline. Upon releasing the mouse button,
the tool calculates statistics for all pixels within the selected area
and displays the results in the status bar.
---------
Signed-off-by: Danielle Imogu <dannieim689@gmail.com>
OIIO did already have a decompressor for HTJ2K via the OpenJpeg library,
but nothing for encoding. This adds the encoder using the OpenJPH
library, which will be needed for the HTJ2K encoding for OpenEXR too.
This is a merge with the OpenJpeg plugin, the reader will attempt to
use the OpenJPH library (if found) to read the input file, if it cannot
read it, it will fall back on OpenJpeg to read it. For writing with the
OpenJpeg library, if the compression flag is set to htj2k or the file
extension is j2c the OpenJPH library will be called to write out the
file.
If you want to do a lossy encode you can do it with:
```
oiiotool -i INPUTFILE --attrib jph:qstep 0.03 -o OUTPUTFILE.j2c
oiiotool -i INPUTFILE --compression htj2k --attrib jph:qstep 0.03 -o
OUTPUTFILE.j2k
```
Other options include:
```oiiotool -i INPUTFILE --compression htj2k --attrib jph:qstep 0.03
--attrib jph:num_decomps 3 --attrib jph:block_size 32,32 --attrib
jph:prog_order CPRL --attrib jph:precincts 128,128,256,256 -o
OUTPUTFILE.j2c
```
Added a htj2k test, it tests encoding a dox and a jpeg file, both
lossless and lossy.
---------
Signed-off-by: Sam.Richards@taurich.org <Sam.Richards@taurich.org>
As a reminder, the global OIIO attribute "missingcolor", or the
file-specific ImageInput open hint "oiio:missingcolor" is designed to
treat missing scanlines or tiles as not a true error, but instead just
pass over them and substitute the missingcolor value for those missing
areas on the file.
The nature of this PR is that while we had implemented this behavior
fully for tile-based exr files, we never did so for scanline-based
files. So here we do.
Now then, unfortunately, it does not work. I think it's a bug in the
OpenEXR library itself, but we're still trying to figure that out. But
nonetheless, I believe this change is correct.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We documented this function as checking the valid range of pixel
coordinates, but we CALLED it everywhere as if we were only checking the
resolution.
But we IMPLEMENTED as if we were checking the range, generating errors
for outputting exr files with negative pixel window origins. But why was
nobody seeing those errors when writing exr files with negative origins?
Because the openexr writer BOTCHED it by not noticing the return value
and considering it a failure.
Oh boy. All my fault.
So the fix:
* Change the comment documenting check_open() to match the way we had
been using it at the call sites.
* Change the implementation of check_open() to match this by no longer
checking whether all four corners were in the range -- only check if
each dimension's size is within the limit for that dimension.
* Fix the OpenEXR writer's failure to notice the check_open failure.
* Does this eliminate necessary checks? No! We already separately
checked that nonzero origins were only allowed for formats that support
it, and that negative origins were only allowed for formats that support
that. So the flawed check was redundant anyway. Ha!
* Add a test verifying that we can write exr files with negative
origins. (Technically, that would have passed before, but only because
we were ignoring the incorrect error.)
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
A utility to confirm that an image_span (possibly with non-contiguous
strides) entirely fits within another contiguous span.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
After adding the new `htj2k` compression codec to OpenEXR, here we
enable it in the OIIO stack so it will be recognized by the library and
binary tools, such as using `oiiotool` to directly change compression
type into htj2k.
---------
Signed-off-by: lji <lji@ilm.com>
Backend changes for issue
https://github.com/AcademySoftwareFoundation/OpenImageIO/issues/4436.
## Description
- add `ImageDims` struct to store only a subset of `ImageSpec` or use it
as a view on an existing ImageSpec.
- move `ImageDims` and `ImageSpec` ownership to `ImageCacheFile` and use
them as a pool
- add `ImageSpec* m_spec` in `SubimageInfo` to access the descriptors
that are common across all mip level
- add `ImageDims* m_dims` in `LevelInfo` to access the despcriptors that
differ per mip level
- add functions to access to mip level dimensions:
```c++
const Dimensions& SubimageInfo::leveldims(int miplevel) const;
```
- do the plumbing across all `libtexture` to use the new interface
## Tests
I performed a batch of testing to compare the memory usage and
performance of different strategies using 17 `ptex` files from the PTex
library samples (some from the Moana Island dataset, so they are quite
heavy). The baseline numbers are approximates but gives a good estimate
of the memory usage.
Baseline | Total (MB) | Specs (MB) (%) | # specs | Subimages (MB) (%)
-- | -- | -- | -- | --
OIIO 3.0 main | 1500 | 1200 (80%) | 4936364 | 300 (20%)
Method | Total (MB) | Specs (MB) (%) | # specs | Subimages (MB) (%)
-- | -- | -- | -- | --
No reuse | 787,2 | 454,3 (58%) | 4936381 | 332,5 (42%)
Reuse dist=1 | 442,2 | 109,3 (25%) | 1193485 | 332,5 (75%)
Reuse dist=10 | 368,5 | 35,6 (10%) | 380506 | 332,5 (90%)
Reuse dist=inf | 333,2 | 0,292 (<1%) | 3020 | 332,5 (>99%)
These numbers shows that the memory is now bounded by the `SubimageInfo`
+ `LevelInfo` structure sizes, and not from the descriptors `ImageSpec`
+ `Dimensions` with the right amount of reuse across subimages and mip
levels.
---------
Signed-off-by: Basile Fraboni <basile.fraboni@gmail.com>
Two PRs crossed paths in flight last week -- one adding the ARM tests,
the other doing something that changed the reference output of this test
slightly. Both passed on their own, but the combo left one platform's
output needing the update.
Also, edit nicknames of two tests, they were not unique.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
GHA offers free runners for Linux on ARM based machines. Let's throw
them into the testing mix!
A very small number of tests needed additional reference outputs saved,
due to a few LSB type errors in the math.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The INSTALL.md Windows building instructions were out of date in two
important ways: (1) they neglected to mention OpenColorIO as a (newly)
required dependency nor give instructions for how to build it; (2) it
doesn't mention the new BUILD_MISSING_DEPS capabilities.
So basically, this PR modifies our Windows build guidance to set
OpenImageIO_BUILD_MISSING_DEPS=all and rely on that without needing to
document any additional steps for dependencies.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Extended eraseattrib to be able to read regex patterns from a file,
one per line, with `--eraseattrib:fromfile=1 filename`.
Erases all metadata attribute fields that match.
Added a new test to `oiiotool-attribs` which has been verified
painstakenly to be working!
Also tried running the command directly from dist/bin and verified with
`oiiotool --info -v`.
More detail on the motivation:
It's common in a studio to have tools insert all sorts of metadata into files as they're rendered, etc. But a lot of that are things that SHOULD NOT leave the facility -- like security issues from internal pathnames or network info, or secrets like plot details revealed by sequence or character names. So generally you want to scrub the metadata from the final images before they are shipped to the client or to marketing or whatever.
oiiotool's --eraseattrib is handy for this, but it takes a single regex. But we were thinking that:
- You may have several patterns to fully describe the metadata you want to be sure not to ship. This could be an unwieldy regex or a cluttered series of multiple --eraseattrib commands.
- That list of patterns may change over time, and it would be error prone to have to change it in every script that calls oiiotool.
- That list of patterns may need to be different per show.
We thought a solution to facilitate this might be a version of --eraseattrib that takes a filename that contains a list of regexes, one per line of the text file. So that master list of things to scrub could be maintained in a central place and not need to update every oiiotool-calling script in order to update it. Furthermore, you could have the command say something like
$ oiiotool ready_to_ship.exr --eraseattrib:fromfile=1 /blah/$SHOW/nometadata.txt -o clean.exr
to make a single command that allows per-show customization.
---------
Signed-off-by: Lydia Zheng <lydiamzheng@gmail.com>
Fixes#4598.
It was noted in the issue that running `oiiotool` with output filename
inside a folder that doesn't exist will cause OIIO to consume time and
resources, but fail eventually due to the folder not existing.
Take for example the command:
```
oiiotool --create-dir -i input.png -o ./output_folder/output.png -o ./output_folder_2/output.png
```
During the output action, it creates the directories if needed
(`output_folder` and `output_folder_2`), and outputs the images into the
folder.
Added unit tests. Tested it locally, and it creates both the above
folders and outputs the image.
---------
Signed-off-by: DharshanV <dharshanv1999@gmail.com>
WEBP uses the `WebPConfig::factor` setting to also control how hard it
tries to compress when lossless compression is used. It also uses
another `WebPConfig::method` setting to further tune both lossless and
lossy schemes. [1]
Expose both of these settings so client applications using OIIO can
adjust these values as they see fit.
This PR makes 4 changes:
- Adds support for using a `lossless:factor` value for the `compression`
attribute. The 'factor' value controls the effort that WEBP uses during
the compression process.
- Adds support for using a `webp:method` int attribute. The value of
this attribute controls the `WebPConfig::method` setting. Note: this is
different than the compression settings but will also affect speed and
quality in various ways.
- Sets `WebPPicture::use_argb` to `true` when using lossless compression
(per the WEBP documentation[2])
- Changes default lossless "quality" from 100 to 70. This results in
much faster image output at a very minor increase in file size (e.g. a
2048x2048 test image goes from 1300ms/364kb to 800ms/371kb after this
change)
Before - just 2 basic forms were possible:
```
Client uses "compression=lossless"
- OIIO uses lossless=true, method=6, quality=100, use_argb=false
Client uses "compression=webp:90"
- OIIO uses lossless=false, method=6, quality=90, use_argb=false
```
After - several additional variations are possible:
```
Client uses "compression=lossless"
- OIIO uses lossless=true, method=6, quality=70, use_argb=true
Client uses "compression=lossless:100" and "webp:method=4"
- OIIO uses lossless=true, method=4, quality=100, use_argb=true
Client uses "compression=webp:90"
- OIIO uses lossless=false, method=6, quality=90, use_argb=false
Client uses "compression=webp:90" and "webp:method=4"
- OIIO uses lossless=false, method=4, quality=90, use_argb=false
```
[1]
https://developers.google.com/speed/webp/docs/api#advanced_encoding_api
[2] See notes around `use_argb` from the above link and see also the
internal Encode helper function which keeps lossless and use_argb in
sync:
https://github.com/webmproject/libwebp/blob/main/src/enc/picture_enc.c#L241
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
Only symptomatic on debug builds, but I was seeing exceptions from
pybind11 about bad GIL release. I traced it to the recent
implementations of demosaic. Apparently it's not safe to release the GIL
until AFTER the call to py_to_stdvector. Which makes sense when you
think about it -- we need the lock whenever we are interacting directly
with Python calls or data structures (such as what py_to_stdvector is
doing), but we release it when we're done with the python and only
calling OIIO internals (such as IBA::demosaic() itself).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
ImageInput methods that read scanlines, tiles, and image are given new
flavors that take image_span, much like the changes of PR #4727 did for
ImageOutput.
Generally, for each, there are three versions: (1) a low-level virtual
method base case that takes an `image_span<byte>` that describes the
underlying memory, and an explicit TypeDesc that specifies the data type
conversion (including allowing TypeUnknown to indicate keeping channels
in their native per-channel format from the file); (2) a templated
version taking an `image_span<T>` that infers the data type conversion
from `T` (must be a single type for all channels); (3) a templated
version taking a `span<T>` that further implies a contiguous buffer in
all dimensions.
For now, the default implementations of these new ImageInput methods are
just wrappers that call the old pointer-based ones. One by one, over
time, we can swap them, changing the format implementations to have a
full implementation of the new bounded versions, and make their raw
pointer versions call the wrappers. The raw pointer ones will be
understood to be "unsafe", merely assuming that the pointers always
refer to appropriately-sized memory areas. Meanwhile, the ones using
spans and image_spans will, due to assertions in their implementations,
make it easier to verify (at least in debug mode), that we never touch
memory outside these bounds.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Make it so that CI test cases that set the GHA variable "benchmark" to 1
will add a benchmarking step to the workflow that runs a new
ci-benchmark.bash script.
The script runs selected unit tests containing benchmarks (currently,
only image_span_test, but we can amend later as needed). Those
designated tests are run, and their output both echoed to the log for
that step and also put in build/benchmarks/TESTNAME and saved as a build
artifact for optional download.
Most test cases will not turn benchmarking on -- it probably will end up
adding a few minutes so do it very selectively (once per major platform
or compiler version is plenty).
I would have previously guessed that any attempts at benchmarking on GHA
runners was doomed, but in practice, I'm surprised to find that there's
almost as much run-to-run consistency as I find doing casual benchmarks
on my own machine. As such, I think this can be a handy way to do some
rough benchmarking using CI, to compare platforms or compilers, or
verify that changes we want to make don't introduce performance
regressions.
Caveats to remember in the future:
* Take it all with a big grain of salt, and watch the benchmark numbers
for the trial-to-trial range of times -- wide variation means that the
numbers probably can't be trusted.
* The GH runners themselves may change without warning, so beware
benchmark stability over time, or if they ever have pools of
heterogeneous machine generations/configurations.
* While my results indicate a decent amount of timing reliability for
purely computational tests, I assume that there will be enormous
run-to-run variation in anything involving I/O or networking. So this is
unlikely to be a fruitful way of testing for performance regressions in
image format I/O speed (but probably is useful for a variety of
in-memory operations).
* As we add more unit tests to what we benchmark in the future, keep an
eye how much time we're spending running these benchmarks. A few minutes
on a small subset of the test jobs is probably fine, but I wouldn't want
it to make the overall wait for a full CI run to become substantially
longer because of it.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Master version of the iv today when opening on mac sometimes opens
without a focus, which causes top menu bar not to be populated (just
`iv` on the left). Workaround is to switch back and forth once to and
from another app for menu bar too get populated.
It is unclear to me how reliably replicate the bug so that I have a
solid prove, that this fix works. But since building with this line it
never happened to me yet.
Maybe more knowledgeable in Qt can verify if it makes sense.
Signed-off-by: Aleksandr Motsjonov <soswow@gmail.com>
This was causing the Rust bindings to fail because the pkgconfig was
pointing to a path that didn't exist (the libdir was the full root
appended to itself).
Signed-off-by: Scott Wilson <scott@propersquid.com>
Similar to other formats like PNG, make WEBP respect the
`oiio:UnassociatedAlpha` attribute value in case client applications are
already handling the alpha association themselves.
---------
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
II and IO were documented as initializing their internal "how many
threads to use when reading/writing this particular file" to the global
"threads" OIIO attribute. But they did not, they always initialized to 0
("all the threads") even if the global attribute had been set
differently prior to creation of the reader or writer.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Our fmath_test (a.k.a. unit_fmath) test was failing with an exception
thrown by the fmt library because we used a width specifier when
formatting a TypeDesc.
It was really only an issue when using old fmt < 10.0, so I'm just
solving it trivially by ignoring that specifier for old fmt. It doesn't
seem necessary to make it work correctly with width specifiers for old
versions, when we don't in any way rely on it.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#4680
Adds a `_repr_png_` implementation to the Python bindings of ImageBuf.
This allows ImageBuf objects to be displayed in Jupyter
notebooks, as requested in #4680
Uses the IOVecOutput IOProxy to write the ImageBuf as a png into memory.
Then, reads and returns that as a python bytes object.
---------
Signed-off-by: glowies <comuokta@gmail.com>
Fixes#4718
See the demo of problems that were fixed and of fixes in action over
here: https://www.loom.com/share/3bdee20b32434e0bbba47d393928d4f7
There are fixed for three problems:
1) Window is not resizing when new image is loaded in (it still does
when image being loaded is the only one)
2) When you using scroll mouse wheel it is taking into account where the
cursor is (there is some corner case when it doesn't do it perfectly,
that I can't catch when it doesn't follow it exactly, but a bit of
panning fixes it)
~~3) Switching between images recenters (if image is smaller) or making
some other panning nessery to have image in the viewport. I believe it
is doing same stuff what happens when window is resized.~~ This has been
reverted.
---------
Signed-off-by: Aleksandr Motsjonov <soswow@gmail.com>
Freetype master (presumably a change that will be in the next release)
broke our CI this week by changing how the accents on certain unicode
characters are rasterized. So we needed a new reference image on our
end.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Apparently, the runners used to check things out on drive D:, but today
they are doing so on C:, and that changes some filenames in the
reference
output.
Also, fix missing nametag needed for Windows artifacts.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I've found a couple issues with iv when launched with a directory name
as input, while iterating through the files in the directory:
1. Crash on .DS_Store file
2. The files with extension in capital letters are not recognised as
valid images, due to case-sensitive compare. Capitalised extensions are
common in raw camera images.
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
If the python stubs test fails (because the API changed in some way that needs
the stubs regenerated, but the submitter of the PR did not or could not,
the wheel CI will regenerate the stubs and include them in the failure artifact.
This provides a path for users who can't build the stubs locally.
Signed-off-by: Chad Dombrova <chadrik@gmail.com>
Add python stub files for code completion and static analysis.
Fixes
https://github.com/AcademySoftwareFoundation/OpenImageIO/issues/4682
No new test cases, since there is not any change to the behavior. I do
have an idea for a kind of test that might validate the stubs against
the python unit tests, but I don't know if it's worth it.
---------
Signed-off-by: Chad Dombrova <chadrik@gmail.com>
While I was investigating some SonarQube warnings about our simd
`shuffle<>` templates (which are a false positive and I have a separate
PR to simply silence it), I did get to thinking about the naming, and
want to make a couple changes:
* I decided that the 1-template-argument version of this function,
`shuffle<int i>(simd_type)` would actually be more clear and
self-documenting if renamed `broadcast_element` to emphasize that it is
taking just one simd lane/element and broadcasting it to all lanes. (The
multi-argument shuffle really is doing a true shuffle, giving an index
for each lane to make a permutation of swizzle, so I'm not renaming that
one.) To avoid breaking source compatibility, I am leaving the old name
as well as a synonym, but commenting it as deprecated and I will phase
out its use. It will disappear entirely from a future OIIO version
that's safe to break compatibility.
* For 16-wide simd, the 1-arg template we called shuffle was not doing
the same operation -- it was replicating one group of 4 elements instead
of a single element. We didn't use it anywhere, so I'm redefining it to
do the analogous thing as it does for 4-wide and 8-wide.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* ImageOutput methods that write scanlines, tiles, and images, which are
the main customization points for format output implementations, are
given additional methods that take `image_span` in place of raw pointers
and strides.
* Generally, for each, there is a templated version that takes
`image_span<T>` that communicates both the memory area and the data type
conversion requested, a "base case" that takes an explicit TypeDesc for
the conversion type and a `image_view<std::byte>` giving the raw memory
layout, and a `span<T>` convenience version for when there are
contiguous strides. Note that when reading mixed channel data types in
"native" mode (no type conversion, just leave the data in its original
types), you have to use the std::byte image_span version, since the idea
is not to do any format conversion, and there may not be a single type
involved.
* For now, the default implementations of these new ImageOutput methods
are just wrappers that call the old pointer-based ones. One by one, over
time, we can swap them, changing the format implementations to have a
full implementation of the new bounded versions, and make their raw
pointer versions call the wrappers. The raw pointer ones will be
understood to be "unsafe", merely assuming that the pointers always
refer to appropriately-sized memory areas. Meanwhile, the ones using
spans and image_spans will, due to assertions in their implementations,
make it easier to verify (at least in debug mode), that we never touch
memory outside these bounds.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Added support for reading and writing ZBUF chunks, also known as
Z-buffers or depth maps.
Removed noproxy versions, no longer needed.
---------
Signed-off-by: Mikael Sundell <mikael.sundell@gmail.com>
A constellation of closely related fixes and additions related to
oiiotool `-i` behavior:
* When using `-i:now=1` to force for immediate read rather than using an
ImageCache, it would "forget" any prior setting of `--native`. This is
fixed now to correctly combine desire for bypassing the cache with
hinting to maintain native data type.
* When restricting channel subset with `-i:ch=...`, use of the
`--native` flag was incorrect in some cases -- it would use the overall
best single data format of all channels, rather than restricting that to
the selected channel range. So, for example, if you said `-i:ch=R,G,B,A`
and those channels were all `half`, but channel 17 (not selected) was
`float`, you would get a conversion to float rather than keeping the
selected channels at their native half. The solution here is that the
type chosen for `--native` behavior should only consider the selected
channel subset, not all channels.
* Add a new `-i` modifier: `:native=1`, allowing you to select the
"native" behavior on a per-input-file basis (the existing `--native`
command sets the default).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
PR #4736 bumped the version of sphinx we use on readthedocs, and that
was incompatible with the old version of furo we specified.
I'm not sure how the problem was not caught by our "docs" workflow.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Certain version combinations of fmt and gcc were warning, and it all
boiled down to the knots we tied ourselves into to replace the function
fmt uses instead of throwing exceptions, when they are disabled. I can't
seem to find any harm in letting it do its default thing, so simplify
our side to silence the warnings.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I discovered, by testing against the latest libultrahdr, that only
starting in 1.3 does it support the "64 bit 'half' RGBA" variety of UHDR
that is the flavor used by the testsuite example. But our "auto-build"
is pegged to 1.2, which means that all along, we were not decoding that
test file properly as UHDR and our test reference output was therefore
wrong. We also didn't detect libuhdr's version at all, let alone exclude
versions that were too old.
Remediations:
* Amend Findlibuhdr.cmake to detect the version from symbols in its
header files.
* Have our search for libuhdr enforce a minimum version of 1.3. (I feel
bad about "changing minimum dependencies" in a release branch, which we
usually disallow, but I think this time is an exception since we know
versions < 1.3 are broken.)
* Change the auto-build selected version to 1.4, the latest.
* Mention libuhdr in the INSTALL description of dependencies and
acknowledgement in the docs, where it had previously been omitted in
those places.
* Update the testsuite/jpeg-ultrahdr test reference output. (You can
tell it was wrong before because the "max" channel value was 1.0, and
now it's 4.6.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Seems to shave about 5s off of each and every CI run variant. Every
little bit helps.
Also update the versions of some actions we're using to the latest.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I think what we were doing here was fine all along, but the idiom was
confusing to static analyzers who identified a danger that we were
memcpy'ing into a field that was potentially not big enough. A minor
restructuring of the code and a new assertion should verify that it's
safe and also make it clear to the static analyzer that we aren't
falling into the case it warned about.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Sonar was complaining that initialize_cuda() contained a loop that would
only iterate once. Upon looking, this was true, but mainly because we
just didn't adequately check errors. Fix up, which should both make the
code more robust and also silence the Sonar complaint.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This useless symbol was removed in PR #4724, but maybe downstream things
used it and will break simply because it was removed, even though it
never would have been nonzero. It should not have been completely
removed except at a compatibility-breaking version boundary.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* `image_span<T,Rank>` is a 2D-to-4D bounded span with byte-measured
strides (describing data layout with dimensions for channel, x, y, z)
for us to have future APIs deal with bounded+strided regions rather than
YOLOing raw pointers. Among other advantages, image_span will have
robust bounds checking in debug builds. It also simplifies interfaces to
pass a single image_span that encapsulates everything about an up-to-4D
data layout, rather than needing to pass a long list of individual
pointers, sizes, and strides.
* I've templated image_span on Rank, but default to Rank=4 and am using
that version only in practice. The templating by rank allows future
expansion to describe one scanline (rank 2) or 2D image plane (Rank 3)
if we want to do so for certain API calls, but I'm not sure yet whether
to bother with that, so for now, the ability to template by Rank is just
flexibility for the future.
* Add image_span based versions of copy_image(), contiguize(),
convert_pixel_values(), convert_image(), and parallel_convert_image()
utilities. I added tests and benchmarks to verify their correctness and
that they have similar performance to the pointer based versions.
* Deprecate image_view, which we did not ever use internally to OIIO or
in its APIs. But since we published the headers, there's a chance it's
used downstream and would be unwise to change its behavior or data
layout. The new image_span is what we will use moving forward so that we
are free to modify it and start with a fresh slate. Keeping the
definition/header for now, but marking it with a deprecation warning.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* We passed a bad flag to _mm512_cvtps_ph intrinsics because of what I
can only imagine was a bad cut-and-paste years ago. (Harmless because it
set bits never used, but a warning on Windows.)
* Remove clauses for AVX512-EL, I'm not sure why they were there,
because those instructions were only for "Knights Landing" which was
never made available to the general public and has been discontinued. It
must be that years ago, I got a little overzealous with the SIMD
intrinsics.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Somewhere along the road to 3.0, the build files changed in such a way
that OIIO_SITE was not correctly passed down to C++ symbols, and some
SPI-specific (sorry) color space handling was inadvertently disabled as
a result. This restores things to the way it was in 2.5, and should not
affect anybody else.
Also eliminate some old SPI-specific code that's no longer needed (and
clearly hasn't been needed for a long time, because we had been naming
the site "spi" but testing "SPI", and no problems were evident with
skipping that ancient code).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Use constexpr inside BaseTypeToC templates.
* Add BaseTypeToC_v and TypeDescToC_v convenience aliases, much like is
done in C++ standard type trait helpers.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Inspired by a link from EmilDohne in an unrelated PR, this introduces
`default_init_allocator`, an experimental custom allocator that default
initializes, rather than value-0-initializes, POD/non-class types.
This is useful for being able to make a std::vector with this allocator
(`default_init_vector`), thus having the property of not incurring a
large cost of needlessly initializing the elements when they are
allocated, as a default std::vector does.
I picked two spots sort of randomly, in which we previously avoided
using std::vector and instead did a unique_ptr / new[] dance to do
scoped heap allocation without paying the vector initialization tax. By
using default_init_vector, we avoid the extra cost while still using the
much more convenient vector type.
These two places are just to prove it works. There's no particular need
to proactively root out every use of the unique_ptr trick. But for
future development, we can use now use std::vector without the
construction cost when we allocate.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
INSTALL.md bolds *changed* minimums of dependencies since the last
release. Now that 3.0 is released and main is working toward 3.1, remove
those highlights from the last release so they won't get mixed up with
future changes for 3.1.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Raise auto-build version of Robinmap to 1.4.
* Add the magic CMAKE_POLICY_VERSION_MINIMUM to our auto-build of
Robinmap in case an older version is selected for auto-build. (Mimicking
the fix we did for other packages.)
* Add Robin-map to the INSTALL.md list of dependencies, somehow it had
been overlooked before.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The raw input plugin populates the attributes in the ImageSpec from the
libraw structures in the RawInput::open_raw() method. That is done
before the actual image data being decompressed on demand in
RawInput::read_native_scanline(). Libraw re-calculates some values after
the pixels get available, it is impossible to access those, as the
ImageBuf's spec does not get updated, and the updated ImageSpec is not
available from outside of RawInput when/after the pixels are being
processed.
An example: the initial values for the white balancing weights get
populated from the image file metadata, those get copied into the
ImageSpec. If the libraw was configured to white balance by averaging
the pixels of the whole image or a region, the white balancing weights
would get re-calculated by libraw after processing the pixels, but the
updated numbers are impossible to access from outside of the RawInput
plugin.
This change adds a hint "raw:ForceLoad", which, if set to 1, forces
libraw to decompress and process the image in RawInput::open_raw(), so
the image attributes returned from that method would contain the final
values.
I am not certain if this is the best way to achieve what I needed. I
have also looked into making the ImageBuf retain the ImageInput object
instead of spawning multiple transient objects, but that seemed too
complicated.
I have not added any unit tests, not sure how to approach that. I have
tested that the image does not get decompressed multiple times in a
bunch of scenarios.
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
This change embeds version information into our Windows binaries. This
is done by ways of a `version_win32.rc` resource file[1] populated at
CMake configure time with our major.minor.patch version information.
This not only allows at a glance to discover what version a particular
OIIO binary is (see screenshot), but it also impacts certain MSI
installer behaviors. If this is a MSI install which updates a prior
installation, and the shared libs/dlls don't have version information,
then the installer assumes the binaries are equal and just skips them.
This can be problematic for applications that package OIIO etc.
Old on left, New on right:
[1]
https://learn.microsoft.com/en-us/windows/win32/menurc/versioninfo-resource
---------
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
`--render_text:measure=1` causes the render_text command to set user
variables `TEXT_X`, `TEXT_Y`, `TEXT_WIDTH`, and `TEXT_HEIGHT` (as if set
by `--set`) to the origin offset and dimensions of the rendered text
size. These can then be accessed in subsequent expression evaluation.
`--render_text:render=0` causes the text to not actually be rendered
into the image. (The default is 1.) This is primarily of use in
conjunction with `measure=1` if you want to measure the size that text
would have taken, then use the variables for subsequent commands, but
not actually draw the text.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The SGI IRIS format is big-endian so incoming multi-byte fields need to
be swapped. This wasn't being done in the `valid_file` call so the magic
test would always fail on little-endian platforms.
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
From the PDF:
https://pub.smpte.org/doc/st268/20030505-pub/st0268-2003.pdf
3.31 X scanned size: Defines the horizontal size of the original scanned
optical image in millimeters. [42.1]
3.32 Y scanned size: Defines the vertical size of the original scanned
optical image in millimeters. [42.2]
----
This is a small PR that simply clarifies the units in the documentation.
Signed-off-by: Campbell Barton <ideasman42@gmail.com>
These values are read & written as integers
and don't work unless they are accessed using int accessors.
m_dib_header.hres = m_spec.get_int_attribute("XResolution");
Correct documentation.
BMP uses integer access:
- `./src/bmp.imageio/bmpinput.cpp:`, `m_spec.attribute("XResolution",
(int)m_dib_header.hres);`
- `./src/bmp.imageio/bmpoutput.cpp`, `m_dib_header.hres =
m_spec.get_int_attribute("XResolution");`
Signed-off-by: Campbell Barton <ideasman42@gmail.com>
Seems that for quite some time, there have been particular circumstances
in which the Python bindings have leaked ImageInput or ImageOutput
objects. That's bad enough on its own, but it also leaves file handles
open if code was depending on the fact that the ImageInput and
ImageOutput destructors close the files if they haven't already been
closed.
I think the offending part is that the `unique_ptr<ImageInput>` extracts
its raw pointer and gives it to Python, on the mistaken assumption that
it would be reference counted as a python object. But apparently, it's
not. Oops.
Leaving it as a unique_ptr turns out to be fine, as far as pybind11 is
concerned! And on top of that, if you give an uninitialized unique_ptr
to pybind11, it considers that a Python `None`, which means we can
further simplify some places where we explicitly had to return a
py::object and figure out if it should be None or not, but actually it's
all ok to just give the unique_ptr and pybind11 figures the whole thing
out for us.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We recently started getting some CI failures on the "bleeding edge" test
that uses the new gcc 14, seems that the newest cut of that compiler was
giving us new warnings where it suspected we were writing past the valid
bounds of some memory areas, but it was very hard to decipher the
circumstances and either find the bug or convince myself that is was a
false positive.
To help me debug it, I switched some of the internals of exif.cpp that
passed around raw pointers in very hard-to-validate ways to instead
use spans with explicit lengths, in the hopes that the debug-mode
bounds checking it provides would allow me to more easily track down the
problem.
Now it no longer gets compiler warnings.
Did I fix a very subtle bug in the process?
Does the reformulation with spans make it easier for the compiler to
verify that we aren't stomping on memory, and that suppresses the
warning?
Either way, it now passes all CI tests and I think the new code is safer
and aligned with our strategy of slowly changing raw unbounded pointers
to using spans in order to make these kinds of bugs harder to introduce.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The latest cmake in some package repos is 4.0. CMake 4.0 drops support
for compatibility with cmake < 3.5, and thus errors when building our
own dependencies that happen to have their own cmake scripts asking for
compatibility with something older than that.
Setting CMAKE_POLICY_VERSION_MINIMUM=3.5 environment and/or cmake
variable seems like a hack that could hide true incompatibilities, but
will get us past this for now.
Change our own minimum to the range of 3.18.2...4.0 to correctly reflect
the range of versions that we believe will work.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Add new ImageInput varieties of read_native_scanlines,
read_native_tiles, and a new read_native_voluemetric_tiles (all-channel
and channel-subset varieties of each) that take a `span<byte>`
describing the area to deposit the contiguous native-datatype values.
Moving forward, these will be the primary pixel-reading customization
points of the format readers. For now, these default to calling the
pointer-based varieties (so that old plugins keep working), but
eventually, the pointer-based varieties will be deprecated.
Notes:
* Omit the single-scanline and single-tile versions. Just keep the more
general multi-scanline, multi-tile versions, which of course people are
free to call with ranges that request just one scanline or tile.
* Omit the z coordintes for read_native_scanlines, since we know of no
volumetric format that allows reading of individual "scanlines".
* Omit the z coordinates from read_native_tiles to simplify the common
case of 2D images, but add a new read_native_volumetric_tiles() to
handle the volumetric case separately. This only needs to be supplied
for volume formats,
* The default implementations of the new methods is simply to call the
old, unsafe, raw pointer based versions. Thus, an ImageInput that still
supplies pointer based versions but not the span-based ones will not
break. Eventually, we will switch to the other way around, where all
ImageInput subclasses are expected to fully implement the span version,
and the pointer version will default to just be wrappers around the span
version for backward incompatibility. Way, way down the line, we'll
probably remove the pointer based varieties entirely.
Note that for this PR, we are only tinkering with the `read_native_`
functions that the ImageInput format readers supply. In a later PR,
we'll handle all the client-facing API calls to have span-based
varieties, as well as the analogous functions on the ImageOutput side.
In this PR, we do the overhaul on the JPEG and TIFF readers, to prove
that the whole scheme works. Over time, we will do the similar overhaul
of the other format readers.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Bump to the latest in several cases. Also make sure the INSTALL.md is in
line with what we support and/or test.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
When using `OpenImageIO_BUILD_LOCAL_DEPS` environment variable to build
all OIIO's dependency locally, OIIO's build script specifies a set of
default versions of the dependent libraries. In particular,
`ebiggers/libdeflate` is set to `1.20`.
This 1.20 version of libdeflate fails to build on platforms where AVX512
is not available and the make tool chain did not correctly tell the C
compiler about it.
```
-- Building package TIFF locally
-- Building package libdeflate locally
-- Building local libdeflate 1.20 from https://github.com/ebiggers/libdeflate
/usr/tmp/ccCelbo5.s: Assembler messages:
/usr/tmp/ccCelbo5.s:649: Error: unsupported instruction `vpdpbusd'
/usr/tmp/ccCelbo5.s:650: Error: unsupported instruction `vpdpbusd'
```
This issue is fixed in libdeflate Merge Request 389
https://github.com/ebiggers/libdeflate/pull/389, by explicitly calling
CMake facilities to detect whether AVX512 is present, and set compiler
preprocessor flags accordingly in the cmake file. This libdeflate MR is
after version 1.20.
To fix OIIO's build error when building this dependency locally, one can
either set `libdeflate_BUILD_VERSION` to include this patch, (for
example `export libdeflate_BUILD_VERSION=1.23`) or change the default
version in OIIO's local dependency build script to 1.23. This PR makes
the change in OIIO's local dependency build script.
Assembler build error resolved after using libdeflate v1.23 for local
TIFF build.
Signed-off-by: lji <lji@ilm.com>
The Intel Macs had this problem last week and #4668 fixed it. Now
suddenly the ARM Macs had it this week. Seems to be something changing
on the skbuild side.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I still think that the "abi break" is a false positive. But I really
didn't need the new constructor that seemed to confuse the abi checker,
so just to be extra sure that there is no break that I simply
misunderstand, change to make abicheck happy.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
After merging #4672, we noticed some things that could be improved, so
this is a further revision (and partial retraction).
There was already an oiiotool flag, `--metamerge`, which in retrospect
make it seem unnecessarily confusing to add --mergemeta. So this PR
removes this extra new command, instead simply piggy-backing on the
existing --pastemeta by adding to it optional modifiers:
- `:merge=` controls the merge strategy, 0 (default) means discard the
existing metadata entirely and use only the pasted, matching the
original --pastemeta behavior; 1 means merge nondestructively (no
existing metadata will be altered or removed); 2 means destructive
merge, where pasted metadata will overwrite existing metadata having the
same name, but conflict-free existing metadata will remain.
- `:pattern=` regex selects only some metadata.
Additionally, we noticed an interesting corner case that surprised users
when testing the new metadata merging feature from the last PR: What
happens if the metadata source image has fewer subimages than the pixel
source images? The behavior generally for oiiotool is that when binary
operations (those taking two images as input) are on images with
differing numbers of subimages, the output gets the number of subimges
as the FIRST input (assuming -a instructed us to consider subimages at
all). But that's not what we want here! If the metadata source has one
subimage and the pixel source has many, clearly the intuitive result is
for that one metadata set to be pasted into all of the subimages from
the pixel-supplying image.
So we fix that so that for --pastemeta, it's the second input image (not
the first) that takes the lead in determining the number of subimages we
will process. This in turn took a bit of extra infrastructure fixing in
the OiiotoolOp class that controls this logic.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Clarify that IB::copy_metadata and oiiotool --pastemea fully replace all
existing metadata with that of the other IB.
Add a new IB::merge_metadata() and oiiotool --mergemeta that merges in
metadata from the other IB, preserving metadata in `this` that was not
in the other IB. Optionally, metadata that existed in both can be
overridden. Optionally, a subset of metadata to be merged can be
selected via a regex.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Give names to some CI steps that didn't have them -- makes it nicer to
read the logs and CI status dashboard.
* Make sure no job nametag is a substring of another, or it might find
the wrong cache, which is based on the job name.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The following command fails for images with mip map levels due to a bug:
```
oiiotool mipmap_images.*.exr --siappendall -o all_images.exr
```
If the option `-a` is not appended to the command, mip levels higher
than 0 should be ignored, instead it just segfault due to out of bound
access; I just fixed the it in the PR cf. [my comment
below](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/4671#discussion_r1999705885).
Signed-off-by: Basile Fraboni <basile.fraboni@gmail.com>
The default value for m_bitspersample should be 1 when
TIFFTAG_BITSPERSAMPLE is not present in the directory.
Signed-off-by: vernalchen <shaochun.chen@autodesk.com>
Additions to `OIIO::span<>`:
* New flavors of `as_bytes()`, `as_writable_bytes()` that take a raw
pointer + length, turning them into `cspan<byte>` and `span<byte>`,
respectively.
* `span_cast<T>()`: "cast" a span of one type into a span of a different
type covering the same memory. Use with caution!
Safety improvements to existing functionality:
* front(), back(): Add a debug assertion to validate that there is at
least one element.
* first(), last(), subspan(): clamp offsets and counts so that the
result can't possibly return a span referencing memory that wasn't
inside the original bounds.
* The copy constructor that takes a non-matching span was much too
flexible. Now it uses enable_if to ensure that it can't be used to
change data types or lengths, it can only turn span of non-const into
span of const, and span of fixed extent into span of dynamic extent.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* The GitHub account has changed (the old one redirects, but still,
let's update to the current canonical name).
* Update the version we use in CI and for "autobuild" to the latest
tagged release instead of one that's years old.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This used to work until a few days ago. Then it stopped. I think that
GitHub changed the Mac runner configuration in some way that broke it,
now it's complaining about not being able to find ninja, so easily fixed
by instructing it to generate regular makefiles instead.
Along for the ride: If ONLY the wheel making workflow changes, there's
no need to rerun the regular CI, docs, or analysis workflows.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
From an array of known size, we can implicitly construct a span of
dynamic length, or of fixed length equal to the array length. Use
template enable_if trick to restrict to these cases.
Previously, the constructor would have allowed a fixed-length span to be
constructed from an array of differing length, which should obviously
have been disallowed.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Also detect patch level version. E.g. useful when checking for included
security fixes.
I ran `oiiotool --help` manually, and checked that the macro is
available already in 2.1.0.
Signed-off-by: Joachim Reichel <43646584+jreichel-nvidia@users.noreply.github.com>
There were some psd/psb files that were not opening with oiiotool, failing
on the signature.
Some keys were missing for the additional layer information, leading to
the pointer to misalign when reading the file, expecting the next 4
bytes rather than 8 bytes where needed for parsing.
No additional test suites added.
After deliberating, it is unclear if we should have a strict mode, or
keep parsing the rest of the file after a failed signature, given that
in this case the signature is not the part that's actually causing the
issue. Might be useful as a future feature, if there is an use case for
that?
---------
Signed-off-by: Lydia Zheng <lydiamzheng@gmail.com>
Signed-off-by: Lydia Zheng <36296057+lydia-zheng@users.noreply.github.com>
The current width/height limit enforced by the `check_open` API is too
low for PNG files. Each dimension can technically be up to `2^31 - 1`
but using that value would defeat the purpose of the check here. Use `1
<< 20` like what is used for TIFF and EXR instead.
The situation was triggered while trying to write an 85000x28400 px
image, which is smaller than 65536*65536 but the dimensions are checked
individually rather than as a total pixel count or similar.
Also looks like the PNG output was allowing too many channels so fix
that too.
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
Since C++17 defines std::size(C) (defaulting to returning C.size()), we
don't need to define our own now that C++17 is our minimum.
It seemed harmless to define it anyway for quite a while... but then a
few days ago, the GitHub Windows CI runners upgraded their version of
MSVS, and we started getting some very odd test failures that I was able
to trace to std::size somehow returning the wrong size in some cases,
and printf debugging revealed that the one we supplied wasn't even the
one being called.
I can't say I understand exactly what was going wrong. I was debugging
by repeatedly submitting small changes with debugging scaffolding to GH
to see how it works in a CI run, since I have no access to a local
Windows machine (at about 1/2 hour turnaround time for each iteration!).
Once I discovered that not supplying the redundant definition somehow
fixes the problem, I couldn't justify spending more of my weekend on
this to find out why. "Because MSVS" is a sufficient explanation for a
lot of weirdness in the computing world.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Old: Do a build with target 'clang-format', relying on our build system.
But that means that we need to do the cmake config step, and need a
bunch of dependencies set up so that the config doesn't fail.
New: Make a new run-clang-format.bash utility to issue the commands for
the right files. No "build" is requred, and the only dependencies we
need are the minimum it takes to run clang-format itself.
These changes reduce the approximate total elapsed time of a CI
clang-format job from nearly 7:00 to around 0:45.
Also, I noticed that we never had been clang-format testing the .cpp and
.h files in the testsuite itself. So enable that now, and commit the
small amount of reformatting that it identified.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
When test building the docs, lock down breathe and furo versions to
exactly what we use on RTD.
Also, to speed up the workflow: run on native Ubuntu rather than on an
ASWF container (eliminates needless container download), and make sure
that we don't waste time installing dependencies that we don't need.
These changes speed up the docs workflow from around 3:40 to around
2:00.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The main thing here is that it turns out that when trying to parse the
ICC profiles from Photoshop PSD/PSB files, we weren't storing the
attributes in the correct way. Everything else, we shoved in both the
"common" and "composite" imagespecs, but we left ICC data out of the
composite spec.
While I'm at it, I made two other ICC related changes:
* Among the several formats with ICC profiles, we were inconsistent
about whether we report errors for corrupt ICC profiles and treat the
whole image as broken, or just ignore the errors entirely. Fixed in
various places to make a consistent policy of always checking the
errors, and reporting/terminating if the "imageinput:strict" global
attribute is true, allowing it to just ignore the broken ICC record if
that attribute is false.
* Found a spot in JPEG handling of ICC where we were dangerously
trusting of the sizes of things. Replace a risky memcpy with a safer
spancpy that can't overrun the buffer bounds when copying.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Replaces the system() call with posix_spawn() on macOS to preserve
environment variables. This is essential for Debug builds using
DYLD_IMAGE_SUFFIX=_debug to prevent Qt from loading mixed
configurations.
## Tests
Run the iv application in background mode on macOS without the -F flag,
using a Debug configuration. Previously, the system() call did not
inherit the parent environment variables, causing a mix of Debug and
Release versions of Qt to be loaded.
---------
Signed-off-by: Mikael Sundell <mikael.sundell@gmail.com>
Existing versions could return the expected bytes for a scanline, tile,
or image for either the native per-channel types, or the single
composite channel type.
The new flavor lets you pass in any data type and it tells you how many
bytes you'd need for a scanline, tile, or image if all channels were the
passed type. (As a special case, the answer for TypeUnknown is the
"native" size, same as `thing_bytes(true)`).
This is handy for computing how big your buffer needs to be to hold a
type-converted scanline or tile.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Always use patch-specific SO versioning for scikit-build-core-based
builds, even when OpenImageIO_SUPPORTED_RELEASE=ON.
Fixes#4621
Ensures that Python-based builds only bundle a single copy each of
libOpenImageIO and libOpenImageIO_Util shared libraries.
## Tests
Previously, `$ uv sync -v --reinstall
-Ccmake.define.OpenImageIO_SUPPORTED_RELEASE=1` would bundle both
.so.3.0 and .so.3.0.3 copies of libraries when installing to a local
virtual environment.
With this fix, only .so.3.0.3 copies of libraries are installed.
Signed-off-by: Zach Lewis <zachcanbereached@gmail.com>
Also, make the wheel workflow work on pushes when the branch name
contains the substring "python" or "wheel", and when any of the cmake
build files in src/cmake (which control building of dependencies,
among other things), even if none of the python bindings themselves
are touched. This means that if you suspect your work might cause a
wheel to break even though you're not modifying the python bindings
directly, you can ensure that the wheel workflow will run on CI simply
by naming your branch to contain one of those words.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Included two additional metadata boolean checks for constant and black
frame images, so that handling for those image cases are facilitated.
IS_CONSTANT and IS_BLACK return 1 when true, and 0 when false.
Signed-off-by: Lydia Zheng <lydiamzheng@gmail.com>
There was a bug in span_from_buffer when strides were negative
(basically, we were subtracting, when adding was correct since those
strides were negative values).
Also added a negative stride test to imagebuf_test.cpp. Embarrassed that
we didn't already have this, since it would have caught the problem.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fill in OpenEXR lineOrder attribute when parsing the header in
OpenEXRCoreInput::PartInfo::parse_header()
---------
Signed-off-by: vernalchen <shaochun.chen@autodesk.com>
When you use frame or view wildcards, for example:
oiiotool foo.%04d.tif -o bar.%04d.exr
we noticed that if NO files matched, no iteration would occur. You would
get this output:
```
$ oiiotool foo.%04d.tif -o bar.%04d.exr
oiiotool WARNING : oiiotool produced no output. Did you forget -o?
```
But maybe this is not helpful and it should be some kind of error to ask
for a match and not get one? Or at least give a more clear explanation
of what happened.
This PR detects the case where there is no match of the wildcard at all,
warns about that, and then tries interpreting the command line literally
(no wildcard expansion). So now the output is
```
$ oiiotool foo.%04d.tif -o bar.%04d.exr
oiiotool WARNING : No frame number or views matched the wildcards
oiiotool ERROR: read : File does not exist: "foo.%04d.tif"
Full command line was:
> oiiotool foo.%04d.tif -o bar.%04d.exr
```
Closes#4623
Signed-off-by: Larry Gritz <lg@larrygritz.com>
As part of the recent big python wheel merge, compiler.cmake got changed
in a subtle way in how it sets CMAKE_INSTALL_RPATH.
This broken iv for me on a Mac with dynamic libraries. Strangely (and
I'm too lazy to chase down exactly why), oiiotool seems fine. So this
was missed entirely by the CI, which never runs 'iv', since it's an
interactive app. (Note to self: we should find some way to at least
verify that it can execute as part of the testsuite.)
Anyway, I believe that this change adds the install lib area back to the
path, and it does seem to repair my ability to run iv on my end.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The implementation of IB::pixeltype() was:
- Make sure at least the spec had been read (lazy read).
- If there are "local pixels", return the pixel type from the spec,
otherwise return the cache pixel type.
But there is a flaw in this logic: If only the metadata has been read
but not the pixels, the local pixels might not be allocated yet, but
also there is not a cache and no cached pixel type, so pixeltype() will
return UNKNOWN even though we do already know what type of pixels will
be stored when they are eventually read.
The correct logic is:
- Make sure at least the spec had been read (lazy read).
- If it's a cached image, return the cache pixel type, otherwise return
the pixel type from the spec (which will be correct because we
definitely have read the spec at this point).
Also, for legit cached images, ensure that cachedpixeltype is set upon
read_spec() and doesn't have to wait for a full read().
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We were only running the wheel workflow on pushes of TAGS, but this
change also will run it when we push to release, and dev-3.* branches.
My usual release workflow is to first push the "release" branch, watch
for it to pass CI, then add the tag and push the tag and draft the
release. If a problem with the wheel workflow isn't discovered until
we've already pushed the tag, then a fix will require a new tag and new
release (we never re-tag a release). So this change will allow us to
catch any problems with the wheel worflow BEFORE we actually irrevocably
make the tag, since it's no big deal to add corrections to the branch
before tagging. It still won't upload the wheels until the tag itself is
pushed.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Use the publicly-available ARM-native linux runners for CI checks and
building Python wheels
---------
Signed-off-by: Zach Lewis <zachcanbereached@gmail.com>
Quite often I find myself debugging code or CI specific to just one
platform. It makes for much longer iteration for a huge CI job that
reruns lots of tests that won't be affected by the changes I'm
experimenting with. Sometime I make a commit that places `if: 0` in
strategic places to exclude irrelevant batches of tests. But this is a
pain to add, and has to be removed again to re-enable the tests prior to
turning the experiment into a proper PR.
This PR, then, adds some logic to ci.yml that lets me do the same just
with special branch names, rather than needing to edit ci.yml. Any
branch you push containing the substring "windows-only" will not run any
Linux or MacOS CI tests. And the analogous behavior is added for
"macos-only" and "linux-only".
This shouldn't affect any CI jobs in which people are not purposely
using those special branch names. You should NOT use these special names
for branches from which you submit PRs, since obviously we want every PR
to validate on all CI combinations. But it is very helpful for topic
branches on your own fork as you are developing and iterating over many
CI runs but only need results from one platform.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This got lost with the 2-level namespace changes in #4567, but is used
by e.g. OpenUSD. Might as well not break the API here.
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
Implement FR from #4546
Add a new `--layersplit` command to `oiiotool` to split an image into
its channel-name-based layers onto the stack.
The extracted layer names are then stored in the `oiio:subimagename`
metadata (in case it is needed for later use, e.g merging these "layers"
as sub-images), and the extracted channel names replace the old channel
names in the new images.
Example:
an image with channels `R, G, B, A, diffuse.R, diffuse.G, diffuse.B`
will be split into two images, the first one with channels `R, G, B, A`
and the second one named `diffuse` with channels `R, G, B`.
> Note: we did not implement the `--layertosi` command suggested in the
FR as it can already be done with a combination of `--layersplit` and
`--siappendall`
We add a test in the test suite that takes a "multi-layer" image, splits
the layers and merges them as sub-images. The final image should have 3
sub-images.
---------
Signed-off-by: Loïc Vital <mugulmotion@gmail.com>
### Summary
This PR is the spiritual successor to #4011. It implements a
`scikit-build-core`-based python build-backend, making it possible to
use `pip install .` to build from source; and it adds a Github workflow
for building with `cibuildwheel` and publishing to pypi.org binary
distributions (bdists) of the Python module / extensions / CLI tools for
cpython 3.8-3.13, across major operating systems and architectures.
When you `pip install OpenImageIO`, pip attempts to retrieve an
OpenImageIO bdist from pypi.org for the host's platform / architecture /
Python interpreter. If it can't find something appropriate, pip will
attempt to build locally from the OpenImageIO source distribution
(sdist), downloading and temporarily installing cmake and ninja if
necessary.
### PEP-Compliant Packaging: `pyproject.toml`
The `pyproject.toml` file is organized in three parts:
1. **Package metadata**: standard attributes identifying and describing
the Python package, its run-time and build-time requirements,
entry-points to executable scripts, and so forth.
2. **scikit-build-core options**: governs how `pip install ...`
interacts with `cmake`.
3. **cibuildwheel options**: additional steps and considerations for
building, packaging, and testing relocatable wheel build artifacts in
isolated environments.
### Additions to `__ init __.py`
Previously, we were using a custom OpenImageIO/__ init__.py file to help
Python-3.8+ on Windows load the shared libraries linked by the Python
module (i.e., the .dll files that live alongside oiiotool.exe under
$PATH).
This PR adds an additional method for loading the DLL path, necessitated
by differences between pip-based and traditional CMake-based installs.
It also adds a mechanism for invoking binary executables found in the
.../site-packages/OpenImageIO/bin directory. This provides a means for
exposing Python script "shims" for each CLI tool, installed to
platform-specific locations under $PATH, while keeping the actual
binaries in a static location relative to the dynamic libraries. Upshot
is, in `pyproject.toml`,
each item under `[project.scripts]` is turned into a Python script upon
installation that behaves indistinguishably to the end user to the CLI
binary executable of the same name.
### Relocatable Binary Distributions with `cibuildwheel` + `repairwheel`
[cibuildwheel](https://github.com/pypa/cibuildwheel) is a widely-used
tool for drastically streamlining the process of building, repairing,
and testing Python wheels across platforms, architectures, interpreters,
and interpreter versions.
Additionally, the cibuildwheel-based builds set CMAKE_BUILD_TYPE to
"MinSizeRel" to optimize for size (instead of speed) -- this seems to
shave ~1.5MB off each .whl's size, compared to "Release"
### "Wheels" Github workflow
I straight-up copied `.github/workflows/wheel.yml` from OpenColorIO and
made a few OIIO-specific modifications. When pushing a commit tagged
v3*, the workflow will invoke a platform-agnostic "build sdist" (source
distribution) task, followed by a series of tasks for building OIIO
wheels for cpython-3.8-3.13 on Windows, Linux (x86_64 + aarch64, new
libstdc++), and MacOS (x86_64 + arm64) and persisting build artifacts;
followed finally by a task for publishing the build artifacts to
pypi.org
Note: For the sake of simplicity and troubleshooting, I've made as few
changes to OpenColorIO's wheel.yml as I could get away with; but in the
future, we can also build wheels for the PyPy interpreter, and possibly
pyodide.
Note: A "trusted publisher" must be set up on pypi.org. See
https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/
### Other Changes
I made some minor adjustments to `pythonutils.cmake` and
`fancy_add_executable.cmake` that only affect scikit-build-core-based
installs:
- -- namely, on Linux and macOS, I'm setting the INSTALL_RPATH property
to point to the relative path to the dynamic libraries, for the Python
module and CLI tools, respectively. This helps ensure that pip-based
builds and installs from source (as opposed to installs from repaired,
pre-built wheels) yield relocatable, importable packages, without
needing to mess with $LD_LIBRARY_PATH etc.
## Tests
`cibuildwheel` tests if `oiiotool --buildinfo` runs. If that command
elicits code zero, it means the "oiiotool" Python script installed by
the wheel is able to `import OpenImageIO`; that the actual binary
executable `oiiotool` is properly packaged and exists in the expected
location (e.g., at `.../site-packages/OpenImageIO/bin`); and that all
runtime dependencies are found.
## Inspiration, Credit, Prior Art
- @aclark4life's and @JeanChristopheMorinPerso's efforts + direction +
discussion + advice. See
[#3249](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/3249),
and
[#4011](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/4011),
as well as JCM's
[python_wheels](https://github.com/JeanChristopheMorinPerso/oiio/tree/python_wheels)
and
[python_wheels_windows](https://github.com/JeanChristopheMorinPerso/oiio/tree/python_wheels_windows)
branches. This PR is an attempt to leverage OIIO-2.6+
self-building-dependency features with # 4011's minimalist and modern
approach to packaging.
- OpenColorIO -- I tried to copy as much as I could from @remia et al's
fantastic work with all things wheels-related. The __init __.py
modifications, the way we're wrapping the CLI tools, and the github
Wheels workflow are lifted almost-verbatim from OCIO. Insert pun about
reinventing the wheel here.
- @joaovbs96's help and patience with testing stuff on Windows.
---------
Signed-off-by: Zach Lewis <zachcanbereached@gmail.com>
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
Signed-off-by: Basile Fraboni <basile.fraboni@gmail.com>
Signed-off-by: Vlad (Kuzmin) Erium <libalias@gmail.com>
Signed-off-by: Joseph Goldstone <joseph.goldstone@mac.com>
Signed-off-by: Darby Johnston <darbyjohnston@yahoo.com>
Signed-off-by: Jeremy Retailleau <jeremy.retailleau@gmail.com>
Signed-off-by: zachlewis <zachlewis@users.noreply.github.com>
Co-authored-by: Larry Gritz <lg@larrygritz.com>
Co-authored-by: Anton Dukhovnikov <antond@wetafx.co.nz>
Co-authored-by: Basile Fraboni <basile.fraboni@gmail.com>
Co-authored-by: Vlad (Kuzmin) Erium <libalias@gmail.com>
Co-authored-by: Joseph Goldstone <joseph.goldstone@mac.com>
Co-authored-by: Darby Johnston <darbyjohnston@yahoo.com>
Co-authored-by: Jeremy Retailleau <buddly27@users.noreply.github.com>
Co-authored-by: Jean-Christophe Morin <38703886+JeanChristopheMorinPerso@users.noreply.github.com>
Adds attributes to the input's ImageSpec for the black level and bit depth
when the raw bayer buffer is requested.
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
Fixes a bug in demosaicing where the code ignores the ROI channels.
The old behaviour was to always decode into the first 3 channels.
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
Perform the endian byteswap on the correct buffer area for PSD RLE
decoding. The original code used `dst` but that pointer is modified from
its original value as it progresses through decoding.
Fixes#4599
---------
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
Also some very minor code cleanup that was helpful in setting up the
test cases and getting better code coverage.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
TL;DR: In main, change the enclosing namespace from `OpenImageIO_v3_1`
to a 2-level scheme: `OpenImageIO::v3_1`. This can't be backported to
the current release for obvious reasons; it is a forward-looking
change only.
More ponderous explanation follows:
We have for some time had a single namespace that contains the
version, defaulting to `OpenImageIO_v3_0` in the current release, for
example. A namespace alias, `OIIO`, always aliases the current
namespace, so client applications can just say `OIIO::foo` without
needing to change code for every minor release (let alone if a
build-time option sets up a custom namespace).
By bumping the namespace for every minor release, we make the symbol
names themselves enforce a rigid ABI compatibility test, so you can't
accidentally compile against one minor release and link against an
older minor release. This gives us the freedom to break the ABI (link
compatibility) for each minor release, without subtle user errors.
(Doing it wrong makes a total failure to link, which is hard to miss.)
But being able to introduce ABI changes annually by changing the
enclosing namespace comes at the cost of a *complete* ABI compatibility
break with every minor release. Even classes or functions that haven't
change at all will be incompatible by virtue of their changed symbol
names.
This is an unfortunate limitation, and in an ideal world, we would
like downstream users to be able to upgrade to a newer minor release
more painlessly, and confident that they could even relink or perhaps
compile with a request to use an old ABI.
I haven't fully worked out all the details, or even if this is going
to be worth the trouble, but I think that a change we can introduce
now that will allow more flexibiity in the future is to switch to a
2-level namespace scheme, for example, `OpenImageIO::v3_1` instead of
the current `OpenImageIO_v3_1`.
The reason this might be helpful in the future is that we can use
"inline" namespaces that default to finding things that are in, say,
OpenImageIO::v3_2 without needing to specify it explicitly, while
giving the ability to explicitly give an alternate inner versioned
namespace. I haven't implemented that part here, it's reserved for
future expansion (though I have experimented with it, it does work,
but I want to discuss separately whether or not to actually do it).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Bump the 'latest' test to test the actual latest, 1.15.
* Be sure that the 'oldest' tests actually test the oldest we support,
1.8.
* Sprinkle around a couple other versions that we don't test currently.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Users can enable the legacy behavior by setting env var
`OIIO_PYTHON_LOAD_DLLS_FROM_PATH=1`.
This commit also changes the environment variable name from
`OIIO_LOAD_DLLS_FROM_PATH` to `OIIO_PYTHON_LOAD_DLLS_FROM_PATH` to match
the convention used by OCIO.
This PR addresses issue #4519
---------
Signed-off-by: Zach Lewis <zachcanbereached@gmail.com>
Implements X-Trans pattern demosaicing using a simple bilinear
algorithm. Adds the Xtrans pattern layout to the ImageBuf attributes in
the raw reader plugin.
I have implemented unit tests in imagebufalgo_test.cpp which cover all
permutations of the Bayer and X-Trans layouts, as the X-Trans pattern is
a bit too complicated to generate using oiiotool.
---------
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
For reasons that are unclear (maybe a change to which python versions
are installed where on the GH runners?) this seems to have broken within
the past couple days, in a way that did not correspond to any change we
made on the OIIO side.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* When the script was run on aarch64, it was actually hard-coded to
retrieve a version older than we currently require as a minimum. Use the
CMAKE_VERSION hint just like we do for x86_64.
* Bump the default version that this script makes to the latest, 3.31.3.
(This does not change OIIO's minimum required cmake version, it merely
changes the default if this hepler script that we use for CI is run
without any specific version requested.)
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This improves the mediocre testing line coverage in nullimageio.cpp from
50% to 95%.
* Test writing
* Test reading of tiled and MIP-mapped null images
* Test setting and reading of attributes
* Simplify some code
* Along the way, identified and fixed a nullimage bug -- it was
answering that it supported "rectangles" but had no write_rectangle
method. Testing works!
Signed-off-by: Larry Gritz <lg@larrygritz.com>
See discussion
https://github.com/AcademySoftwareFoundation/OpenImageIO/discussions/4054
and PR
https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/4315
In PR 4315, we fixed PNG read/write to do the required premultiplication
in linear space (that is, linearize, multiply by alpha, then go back to
the sRGB or gamma space). Which we really believe is "correct." Except
that maybe there are a ton of incorrectly made PNG files out there
(maybe most of them?) where partial alpha pixels had their
premultiplication occur on the sRGB or gamma values directly.
In this patch, we partly revert, allowing both potential behaviors,
controlled by an attribute "png:linear_premult", which instructs the PNG
driver to do any premultiplication in linear space if it's set to
nonzero. It can be set globally (via `OIIO::attribute()`), as well as
set/overridden on any individual file by setting the attribute in the
configuration hints for an ImageInput or in the spec for an ImageOutput.
As presented in this patch, we're setting the default to 0, meaning that
by default we are reverting back to the old behavior of doing the
premultiply of partial alpha pixels on the direct values intead of in a
linear space. Applications or sites that are confident that any PNG
files they encounter are "correct" can set the attribute to do the
multiplication linearly.
I'm not 100% confident about the default, and so am very happy to
entertain arguments for keeping the default set to do the multiplication
in linear space.
I had to change an internal spin mutex to a recursive mutex in order to
address a latent misdesign that was made symptomatic by this change.
Basically, asking for an attribute inside an ImageInput::init could make
a deadlock because certain attribute queries (that catalogue the list of
plugins) might instantiate those plugins, thus causing their init
functions to run, leading to recursive dependence on the mutex that
guards attribute queries.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Co-authored-by: Brecht Van Lommel <brecht@blender.org>
* Ptex testsuite entry had been accidentally disabled somewhat recently
when we switched to using Ptex exported cmake config instead of our own
find module, because the "found" variable changed from PTEX_FOUND to
Ptex_FOUND.
* Change the test to print stats, thus forcing the ptex test to read
pixels instead of just header info, which improves the testing coverage
of ptexinput.cpp.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Upgrade upload-artifact action, old one is being deprecated.
* Update checkout and cache actions to latest.
* Fix workflow warning: save-always is not really an input to
actions/cache.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Chipping away at minor warnings and areas identified by static analysis
as potentially problematic or hard to analyze.
* benchmark.h: suppress warnings about uninitialized Benchmarker
members.
* dpxoutput.cpp: change some raw pointers to spans
* dead code removal (color_ocio.cpp, filesystem.cpp, sysutil.cpp)
* exclude build-scripts from static analysis
* some misc warning suppression
Signed-off-by: Larry Gritz <lg@larrygritz.com>
It escaped our notice before that the IPTC spec dictates length limits
for many fields. Getting this wrong can confuse some software, including
Photoshop, apparently.
This patch enforces length limits wherever we could figure them out from
the IPTC spec -- it will simply truncate those strings that are too long
before writing them to a binary IPTC tag.
For this reason, we also are ending the practice of automatically
translating several IPTC fields to and from what we figured were the
equivalent generic metadata names. This was perhaps a dubious practice
to begin with, but now the enforcement of length limits for IPTC (but
not the generic metadata) makes it even more frought. There is probably
nobody depending on this behavior (and maybe few OIIO users depending on
IPTC support at all?), so now anybody purposely using IPTC metadata is
fully responsible for setting it and dealing with any issues of whether
it's "out of sync" with any other metadata that OIIO stores in or reads
from a file.
Fixes#4342
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
It was syntactically correct, but without an actual variable, there was
no RAII scope and so it destructs immediately, causing the timers to
capture 0 time.
Found by Sonar warnings.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Fix missing secrets that accidentally disabled the analysis workflow a
few commits back. After splitting into reusable steps, we needed the
main analysis workflow to let the steps workflow inherit secrets.
* Run analysis on TOT dev branches.
* For analysis, only build OIIO as debug, not dependencies.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The logic for the GHA steps was repeated across multiple workflows and
platforms. Consolidate it all into a common "ci-steps" workflow, and
have all the other workflows reference that when possible. This should
hopefully make maintenance of the CI scripts easier in the future, with
less redundancy of the step logic.
The only major place where we deviate is that we have a separeately
specified set of steps for the "old" VFX Platform 2021 and 2022 based
tests (because we need to specify different container arguments to it).
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We don't do anything with this at the present time, but this PR reserves
and documents this attribute for future use.
The intent is to be able to set whether we want image readers to try
being as tolerant as possible when reading a file with flaws (press on
and see if the rest of the file is ok?), or be more conservative and
abandon reading any file as soon as a corruption or invalid data is
encountered (because that might be a clue that the file is arbitrarily
corrupted or even maliciously constructed).
I documented it as defaulting to 0 (err on the side of being permissive
of bad input), with high-security applications being responsible for
setting it to 1. But it's open for debate if people think that a better
default is to be strict and let applications who want to be more
tolerant be responsible for accepting the risk and switching the mode.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#4552
Caught during fuzzing with address sanitizer.
The source of the problem was a corrupted/truncated pgm file. Several
minor modifications in this PR shore up various cascading errors that
followed. Not all were directly causal to the sanitizer trigger, in some
cases I fixed what appeared to be related areas.
* In imagebuf.cpp, any time we free the local pixel memory m_pixels,
also explicitly clear the m_bufspan that has a span representation of
the usable memory and its bounds.
* An extra check related to oiiotool --printstats to make sure that the
image is valid before passing along to stats collection.
* In pnminput.cpp, a better error message when we hit a premature end of
file.
With these fixes in place, we seem to get a graceful error message and
exit when running the POC that was provided with the bug report.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#4553
Caught during fuzzing with address sanitizer. The file appeared to have
a resolution so big it would not be able to satisfy the memory
allocation.
Solution: add the check_open to take an early abort if resolutions are
bigger than could possibly be valid.
Also have Strutil::stoi hande 32 bit overflow without UB overflow that
the sanitizer complains about (that was the other cascading error that
this same test case encountered in the sanitizer after the bad
allocation).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Affects jpeg, tiff, and other formats with embedded ICC profiles.
Corrupted/invalid tag offset and/or length values were causing us to
touch memory outside the memory allocated for the profile.
Fixes#4551
Not sure if this is wise or not, but for JPEGs, I also enabled some
previously commented-out code that has the effect of treating a file
with a verifiably corrupted/nonsensical ICC profile block as a failed
file. If people ever complain about this in the future, maybe we'll come
back and add some kind of global option that lets the app express how
tolerant OIIO readers should be about corruptions in files -- do you
assume that the whole file is bogus or malicious upon discovering the
first invalidity, or do you press on and hope you can get something else
useful from the pixels? Let's be cautious for now.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
When using pkgconfig the value of `<NAME>_CFLAGS_OTHER` can contain
compiler flags along with preprocessor definitions. If there is a
compiler flag and its passed to `target_compile_definitions` then it
will be treated as if it is a preprocessor definition.
Found a case where a statically compiled LibRaw with a statically
compiled Little-CMS caused `LibRaw_DEFINITIONS` to have `-pthread` in
its listing which was passed to the compiler as a definition,
`-D-pthread`. This caused a build failure with `auto-moc`.
Add an additional parameter for the `add_oiio_plugin` macro to take
`COMPILE_OPTIONS` and pass those to `target_compile_options` which can
differentiate between compiler flags and preprocessor definitions. Pass
`LibRaw_DEFINITIONS` to the CMake macro to prevent the above error.
Issue found while updating Little-CMS in
https://github.com/microsoft/vcpkg/pull/42187 and the change fixes the
build issue found there. This is an attempt to upstream it so the issue
is fixed.
Signed-off-by: Don Olmstead <don.j.olmstead@gmail.com>
Implements IBA::scale, an algo which takes two images, one of which is
required to be a single channel image, each channel of the other image
gets multiplied by that single channel, per pixel.
Note that this is subtly different from IBA::mul(), which is a
channel-by-channel multiplication, and so doesn't have a provision for
allowing you to scale all channels of one image by the single channel of
a second image.
I have added tests for the C++ code, python code, oiiotool, and the
documentation examples
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
OIIO 2.3.13 with PR #3307 changed
MultiByteToWideChar/WideCharToMultiByte usage to C++11 <codecvt>
functionality, but that has two issues:
1) it is *way* slower, primarily due to locale object access (on Visual
C++ STL implementation in VS2022 at least). Since primary use case of
these conversions is on Windows, maybe it is better to use a fast code
path.
2) whole of <codecvt> machinery is deprecated with C++17 accross the
board, and will be removed in C++26. I've kept the existing functions in
there since otherwise it would have been an API break, but really maybe
with OIIO 3.0 they should have been un-exposed. Too late now though :(
## Tests
Performance numbers: doing ImageInput::create() on 1138 files where they
are not images at all (so OIIO in turns tries all the input plugins on
them). Ryzen 5950X, VS2022, Windows:
- utf8_to_utf16 3851ms -> 21ms
- utf16_to_utf8 1055ms -> 4ms
Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
Primary cost of detecting whether a given file is valid jpeg2000 file is
the thread pool initialization and shutdown that `Jpeg2000Input::open`
does (the actual thread pool part is inside OpenJpeg code).
So add a dedicated `valid_file()` implementation that only needs to
check 12 bytes of the header. While at it, I changed already existing
`isJp2File()` to `is_jp2_header()` to better match naming conventions
used elsewhere, and instead of trying to handle both little and big
endian cases by manual repetition of two sets of magic integers, let's
do just byte comparisons with `memcmp` instead.
On my PC (Ryzen 5950X, SSD, Windows VS2022), doing
`ImageInput::create()` on 1138 files where they are not images at all
(so OIIO in turns tries all the input plugins on them):
- Before: **3.4 seconds** spent in `Jpeg2000Input::open` (1.9s
`opj_thread_pool_create`, 1.3s `opj_thread_pool_destroy`)
- After: **33 milliseconds** spent in `Jpeg2000Input::valid_file`
No new tests. I checked behavior on the official Jpeg2000 conformance
data set
(https://github.com/uclouvain/openjpeg-data/tree/master/input/conformance),
seems to work.
Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
The background here takes some explaining...
The ASWF's prepared Docker containers that reproduce a CentOS7-based
environment corresponding to VFX Platform 2021 and 2022 years contain a
glibc that is too old to run a newer version of "node" -- which GitHub
Actions themselves use. The old one is no longer receiving support, and
GitHub finally changed the policy that was allowing its GHA runners to
still run the old version that worked with those ASWF containers. The
new node version that is happy on the GHA runners requires a glibc newer
than the one in the containers. So that breaks all our CI runs in 2021
and 2022 VFX Platform configurations.
See this thread, if you dare:
https://academysoftwarefdn.slack.com/archives/C0169RX7MMK/p1732574400981949
Anyway, Jean-Francois Panisset proposed and tested a fix, which lets us
continue working (for now) by downloading an old node implemenation
(which is happy with the old glibc) and installing it atop the
conainers, so it's this old one that the GH actions will use. It feels
like it's all held together by chewing gum and duct tape, but it
unbreaks our CI and hopefully it will keep working until all the studios
have switched fully to RHEL/Alma/Rocky 9.x and we no longer care about
testing on CentOS-7 based containers or supporting the VFX Platform
years that are too old to be relevant to the new OS that the studios are
using.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Having such a document is one of the remaining steps to getting our
OpenSSF Best Practices silver level badge.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Having such a document is one of the remaining steps to getting our
OpenSSF Best Practices silver level badge.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Get rid of the FORTIFY options, which were too specific to one compiler
feature. Add OIIO_HARDENING that takes a level meaning:
- 0 : do nothing, not recommended.
- 1 : enable features that have no (or nearly no) performance impact,
recommended default for optimized, shipping code.
- 2 : enable features that trade off performance for security,
recommended for debugging or deploying in security-sensitive
environments.
- 3 : enable features that have a significant performance impact, only
recommended for debugging.
Default to 1 for optimized builds, 3 for debug builds (so will be
thoroughly tested by our sanitizer and other CI tests). Users that have
more stringent security requirements may choose to build with 2 even for
shipping code (they should benchmark to see if that is acceptable to
them).
These levels turn on a variety of compiler options that are recommended
for additional safety. We will add more as they are developed.
There are also some warning suppressions that needed to be added to code
in a few areas where it was unavoidable to use some constructs that
trigger the elevated safety checks.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
These commands, which are intended to alter the top image on the stack,
were each allocating and copying an image unnecessarily for each
operation. By changing the implementation to the right calls that know
it doesn't need to make a copy (already a feature I'd implemented in the
OiiotoolOp helper class) greatly speeds them up.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
When the exr "name" metadata (used for the names of subimages/parts)
existed but was the empty string, we were treating it like it didn't
exist (and upon write substituted our own "subimageXY" name, as we are
wont to do for unnamed parts). But now we've seen multi-part exrs where
the name is "" (exists, but empty string) and we want a read-write cycle
to preserve that. Even though it's not a very sensible or helpful part
name, it's technically legal, so we should honor it.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This is needed to port Blender's current JPEG IO code to using OIIO, but
is also a useful feature to have in general.
For reading, the code tries to parse comments as colon-separated
key-value pairs and sets metadata accordingly. For writing, this needs
to be explicitly enabled by setting jpeg:com_attributes to 1 in order to
avoid accidentally bloating files for existing applications.
Tests:
I've added a small (~10KB) JPEG file containing Blender metadata and a
basic test that parses it, checks that the metadata was read correctly,
writes it twice (once with and once without `jpeg:com_attributes`), and
then checks that those files are also parsed as expected.
In case you're wondering why the info for "no-attribs.jpg" still
contains one Blender attribute - that's because the first COM field is
still put into `ImageDescription` just like before, so even without
`jpeg:com_attributes` it ends up being written to the output file and
recognized during parsing.
---------
Signed-off-by: Lukas Stockner <lukas@lukasstockner.de>
The function libraw_filter_to_str() converts the `imgdata.idata.filters`
field to human readable form, by mapping a few hardcoded values. I found
the issue by converting images with disabled demosaicing, which writes
the Bayer pattern into the ImageBuf metadata. The Panasonic GX9 didn't
have any pattern recognised; The Canon R5 was incorrectly recognised as
GMCY.
The `imgdata.idata.filters` actually contains a sequence of 2-bit
values, where each represents an index in a 4-character long string
`imgdata.idata.cdesc`. I have modified the code to extract the channels
from `imgdata.idata.cdesc` as expected.
I have also disabled the normalisation code, which I believe was
incorrect, and never executed due to the issue above.
The current tests pass, however, I'll start a discussion on the piece of
code I have concerns with.
---------
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
* Add build recipes for certain dependencies needed for linking static
OpenColorIO libs: expat, minizip-ng, pystring, and yaml-cpp
* Add SOURCE_SUBDIR option to `build_dependency_with_cmake` that mimics
the behavior of the FetchContent option of the same name
* Tweak OpenColorIO build recipe to always build static libraries by
default.
---------
Signed-off-by: Zach Lewis <zachcanbereached@gmail.com>
Add an API to query available fonts based on family and style name. The
API can be used to build a UI where users can select a font from a list.
They consist of new queries for `OIIO::getattribute()`:
- `string font_family_list`
A semicolon-separated list of all the font family names that
OpenImageIO can find.
- `string font_style_list:family`
A semicolon-separated list of all the font style names that
belong to the given font family.
- `string font_filename:family:style`
The font file (with full path) that defines the given font
family and style.
Fixes#4503.
---------
Signed-off-by: peter.horvath <peter.horvath@autodesk.com>
Replacing 3x IBA + Helper function that generate 4 fulls size image
buffers with single unsharp_mask_impl() that use parallel_image() to
compute unsharp:
src + contr * (((src - blur) < threshold) ? 0.0 : (src - blur))
Added two pass 1D convolution for a kernels higher than 3x3
## Tests
```
ImageBuf sharped(input.spec());
const int repeats = 50;
std::cout << "Start sharpening\n";
auto start = std::chrono::high_resolution_clock::now();
for (int i = 0; i < repeats; i++)
{
//ok = ImageBufAlgo::unsharp_mask(sharped, input, "gaussian", 15.0f, 10.0f, 0.01f);
ok = ImageBufAlgo::unsharp_mask(sharped, input, "gaussian", 5.0f, 2.0f, 0.05f);
std::cout << ".";
}
std::cout << "\n";
auto part1 = std::chrono::high_resolution_clock::now();
std::chrono::duration<double> elapsed_part1 = part1 - start;
std::cout << "Elapsed time: " << elapsed_part1.count() << " s\n";
```
both single threaded (one IB at time) and multithreaded (multiply IB at
time) show pretty good speedup:
~30-40% with less memory use.
for 5x5 gaussian kernels two pass mode should add at least 20% speedup.
(if someone can do independent benchmark, will be great. As soon as I
had a big differences on them depend on real or synthetic use)
---------
Signed-off-by: Vlad (Kuzmin) Erium <libalias@gmail.com>
Initial feature request: https://github.com/AcademySoftwareFoundation/OpenImageIO/issues/4424
Add support in the `jpeg` plugin for reading Ultra HDR images using the reference codec `libultrahdr`: https://github.com/google/libultrahdr
In short, "ultra hdr" images are a clever extension of JPEG where the image file really is an old school JPEG file and will be interpreted correctly as such by old readers not aware of ultra hdr, but readers that are aware will see an extra piece of metadata that contains a gain map, that when applied to the base layer then yields an HDR image. Pretty clever approach!
Images used for testing during development:
https://github.com/MishaalRahmanGH/Ultra_HDR_Samples
---------
Signed-off-by: loicvital <mugulmotion@gmail.com>
The latest libheif patch release changed something very slightly that
requires us to update some saved reference output.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Doing some benchmarking and adding extra LoggedTimer instrumentation
(included in another PR), I discovered that quite frequently, the
in-memory flavor of ImageBuf was reading its image multiple times --
once upon construction, and then again upon later calls to read(), even
though the latter should have taken early outs when the read was
unnecessary.
So I put some additional mechanism for the IB to internally know if it
read the pixels from disk into the buffer, and more effectively make
calls to IB.read() be a no-op. To help facilitate this, I also clarified
in the docs that read() should not be used to move an already-read image
to a different subimage or MIP level or a different data type -- reset()
really should be used for that purpose. (I don't know that anybody ever
relied on read() to do reset's job, but assuming that they won't helps
eliminate some tricky cases.)
Also, I noticed that inside ImageInput::read_image(), depending on
whether or not a data type conversion is needed, it can either read
directly into the user's buffer, or must read the native data into a
temp buffer and subsequently do a copy-and-type-convert into the user's
buffer. The latter is obviously more expensive. The direct read approach
was used when the user asked for no type conversion (just to receive the
native data type in the file), but it neglected to take the same
shortcut if the user asked for an explicit type conversion, but it just
so happened that the requested type was the type in the file.
Additional changes:
The evaluation of whether the buffer was contiguous (for various
shortcuts) only applied to LOCALBUFFER style storage, not APPBUFFER.
Fixing these two problems dramatically speeds up some reads. A benchmark
I was looking at where it reads a 10k x 10k uint8 uncompressed TIFF file
into an ImageBuf while requesting uint8 data went from (timings on my
laptop):
function calls total
IB::read 2 0.505s (avg 0.25s)
II::read_image 2 0.478s (avg 0.24s)
to:
IB::read 1 0.172s (avg 0.17s)
II::read_image 1 0.172s (avg 0.17s)
So almost 3x faster. Most of which was due doing 2 reads instead of 1,
but still each individual read improved from 0.25s to 0.17s because of
the fix that lets us avoid the unnecessary buffer copies.
Grain of salt: this was a very large image, and it was stored
uncompressed (so NO decompression that can sometimes be a significant
amount of overhead), so you may not see this much improvement in real
world scenarios.
This is only scratching the surface, I believe there are lots of other
areas to optimize more carefully.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#4502.
This PR lets you specify fonts (for `IBA::render_text()` or `oiiotool --text`)
by family and style name, rather than only by filename of the font file.
The implementation is to add `init_font_families()` function, that iterates
over the available font files, reads family and style name via the freetype
API and caches them. The function is called from `resolve_font()` on
demand.
Added a new test to the oiiotool-text folder, that loads a font by the
name "Droid Serif Bold".
---------
Signed-off-by: peter.horvath <peter.horvath@autodesk.com>
The GHA macos-12 runner is deprecated and soon to be retired. It's
already flaky and they're doing purposeful outages to remind people (the
cloud equivalent of turning the lights on and off right before the bar
closes).
We still want to test earlier MacOS-12 compatibility to match VFX
Reference Platform, so use the macos-13 runner as the minmum but set
MACOSX_DEPLOYMENT_TARGET=12.0 to test that configurations.
Also, the MacOS-15 runner is in beta, so add that to the mix. (Along the
way, that also verifies that we run fine against python 3.13.)
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Because it's not in std::string_view.
If you must use this functionality (with caution about when it is safe),
then use the freestanding `OIIO::c_str(OIIO::string_view)` function
instead.
I would like to include this warning in 3.0.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes#4504
The PR removes the code that looks for font folders based on a prefix
and suffix list. Instead, adds all known system font folders per each
platform. The global "font_searchpath" attribute, `OPENIMAGEIO_FONTS`
and `OpenImageIO_ROOT` environment variables are still searched as
before, such as fonts directories relative to the currently running
binary. However $HOME and $SystemRoot folders (with all previous
suffixes) are not searched anymore, thus this is a breaking change, and
the documentation might need to be updated accordingly.
The search order did not change. It's still "font_searchpath" attribute,
environment variables, system font folders, folders relative to binary.
---------
Signed-off-by: peter.horvath <peter.horvath@autodesk.com>
I don't understand why, but for certain combinations of gcc, fmt, etc.,
we were getting exceptions from fmt. I finally caught it in the act! It
seems related to formatting specs that looks like `{:-20s}`. Maybe this
used to be allowed but isn't officially supported? Anyway, left
justification is default anyway, so there's no point to the `-`.
Removing it makes fmt happy, no more exceptions.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Newline in strings on Windows is encoded as '\r\n', that is currently
not handled by the text render code and so carriage return (\r) shows up
in the rendered text. As a fix we simply ignore this character.
We came across this issue while exporting user specified text from DCCs
to render on Windows.
I'm not sure how to make a proper test case for this. Current tests
specify a command as a string that is passed to oiiotool as arguments
and executed in the command line. But adding a newline to the text
breaks the command.
Signed-off-by: Peter Horvath <peter.horvath@autodesk.com>
Signed-off-by: peter.horvath <peter.horvath@autodesk.com>
The `y` argument in `OIIO::ImageBufAlgo::render_text()` should represent
the top and bottom edge of the text if vertical alignment is set to
`TextAlignY::Top` and `TextAlignY::Bottom`. However currently the
rendered text is off. The current code adjusts y to point to the top of
the text box before rendering the text, but it has to be adjusted to
point to the baseline of the text.
Tests
There is a existing test case, testing vertical alignment, but the
reference image seems wrong and has now been updated. See the "Topleft"
and "Rightbot" texts.
| Current | Fixed |
| -------- | ------- |
|
|
|
Note: The testsuite contains a reference image for freetype v2.4.11
(aligned-freetype2.4.11.tif). I don't have that version built, and
haven't been able to update the image. I'm wondering if this is still
used or could be removed. I guess CI will tell.
---------
Signed-off-by: peter.horvath <peter.horvath@autodesk.com>
Address a few small issues encountered during CMake configuration step
and during build using Makefile `BUILD_MISSING_DEPS` option enabled.
- Mismatch between expected libraw versions specified in
`build_libraw.bash` 0.21.3 and `externalpackages.cmake` 0.20.0
- OpenEXR link error in libutil tests
- Update top level Makefile to accept `BUILD_MISSING_DEPS` as string
instead of bool. Pass the string option down to CMake
Verified build completes with no errors: `BUILD_MISSING_DEPS=all make`
Signed-off-by: kuba <kuba456@gmail.com>
Detect libbz2 only if necessary, i.e., FFmpeg or FreeType is enabled.
This avoids an unnecessary dependency on libbz2 if these features are
disabled.
In particular, with vcpkg, OpenImageIO might pick up a dependency on
libbz2.so from the host system since bzip2 from vcpkg is not listed as a
dependency of OpenImageIO (if the FFmpeg and FreeImage features are
disabled, and bzip2 is not already installed for other reasons).
Signed-off-by: Joachim Reichel <43646584+jreichel-nvidia@users.noreply.github.com>
A logical error was causing us to not correctly cycle through the
successive MIP levels. Somehow, we missed this all these years.
You can tell that there was something wrong all along by examining the
testsuite/iinfo test output (which we have updated here) -- in the
old/wrong one, the statistics "FiniteCount" suspiciously stayed the same
for all MIP levels, which is clearly wrong. Now you can see that it does
correctly reflect the reduced resolution of each MIP level.
This bug was limited to 'iinfo' itself. There wasn't any inherent
problem with reading MIP levels directly with ImageInput or ImageBuf.
The line with the bug has been unchanged for 14 years! We were trying to
be clever about not double-reading, except that the only context in
which this was called, that would never have happened anyway, so that
cleverness (which was incorrectly implemented) was unnecessary.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Times we DON'T need to test rebuilding the docs:
* When just the ci.yml workflow changes (like when we bump what versions
of dependencies ci tests use).
* When tests changes only by altering their run.py.
But DO try rebuilding docs:
* When cpp files change that are outside of src/, such as the cpp files
in the testsuite that are used to supply code snippets that appear in
the docs.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I got part way through the OSL side changes to accommodate what we did
recently with TextureOpt and TextureOptBatch. Ugh, it's a lot harder
than
I thought and I don't know how to finish the job without postponing the
3.0 release even more, which is unacceptable at this point. So here I do
to TextureOptBatch to get out of this mess:
* "Version" TextureOptBatch like we did for TextureOpt (should have done
that all along).
* Revert "v1" to the same byte-for-byte layout we used to have, which
involves using `int` for some fields as it implicitly did before,
because
the new enum classes totally change the field sizes and struct layout.
* Sketch out in comments what v2 might look like, to more fully match
the
TextureOpt. The introduction of struct versioning means we can make that
change for 3.1, it doesn't need to wait years for 4.0.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
It's failing to get Mesa packages for some reason. Disable Qt and OpenCV
on this one test to work around it for now.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
These are all things long marked as deprecated (often since before the
official launch of 2.0) that we've waited for 3.0 to finally remove. Now
is the time.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Version the class for easier future expansion! Let's call it
`TextureOpt_v2` and we make a `using` alias so that the name
`TextureOpt` means this. We can then revise the struct in the future by
not losing link compatibility.
* The macro OIIO_TEXTUREOPT_VERSION (now 2) can be used to test which is
the current version.
* Get rid of TextureOpt-local enums by finally switching to the
freestanding ones in the Tex namespace. (We've wanted to do this for a
long time, but couldn't break compatibility.) Preserve API compatibility
for now with clever use of `static constexpr` and `using`.
* Make more efficient use of space in the TextureOpt by: field order
rearrangement to eliminate empty space, switching a couple of the enums
that needed only a few values to be `class enum xxx : uint8_t` so they
take only 1 byte instead of 4, switch `anisotropic` to be 16 bits (do we
really need an anisotropic ratio > 65535?), elimiate the time and
samples fields which have never been used (they were reserved for shadow
maps, which we never implemented).
I think this set of changes is minimally onerous to downstream projects
for the 3.0 release, while laying a path forward (via the versioning) to
allow future changes to the struct to happen with a lower pain
threshold.
I don't claim that this perfectly preserves downstream source
compatibility, but the fact that outside of these changes to
delcarations in texture.h, only 8 lines in the entire rest of the OIIO
codebase needed to change to accommodate them. So I think that the
impact downstream will be minimal. (OSL might be the exception, since
its LLVM code generation is exquisitely sensitive to the exact layout of
the TextureOpt struct, but I'm working on those changes, too.)
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This finally switches Strutil::format to use std::format conventions.
You can still access the old printf convention via
`Strutil::old::format()`, and can explicitly access the new convention
uniformly on both 2.x and 3.x via `Strutil::fmt::format()`.
It also removes a long-deprecated vformat function.
We've been waiting for 3.0 to make this non-back-compatible change.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
A few minor changes were needed to keep a working build, due to things
long ago rigged to be disabled as soon as the version bumped.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
A round of color management improvements (I hope).
We were pretty squirrelly about what "linear" meant. Get more exact. Now
we will treat "linear" as a legacy synonym for "lin_rec709" (sRGB
primaries, but linear response). Change file format readers to say
"lin_rec709" when they mean it, and not use the more generic alias
"linear".
"scene_linear", on the other hand, is the internal working color space
for math. Which concrete color space it is an alias for is determined by
the OCIO config in use.
Use "g18_rec709" and "g22_rec709" for rec709 + gamma 1.8 or 2.2,
respectvely.
A few new API calls help out: equivalent_colorspace(),
set_colorspace(spec,name), set_colorspace_rec709_gamma().
OpenEXR always tagged files as linear by default. It was too aggressive
about this, so with this PR, now it defaults to lin_rec709 only for
files that appear to be RGB and not have any other clues about the color
space.
Similarly, the raw reader was too eager to set the Exif:ColorSpace to
look like sRGB even when it wasn't.
Lots of reference outputs needed to be updated.
This is a potentially big behavior change. But I think it's for the
better.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I was profiling the builds and saw that modules with lots of template
expansion dominate the compile time. For example,
imagebufalgo_pixelmath.cpp alone took 290s to compile on my 2020
MacbookPro (!), and imagebufalgo_addsub.cpp took 84 seconds.
This is all due to the combnatorics of expanding IBA templates via the
DISPATCH macros in imagebufalgo_util.h separately for every type that
the arguments can be. But I claim that most combinations are rarely if
ever used. I mean, how often does anybody need IBA::add() to add an int8
image to an int16 image? So this PR rewrites those macros to simplify
the cases as follows:
* The common pixel data types are float, half, unint8, and uint16.
* Specialized versions are fully expanded only when the result and input
images are one of these types. Images not of one of those types are
first automatically converted to float to make them reduce to a common
case. That makes uncommon pixel data types like (signed) int16 not
expand the template, but rather convert to and from float and use the
float specializations.
* For binary and ternary operations (those with 2 or 3 image inputs), if
the pixel types of the inputs doesn't match, we make sure they both are
converted to float. So, for example, we don't need a specialized version
that adds a half image to a uint16 image -- just convert them to float
and use the common case. But we do specalize if the two inputs are both
the same common case, such as adding two uint16 images.
* Assume that commonly, the result image will either be float, or will
be the same pixel data type as the inputs. Other combinations trigger
assignment to a temporary float IB, then copying with convertion to the
uncommon use-supplied result buffer.
* Additionally, we cut down on a little bit more templating by moving
some "deep" methods from the type-templated ImageBuf::Iterator to its
type-generic non-templated base class IteraterBase.
The net result of all this is an awful lot less template expansion. With
this in place, my laptop compiles imagebufalgo_pixelmath.cpp in 97s (vs
290 before) and imagebufalgo_addsub.cpp in 26s (from 84). It takes a big
bite out of all the iba files, and reduces project-wide compile time by
over 10%, around 30s out of 300 for a fresh, uncached, optimized build
with 16 threads.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I refactored the companion project full of testing images, putting all
the clutter in the main directory into a subdirectory for each file
type. These are the main OpenImageIO changes needed as a result.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
When building OIIO as a subdirectory of a super-project (with
`add_subdirectory`), it is better to honor the output directories
specified by the parent project.
In our project, it was causing issues because OIIO libs were ending in a
different folder from USD plugins, which thereby failed to load
correctly in the build tree.
Signed-off-by: Luc Touraille <touraill@adobe.com>
Starting with OpenEXR 3.1.3, the preferred API call for setting
DWAA/DWAB compression level has changed. We never changed the OIIO side.
Luckily, OpenEXR seems to have kept respecting the old API calls we were
making (passing as an attribute). But this modernizes the approach, now
that we don't have OpenEXR 2.x support to maintain.
But in the process, we also fixed a bug! Turns out we weren't
propagating the compression properly -- if the compression name
was plain "dwaa", we ignored any "openexr:dwaCompressionLevel"
attribute, effectively.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Adds new feature: floating point exceptions, to imageinout_test binary.
Running `imageinout_test --enable-fpe` will run the test with FPEs enabled.
Currently it is catching a FP overflow in libjxl.
Once that is fixed, it may find other FPEs.
For now, this is a Linux-only feature.
```
$ ./imageinout_test --enable-fpe
```
Signed-off-by: Bram Stolk <b.stolk@gmail.com>
* Move RELEASING.md from the top directory to docs/dev
* Move 0.x and 1.x CHANGES documents from src/doc to docs
* A couple master->main renames that were MIA
Signed-off-by: Larry Gritz <lg@larrygritz.com>
OpenEXR 3.3 was just released and its main branch has bumped its
advertised version to 4.0 (the latter breaking our "bleeding edge" CI
test).
Raise our "latest releases" test to use 3.3.
Our "bleeding edge" test picking up the now-called-4.0 OpenEXR main
branch required some minor logic changes to not reject the version as
too new to be compatible.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
It's an old flavor not well supported by homebrew any longer, so what's
happening is that some of the dependency packages we ask to install have
to build from source, which means it takes a long long time.
By disabling OpenCV and Qt homebrew installation for it, we can make it
fast again. Neither the minimal OpenCV interop, nor osltoy (the only
thing that needs Qt) are exercised by the automated testsuite anyway, so
I don't think there's any loss of testing functionality in practice.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Some behavior changed for fmt error conditions, an in a way that
requires us to remove a test, because there is no reproducible output
to compare against.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
When writing a partial tile, the unused pixels still go through float
conversion.
This means, that floating point operations are done on uninitialized
data.
This can easily lead to NaN and to floating point exceptions, if those
were to be enabled.
This change will set the shared buffer used for all partial tiles to all
zero pixels.
Tested by running valgrind before and after the change
FIXES: #4461
Signed-off-by: Bram Stolk <b.stolk@gmail.com>
Adding data and display window overlay in iv.
Dashed line: Display window
Solid line: Data window
Off by default, select "Show display and data window borders" from the View menu.
---------
Signed-off-by: tkchanat <andychancse@gmail.com>
First draft of front end changes for issue #4436.
The end goal is to remove redundant ImageSpec from ImageCache internals (issue #4436).
But that necessarily changes some assumptions about what is accessible through the public
APIs for ImageCache and TextureSystem. This PR contains the API-side changes, but without
the internals overhaul that will come later.
* `get_imagespec()` and `imagespec()` lose their miplevel and native parameters. We are
now assuming now that there is no arbitrary metadata varying per-mip-level. Also, loss
of native param means that it's always copying an ImageSpec that reflects the file.
* New `get_cache_dimensions()` is a lighter-weight function that can retrieve the limited
items that really do vary between MIP levels, and may differ between the file and the
in-cache pixels: resolution, tile size, pixel data format.
* Harmonize the order of parameters in the analogous functions exposed by IC and TS
(they confusingly differed before).
* For now, API-backward-compatible wrappers for the old functions, which will eventually
be deprecated and then removed.
---------
Signed-off-by: Basile Fraboni <basile.fraboni@gmail.com>
cumentation to testsuite conversion for cpp and python for the
ImageBufAlgo Chapter, Section: Image Arithmetic up to the div function.
The PR also fixes a python testing image file reference in
..docs-examples-python/ref/out.txt, where reference images "simple.tif"
and "scanlines.tif" should have the docs-examples-cpp prefix (see
..docs-examples-python/run.py line 11 for details).
---------
Signed-off-by: Lydia Zheng <lydiamzheng@gmail.com>
This fixes OIIO's CI "bleeding edge" test that broke when OCIO pushed
to their main the new config files that will be in OCIO 2.4. (Yay!)
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This addresses:
```
==126424== Conditional jump or move depends on uninitialised value(s)
==126424== at 0x16A24E: IvGL::mouseMoveEvent(QMouseEvent*) (src/iv/ivgl.cpp:1208)
==126424== by 0x5FCD5F6: QWidget::event(QEvent*) (in /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6.4.2)
==126424== by 0x5F843AF: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6.4.2)
==126424== by 0x5F7D2C6: QApplication::notify(QObject*, QEvent*) (in /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6.4.2)
==126424== by 0x6E35447: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib/x86_64-linux-gnu/libQt6Core.so.6.4.2)
==126424== by 0x5F79D7D: QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool, bool) (in /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6.4.2)
==126424== by 0x5FDBAD6: ??? (in /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6.4.2)
==126424== by 0x5FDCE74: ??? (in /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6.4.2)
==126424== by 0x5F843AF: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6.4.2)
==126424== by 0x6E35447: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib/x86_64-linux-gnu/libQt6Core.so.6.4.2)
==126424== by 0x6762DE6: QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) (in /usr/lib/x86_64-linux-gnu/libQt6Gui.so.6.4.2)
==126424== by 0x67AD5CB: QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) (in /usr/lib/x86_64-linux-gnu/libQt6Gui.so.6.4.2)
==126424== Uninitialised value was created by a heap allocation
==126424== at 0x4846FA3: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==126424== by 0x11F509: ImageViewer::ImageViewer(bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (src/iv/imageviewer.cpp:140)
==126424== by 0x17D248: main (src/iv/ivmain.cpp:127)
```
With this patch, this particular valgrind error goes away.
Tested by running valgrind on iv before and after change.
Signed-off-by: Bram Stolk <b.stolk@gmail.com>
Fixes#4445
This avoids UB by initializing a variable.
## Tests
I ran valgrind after the fix: no longer issues error.
```
==77256==
==77256== HEAP SUMMARY:
==77256== in use at exit: 25,167,390 bytes in 8,232 blocks
==77256== total heap usage: 183,524 allocs, 175,292 frees, 4,415,783,216 bytes allocated
==77256==
==77256== LEAK SUMMARY:
==77256== definitely lost: 0 bytes in 0 blocks
==77256== indirectly lost: 0 bytes in 0 blocks
==77256== possibly lost: 0 bytes in 0 blocks
==77256== still reachable: 25,167,390 bytes in 8,232 blocks
==77256== suppressed: 0 bytes in 0 blocks
==77256== Rerun with --leak-check=full to see details of leaked memory
==77256==
==77256== For lists of detected and suppressed errors, rerun with: -s
==77256== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
```
---
Signed-off-by: Bram Stolk <b.stolk@gmail.com>
This fixes a problem seen on RHEL 9.2 Linux when launching an image in
iv, eg:
iv testsuite/texture-overscan/ref/out-exact.exr
The image displayed as a constant brown and the terminal showed:
GL error create program 1281 - Invalid value
GL error After attach vertex shader. 1281 - Invalid value
The fix is to initialize m_shader_program and m_vertex_shader in the
class constructor.
ERROR ANALYSIS:
The bug was introduced in 5fe44415 "feat(iv): OCIO color managed display
(#4031)".
Before that commit, create_shaders() initialized those members to 0
before they were used.
After that commit:
* uninitialized m_shader_program is passed to glDeleteProgram() (near
the top of create_shaders) ), causing the
"create program" warning (there isn't an print_error call after
glDeleteProgram so the error is reported as "create program").
* ununitialized m_vertex_shader is non-0 when checked by if
(!m_vertex_shader), causing the vertex shader not to be created and
causing the later call to glAttachShader() with uninitialized
m_vertex_shader to fail. That causes the second error message, and lack
of a vertex shader apparently causes the image to display as constant
brown.
Signed-off-by: David Adler <david1.pub@gmail.com>
This adds a null pointer check in the raw input plugin when the user
requested raw bayer data for a file format which doesn't have it. With
this fix OIIO shows an error instead of crashing in such case.
This fixes the issue mentioned in the comments of #4361. The title issue
of #4361 has been resolved in LibRaw 0.22.0, so we can close that issue,
I guess.
This also fixes the crash on 5 out of 6 examples in #3115. The last one
is unrelated, and I don't have a good fix for that at this time (crash
due to the raw buffer having half resolution, with the pixel aspect
ratio of 0.5). Perhaps we could create a separate issue and close#3115.
I have not added any tests for this change specifically, as it is fairly
trivial.
I have started working on a more comprehensive test suite for the raw
plugin / rawtoaces, but it's not ready yet, and I'm not sure where it
should live at this stage.
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
For old versions of fmt library, some error output was slightly
different. Address it with an additional acceptable reference
output for the oiiotool test.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Basically, this switches to creating the colorconfig that oiiotool uses
from statically constructed to build-on-demand, which should slightly
lower runtime overhead for oiiotool invocations that don't need to do
any color transformations.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Next round of making safer interfaces. Primarily, this adds new
span-baesd versions of ImageBuf::get_pixels, set_pixels, setpixel,
getpixel, interppixel, interppixel_NDC, interppixel_bucibuc,
interppixel_bicubic_NDC.
The old ones took raw pointers and implied sizes -- these still exist
but are considered "unsafe" and aimed at experts and special cases. (If
a downstream project wants to be sure not to use them, they can define
symbol "OIIO_IMAGEBUF_DEPRECATE_RAW_PTR" before including imagebuf.h to
force deprecation warnings.)
The new additions instead take a span -- a combined pointer and length
that can be bounds-checked (in debug mode). I strongly encourage people
to use the new interface, as it forces the caller to think hard about
the exact region of memory that's ok to read from or write to, and then
for the OIIO side to honor that and enfoce/debug with bounds checks
against that if we add such plumbing.
Along the way, I added some interesting helper functions:
In span.h:
* `span_within()` checks that one byte span lies entirely within
another.
* `check_span()` checks if a pointer and length lies entirely within a
span.
* `OIIO_ALLOCA_SPAN` is a macro much like OIIO_ALLOCA, except what it
returns is a span rather than a raw pointer to the stack-allocated
region.
In imageio.h:
* `span_from_buffer()`/`cspan_from_bufffer()` : Given a raw pointer,
pixel data type (TypeDesc), and image dimensions and strides, returns a
byte stride describing the extent of memory that encompasses all pixels
laid out by that description.
I hope the switch to spans is not very onerous for people. I really
think that this will help us write more solid and deliberate code to
begin with, and when things do go wrong, the various debug-mode checks
and assertions that spans provide should help us more quickly identify
exactly where buffer overruns and the like occurred.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Right now, the only option is gperftools, because that's what I happened
to be trying. But this can be expanded later.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Oops.
I didn't really have time to add an "example" right now. I hope to
return to beef this up (or that somebody else takes it on).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This test was running very slowly, taking too long in all circumstances,
but shockly causing GHA CI Mac runners to hit the test timeout
sometimes.
It turns out we were doing a lot of redundant work. There was no need to
make separate pattern and bayer images for every data type. Instead,
hoist the test image generation out of the loop and just make one
(float), then use `-i:type=<type>` to read it into an ImageBuf of the
appropriate data type.
This cuts the time for this test to run by about 3x.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We no longer supply a FindOpenColorIO.cmake, so it makes no difference.
But it was getting in the way of a weird case where a wrapping project
wanted to supply one.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
With 3.0 approaching, it seems like the only time to make these changes
that I made note of a while back:
* It seems to me that `tell()` should be const because it isn't going to
modify or change the state of the IOProxy.
* It seems to me that `flush()` should NOT be const, because it probably
does change internal state.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
When I added perpixel_op recently, I didn't get the function signature
right. It only would work for plain old function pointers, and would not
work properly for "functors" or lambdas with capture. I should have used
function_view which can do a light-weight wrapper of all three, so now I
do. Also modified imagebufalgo_test to explicitly test all three forms
to be sure they work.
Also update function_view ever so slightly.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes the bug where IBA::demosaic only works with float images.
Fixes#4416
I've extended the unit tests to run on float, half, uint16, uint8
images.
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
openexr_copy and openexr-decreasingy
These recently added test were writing into the ref/ directory, which
they certainly should not do. Write into the build test area, compare to
what's in the ref area.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Convert C++ and Python examples from the "Image transformations and data
movement" section of the "imagebufalgo" chapter into tests within the
"docs-examples" testsuites (related to issue #3992).
Update 'oiiotool' tabs to use a simple 'tab' instead of 'code-tab' to
prevent synchronized tab selection, which is inconsistent with the other
tabs using the 'literalinclude' directive.
---------
Signed-off-by: Jeremy Retailleau <jeremy.retailleau@gmail.com>
This allows somebody who is enhancing the docs to check the results
entirely through CI without having to set up the dependencies necessary
for docs on their end. (Perfect for Dev Days.) It also ensures that any
changes we make to the code base don't break the documentation somehow,
because now a successful docs build will be part of the CI.
Note that the docs building job leaves behind an artifact, which will be
a zip file of the directly containing the HTML docs, just like one would
find on RTD, so you can download that zip file and view the docs it made
to verify that everything looks right.
I made the docs builder a separate workflow, but I also changed some
things in the CI workflow for the sake of making them more similar.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The flavor of ImageBuf that wraps caller-owned memory is constructed or
reset by merely passing a raw pointer and it assumes that the range of
memory implied by the strides is entirely safe to read from or write to.
And even if correct, there is not much within ImageBuf internals to
prevent bugs from buffer overruns.
This PR introduces a new constructor and reset() method to ImageBuf
where instead of taking a raw pointer, it takes a span describing the
extent of the safely addressible memory (and also an optional "pointer
to first pixel" within the buffer, which is necessary if you're using
negative strides or for some other reason are describing safe memory
that starts before the location of pixel 0). That span is stored in the
ImageBuf, and also it remembers if it was a cspan (which should indicate
a read-only IB) or a non-const span (which should wrap writable memory).
At this moment, I'm just adding this API and will start using and
documenting it as the perferred method, with the raw pointer versions
being regarded as unsafe for now (and maybe some day will be
deprecated?). But we're not yet using the span internally for safety
checks.
In the future, my intent is to gradually add safeguards to do bounds
checking against the span for IB operations that touch the wrapped
memory. Because we'll most likely bounds check only in debug mode, it
shouldn't ever impact performance, but it will make debugging,
sanitizer, and fuzz tests more robust to catch and zero in on buffer
overruns. I also like how it forces the caller, at the call site, to
think hard about exactly what range of memory the IB is wrapping, and
not merely pass a single pointer and hope it works out.
I also changed all ImageBuf ctrs and resets within the OIIO codebase to
use the new span-based versions.
This is part of an overall trend to migrate from raw pointer based API
calls to span-based API design as much as possible.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
span_util.h is relatively new. I'm honestly no longer sure why I thought
it was necessary to make a second file, but so little is in it that I
now believe its contents should be folded into span.h to keep all the
span-related definitions in once place. It's just not big enough to need
to be divided into several headers.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
When fmt arguments don't match the format string, fmt will throw an
exception, or terminate if we disable exceptions (which we do). For
gcc11+, we tried intercepting these, but let's do it for all platforms,
and let's not terminate ourselves, but insted print and log the error.
But, oof, to do this properly, I needed to move some error recording
functionality from libOpenImageIO to libOpenImageIO_Util and make it
owned more properly by strutil.cpp, so that this all works even when
only using the util library. The logic isn't changing, it's just moving
over to the other library.
This all helps to address #4388
Unfortunately, the exception thrown by fmt doesn't tell us the bad
format string itself. That would have really allowed to probably zero
right in on it. But at least we know it's occurring, and one could put a
breakpoint on pvt::log_fmt_error to catch it in the act and see where
it's being called, revealing the bad line.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
- ColorProcCacheKey: Update definition, signature, and usage to support
caching NamedTransform processor handles.
(Note: **ABI-breaking**)
- ColorConfig: add NamedTransform convenience functions:
`getNumNamedTransforms`, `getNamedTransformNameByIndex`,
`getNamedTransformNames`, `getNamedTransformAliases`
- ColorConfig: add `createNamedTransform` function returning new
ColorProcessorHandle
- Add `ImageBufAlgo::ocionamedtransform` functions
- oiiotool: add `--ocionamedtransform` operator
- python: add NamedTransform convenience function bindings
- python: add `ImageBufAlgo.ocionamedtransform(...)` bindings
- Update `oiiotool` and `python` documentation with examples
## Description
OCIO-2 config authors may define a set of stand-alone "source-agnostic"
`NamedTransforms`. Unlike the `colorconvert`, `ociodisplay`, and
`ociolook` IBAs, which explicitly convert _from_ a color space to
something else, the `ocionamedtransform` IBA behaves more like the
`ociofiletransform` IBA, where the transform itself can be applied
either in the forwards or inverse direction, and does not take into
consideration the input encoding or image state.
Quoting from the OCIO [NamedTransform
documentation](https://opencolorio.readthedocs.io/en/latest/guides/authoring/authoring.html#named-transforms):
> Sometimes it is helpful to include one or more transforms in a config
that are essentially stand-alone transforms that do not have a fixed
relationship to a reference space or a process space. An example would
be a “utility curve” transform where the intent is to simply apply a
LUT1D without any conversion to a reference space. In these cases, a
named_transforms section may be added to the config with one or more
named transforms. [...] This feature may be used to emulate older
methods of color management that ignored the RGB primaries and simply
applied one-dimensional transformations.
Notably, the built-in OCIO-2 Studio config
(`ocio://studio-config-latest`) contains NamedTransforms "sRGB - Curve"
and "Rec709 - Curve", which we could implement as drop-in replacements
for the existing `linear_to_sRGB`, `sRGB_to_linear`, `linear_to_Rec709`,
and `Rec709_to_linear` functions in `color.h` (unless those functions
are deprecated...).
There are several other applications for NamedTransforms as well. They
can be extremely useful as workflow-oriented transforms implemented at
various points in facility pipelines (e.g., for applying shot-specific
conversions, grades, gamut-compression, etc.); they can be used to
provide diagnostic transforms for visualizing "exposure zones", clipped
areas, NaNs, etc; for visually "tagging" problematic clips; for applying
parts of ACES Metadata Files parsed by OCIO's forthcoming AMF parser;
for conversion to non-RGB color models... all sorts of things.
## Tests
Tests are forthcoming. I've tested everything locally, but have not yet
implemented proper tests in the testsuite.
---------
Signed-off-by: Zach Lewis <zachcanbereached@gmail.com>
Background: The ImageCache and TextureSystem had historically been
implemented as pure virtual abstract base classe defining an interface,
then a hidden concrete subclass. The flexibility this was intended to
provide was never really used; instead the imagined main client for the
flexibility, OSL, has a different mechanism for customizing the texture
system (via their RendererServices class). Meanwhile, this has made the
interfaces to IC and TS impossible to change without breaking the ABI.
So this PR does the following:
* ImageCache and TextureSytem are fullly de-virtualized, and are now
concrete classes, no hidden subclassing, no virtual methods. As such,
removing or changing existing methods will be an ABI break, but adding
new methods is not.
* Each now use a PIMPL idiom, where all the data and internal methods
are hidden from the public interface and do not affect the ABI.
* Roughly speaking, we added the PIMPL pointers and made the
ImageCacheImpl/TextureSystemImpl -- what used to be the hidden concrete
subclasses -- into the hidden PIMPL classes.
* Moved a lot of classes that used to be in a "pvt" namespace into the
main namespace. If they are opaque types not exposed in public headers
or as symbols exported from the library, it doesn't matter, so this just
removes some arbitrary clutter.
* Had to rearrange some of the recently added heapsize/footprint code,
moved some inline functions in the headers into methods of the classes,
defined in the cpp files.
This itself is a huge ABI change, so will only be merged into master, to
become part of the 3.0 release. This is not an API change, though! This
is all about the internals, and should not require any client software
to change a single line of code.
I may do further simplification or refactoring of this code in the
future, but that will all be smaller and have no further API/ABI changes
that break compatibility.
Although the structure of the class hierarchy has changed around, none
of the logic about how IC and TS actually *work* has changed, so there
should be no change in observable behavior.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Added as_bytes and as_writable_bytes that convert spans of arbitrary
type to spans of std::byte. (This mimics C++20 utilities of the same
name.)
* spancopy, spanset, spanzero: add parameter defaults for the common
case of wanting to do the operation on the whole span, also simplify the
bound-limiting logic in those functions a bit (but still performing
precisely the same task).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Because IBA::transpose changes resolution, it doesn't pass the input
image to IBAPrep(), and there was a special case there that always set
destination to float if there were no source images.
So change transpose to specifically request a format, to be the same as
the input image.
And that required a tweak to IBAPrep allowing us to specify such a thing
(there was already an option that meant "force it to float", but not the
general case of an arbitrary type).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This change adds additional FFmpeg metadata from the video and data
streams. These streams can hold the start timecode which is very useful
in OpenTimelineIO applications.
I also added `av_guess_frame_rate()` which might be more robust than
just checking `stream->avg_frame_rate`, but I can also create a separate
PR if it is better to break it up.
It doesn't look like there are any FFmpeg tests in `testsuite`?
I tested with the DPEL ALab trailer and some Netflix Open Content
movies, and was able to see the timecode metadata in the OIIO image
attributes.
---------
Signed-off-by: Darby Johnston <darbyjohnston@yahoo.com>
Added IBA::demosaic().
Currently supports 4 Bayer pattern layouts, "RGGB", "GRBG", "GBRG", "BGGR", and 2 algorithms: "linear", "MHC".
Should be expandable in the future.
---------
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
In trying to avoid doing the color conversion pointlessly for pixels
with alpha=0, I accidentally also managed to skip the multiplication by
alpha for those pixels. Oops.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Don't read or store full metadata on every MIP level for every face.
That can really add up! Ptex can only have one set of metadata, the
format itself doesn't allow it to vary from face to face.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
It was disabled to make prettier formatting for getargs, but neglected
to turn it back on for the rest of the file.
I also changed a few old `<<` stream output to `print()`.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Nominally, fix four places where "it's" was used when "its" was
appropriate. Realistically, force the EasyCLA dialog so that I can get
an individual CLA in place, my employment with ARRI having terminated on
15 July of this year.
Signed-off-by: Joseph Goldstone <joseph.goldstone@mac.com>
Change IC::create() and TS::create() to return shared_ptr instead of a
raw pointer. This cleans up a lot of lingering lifetime management issue
of these classes. The switch to 3.0 is an opportunity to make breaking
changes to these APIs.
For the sake of downstream users, we define preprocessor symbols
OIIO_IMAGECACHE_CREATE_SHARED and OIIO_TEXTURESYSTEM_CREATE_SHARED to
signal that the new APIs are present. Some very minor changes may be
needed at the call site.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fully linking OpenCV against libOpenImageIO is burdensome for people
downstream, especially those who want a static version of
libOpenImageIO.
There were only three IBA functions that use OpenCV: to_OpenCV(),
from_OpenCV(), and capture_image(), and all three are fairly short. Move
them to a separate header, imagebufalgo_opencv.h, make them fully INLINE
(so they don't actually produce object code in libOpenImageIO, which
therefore does not need to link against OpenCV).
Applications wanting these three functions should include this header
and will be responsible themselves for ensuring that the include paths
and linkage of their application makes provisions for finding and using
OpenCV. All other applications don't need to deal with OpenCV dependency
at all.
Our own oiiotool (for --capture) and our python bindings still link
against OpenCV to support this functionality (again, still as an
optional dependency, enabled only if OpenCV is found at build time). But
downstream apps using libOpenImageIO and who do not themselves need our
OpenCV functioality now no longer need to link against OpenCV.
We have removed capture_image() from the Python bindings -- Python
scripts that need to capture live camera images can use OpenCV or any
other capture API of their choice without going through OIIO.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Microsoft's runtime library doesn't support the standard setenv()
function. _putenv_s() should be used instead.
Without fix iv do not compiling:
`E:\GH\OpenImageIO\src\iv\ivmain.cpp(119,9): error C3861: 'setenv':
identifier not found [E:\GH\OpenImageIO\build\src\iv\iv.vcxproj]`
---------
Signed-off-by: Vlad (Kuzmin) Erium <libalias@gmail.com>
This version dates from early 2021 (so still supporting back 3.5 years).
The reason for the bump is to simplify a bit on our part and avoid older
versions known to be broken or lacking certain features.
This is for master/2.6/3.0 only, will not be backported to 2.5, since we
never raise minimum dependencies in already-released branches.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
CI sanitizers in master have been failing lately. A different PR
addresses the root cause, but this patch contains some improvements I
made along the way while investigating it:
* CMake: Change the SANIIZE build variables to use set_cache, thus
allowing easier override by env variable.
* Bumped the CI sanitizer test to use the 2024 container and latest
compilers.
* Suppress UB warnings in STL function.h, where it's claiming undefined
behavior in the std::function guts called by ArgParse internals. Only
seen failing on my Mac, wasn't actually failing in the GHA CI. I'm
pretty sure it's a false positive.
* Also, only on my Mac building against OpenVDB 11, it found some UB in
OpenVDB, which I'm suppressing (I think it's also false positive).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fix a bug in oiiotool wherein `-i:ch=...` (input only a subset of
channels when reading from a file) was not adjusting the spec's
alpha_channel and z_channel -- that is, they still contained the OLD
channel numbers of those things, which not only could be wrong due to
the reordering, but they might be out of range of the new number of
channels.
Also a related fix in the TIFF reader, to fix a possible dereference of
an empty vector. This was being triggered by that out-of-range
alpha_channel. Maybe that can't happen anymore with the above bug fix,
but it still feels like the safer way to proceed.
This was all found by the sanitizers in CI -- and only recently, despite
this code having this bug for a long time. So yay for sanitizer CI!
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Make OCIO a hard dependency, at least version 2.2.
Revise and improve the OCIO "auto-build" scripts.
Remove OCIO 1.x testsuite reference output.
No longer need FindOpenColorIO.cmake module
As part of this, I split the CI "oldest/hobbled" test into two different
tests:
* oldest -- test against the oldest supported versions of all
dependencies.
* hobbled -- test against most optional dependencies missing and
optional features disabled.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
First PR of two, following @lgritz and I discussions on memory tracking
in the OIIO::ImageCache.
- Add two template methods and their specializations for types various
to help memory tracking:
```c++
// return the total heap allocated memory held by the object and its members
template<typename T> inline size_t heapsize(const T& t);
// return the total memory footprint including the size of the structure itself
template<typename T> inline size_t footprint(const T& t);
```
- Specialized for: ParamValue, ParamValueList, ImageSpec, ImageInput,
ImageOutput, std::vector, std::shared_ptr, std::unique_ptr,
oiio::intrusive_ptr, ImageCacheImpl related objects.
- New files:
- `include/memory.h` : adds base and few types specialization of
heapsize and footprint.
- `libtexture/imagecache_memory_pvt.h` : adds specilializations for
ImageCacheImpl and related objects.
- `libtexture/imagecache_memory_print.h` : adds a helper function that
print a breakdown of the ImageCacheImpl total memory usage as well, as
well as per image format. Note: this is slow, but gives accurate memory
estimation.
Related PR from Larry :
https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/4317
Signed-off-by: Basile Fraboni <basile.fraboni@gmail.com>
The main purpose of this change is to make it possible to use OIIO for
reading raw files in rawtoaces instead of calling LibRaw directly. There
are the changes:
- add the missing hints needed to implement all combinations of
white-balancing methods and matrix methods provided by rawtoaces.
- add the DNG-specific attributes
The change adds this functionality:
- new "raw:user_black" hint to override the default black point
- new "raw:use_auto_wb" hint to force LibRaw to white balance by
averaging over the whole image.
- new "raw:grey_box" hint to make LibRaw to white balance by averaging
over the given rectange.
- new "raw:dng:XXX" attributes added to the output ImageBuf if the input
image is a DNG file. The attributes consist of 2 sets of [calibration
illuminant; calibration matrix, XYZ to camera RGB matrix]. Note, the
current DNG standard supports up to 3 calibration illuminants, but both
LibRaw and rawtoaces only use 2 currently.
I have manually tested all permutations of white-balancing modes and
matrix methods which are currently supported by raw to aces. The images
match up to a rounding error.
The current unit tests pass, but they only seem to use the default
conversion settings. We may want to extend those. I'm not clear on how
to do that, there are multiple reference images for different versions
of LibRaw, not sure if I will have to re-generate all of them. I intend
to make more changes to the raw plugin soon, may come back to updating
tests during/after that.
There are currently no tests using DNG files, so the new DNG-specific
attributes are not covered. The code relying on those in rawtoaces works
fine.
I have also updated the documentation to add the new hints, however, I
haven't been able to build the documentation.
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
JPEG output configuration hint "jpeg:iptc" (default: 1), if set to 0,
will suppress IPTC block output to the file.
In the process, we changed the return type of utility function
encode_iptc_iim() to return true if anything was successfully encoded,
false otherwise.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
It's not just in oiiotool. This seems clearer and adheres to the env
variable naming convention we chose.
Reminder: This controls whether command line history gets written to
output image metadata by default by oiiotool and maketx. We historically
did it, but recently stopped because of security concerns.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Additional stack manipulation commands:
* `--popbottom` discards the bottom-of-stack image
* `--stackreverse` reverses the order of the whole stack
* `--stackclear` fully empties the stack
* `--stackextract <index>` moves the indexed item from the stack (index
0 means the top) to the top.
Make `--for` work correctly in both directions:
* Correct behavior if `--for` has a negative step value.
* If the end value is less than the begin value and no step is supplied,
assume -1 (analogous to how we usually assueme step=1 under ordinary
circumstances).
* Error if step is 0 (presume it will make an infinite loop).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fixes a simple copy/paste error in a copy constructor where the y
coordinate gets initialised twice instead of y and z.
Signed-off-by: Anton Dukhovnikov <antond@wetafx.co.nz>
* Get rid of some obsolete cmake code.
* Movement (but no change) to some parts of CMakeLists.txt, primarily to
make it closer to the corresponding file in OSL to make it easy for me
to diff them and port innovations back and forth between them.
* Some typo/etc fixes
* Remove unused OIIO_UNUSED_OK macro that's been deprecated since 2.0,
and OIIO_CONSTEXPR and OIIO_CONSTEXPR14, neither of which have been
needed for years.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Even that is quite old -- released in 2012! But it's the first version
that guaranteed thread safety for error reporting. So we're finally
making that our minimum and taking out the locking we did on our side.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Raised from 1.5.13 (maybe? we never carefully checked) to 1.6.0 minimum,
which was released in 2013 (so absolutely nobody should be bothered by
this).
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This new version dates to 2020. Previously, we had supported
(purportedly) WebP 0.6.1, but we didn't test that configuration. By
bumping to 1.1 minimum, we can use WebP's own exported cmake config,
rather than maintaining a FindWebP.cmake mmodule on our side.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Improvements to expression evaluation of oiiotool command line arguments
surrounded by `{braces}`.
* Just like `TOP` refers to the top image on the stack, now `BOTTOM`
refers to the bottom image.
* Referring to images with `IMG[id]` previously worked if `id` was a
literal integer or an image label or an image filepath. But now it can
be any expression that evaluates to any of those things. So, for
example, you could say `{IMG[i].filename}` to get the filename of the
i-th image down the stack (where `i` is a variable), or
`{IMG[NIMAGES-2].width}` to get the x resolution of the 2nd-from-bottom
image on the stack.
* The `--label` and `--set` commands check that the name you use for an
image label or user variable follow the "C identifier" lexical rules:
must be alphanumeric + underscore, but not start with number. It was
previously possibly to do completely chaotic things like `--set 1 2`.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* The old versions (deprecated since 2.0) of IBA::compare() and
computePixelStats() that took a reference to the CompareResults or
PixelStats structure now have deprecation warnings. Use the versions
that return the structure instead.
* The deprecated (often since as far back as 2.0) versions of functions
that took raw pointers to color values now have deprecations warnings.
Use the versions that take `span<>` or `cspan<>` instead. These include
versions of: isConstantColor, isConstantChannel, isMonochrome,
isConstantChannel, colorconvert, fill, checker, add, sub, absdiff, mul,
div, mad, pow, channel_sum, channels, clamp, color_count,
color_range_check, render_text.
* The `histogram()` function that takes a reference to a result vector
(deprecated since 2.0 and previously warned) has been removed. Use the
version that returns the vector instead. The `histogram_draw()`
(deprecated since 2.0 and previously warned) has been removed and has no
replacement since it was always silly.
* The versions of ociodisplay that lacked an `inverse` parameter, which
were marked as deprecated since 2.5, now have deprecation warnings. Use
the version that takes an `inverse` bool.
* The OpenCV-related functions that take old-style IplImage pointers
(deprecated since 2.0) have been removed. Use the modern ones that use
cv::Mat.
* The pre-KWArgs versions of resize, warp, and fit now have deprecation
warnings. Use the versions that take KWArgs instead.
* IBA::type_merge, deprecated since 2.3, now has a deprecation warning.
Instead, use TypeDesc::basetype_merge().
All of the IBA functions with deprecation warnings are additionally
implemented inline, so that they can be easily removed later if
necessary without an ABI break at that time.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Reduces the memory footprint of the `LevelInfo` struct. Three changes
are here, in order of most to least impactful:
- Makes one `ImageSpec` optional, most of the time `spec` and
`nativespec` are the same, it appears they are only different when
autotile is enabled, this saves about 160 bytes.
- Uses `std::unique_ptr<float[]>` (8 bytes) instead of
`std::vector<float>` (24 bytes).
- Reorders members to remove excess padding.
We noticed this memory usage on scenes with per-face textures with
multiple mips per face. For a scene with 12.5 million mips, this saves
about 2GB.
Note, it would probably also be possible to remove the
`polecolorcomputed` member, instead relying on the `polecolor` pointer.
But since this bool occupies existing padding in the struct it wouldn't
save any memory currently.
---------
Signed-off-by: Curtis Black <curtis.w.black@gmail.com>
* Add deprecation warnings to the varieties of ImageBuf constructor and
`reset()` that don't take subimage and miplevel parameters. The
equivalent is to just pass `0` for both of those parameters. The warned
constructors have been marked as deprecated since OIIO 2.2.
* The misspelled `ImageBuf::make_writeable()` has been given deprecation
warnings. Since OIIO 2.2, the correct spelling is `make_writable`.
* The `ImageBuf::error()` method that uses printf-style formatting now
has deprecation warnings. Use `ImageBuf::errorfmt()` instead.
* The `ImageBuf::interppixel_NDC_full()` method, which has been marked
as deprecated since OIIO 1.5, now has deprecation warnings. Use
`interppixel_NDC()` instead.
Undeprecate the name+spec varieties of constructors. They were
documented as deprecated but warnings were never added. In doing this
set of changes, I realized that they were used for something useful
after all -- it makes it easy to remember the name of a new buffer
you're making not from a file, so you can later `write()` it without
needing to know the filename at that time. I also added a new
`set_name()` method to do it directly for times when you don't want to
rely on the constructor or want to change the name.
By the way, I'm being careful also to convert all the warned or marked
deprecated functions to inline, if at all possible, so that in the
future removing them will not be an ABI break.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
It's silly to copy the full metadata from the source (which could
include big things like ICC profiles) to EVERY level of a MIPmap.
This mostly applies to TIFF textures, since OpenEXR MIPmaps don't store
attributes separately for each MIP level anyway.
This can be disabled (that is, make it copy full metadata to all levels
of the MIPmap) by setting a new make_texture configuration hint,
"maketx:mipmap_metadata" to nonzero.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
With 3.0 coming soon, it's time to break compatibility by switching
OIIO::span::size() to return an unsigned value, matching C++20
std::span.
The history is that when we first added OIIO::span, at that time, the
C++ draft proposal for std::span included a signed size() result, but by
the time it got finalized into C++20, it was changed to unsigned to
match all the other std containers, but it was too late for us to switch
without breaking API compatibility rather seriously.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Remove long deprecated and empty header missing_math.h
* paramlist.h: Deprecate the old form of ParamValue constructors.
People should be using the versions with strongly typed params, not
raw bools.
* parallel.h: Remove deprecated parallel_for variants.
* strutil.h: Add deprecation warnings to old-style format.
* sysutil.h: Deprecate Sysutil::physical_concurrency.
Also, I noticed that all the external declarations in this header were
incorrectly OIIO_API but should have been OIIO_UTIL_API to reflect that
they are in the OpenImageIO_Util library.
* thread.h: mark deprecated yield() and this_thread_is_in_pool().
* tiffutils.h: removed deprecated redundant functions.
* ustring.h: Remove deprecated ustringHash.
* type_traits.h: Remove deprecated OIIO::void_t, use std::void_t instead.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Old version of TextureOptions we haven't used since OIIO 1.x.
* Texture methods that were based on the VaryingRef helper. These are
actually remnants of early OSL needs when it was more Reyes-like, BEFORE
we used LLVM!
* Get rid of the varyingref.h header, no longer needed by anything.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* array_view.h: Remove this long-deprecated header.
* benchmark.h: Attach DEPRECATED warning to external use of
benchmark.h's time_trial().
* bit.h: Remove bit_cast that's had deprecation warnings for several
releases, and add deprecation warnings to the old rotl32 and rotl64
* color.h: add deprecation warnings and make the deprecated things
inline so they can be removed later without an ABI break.
* errorhandler.h: remove deprecated old printf style methods
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Initial Exif and XML metadata support for JPEG XL.
- EXIF - works and enabled by default
- XML - works and enabled by default
- IPTC - that part need some help. JPEG XL support XML based IPTC, but Ido not get what IPTC create an OIIO
- JBIG - not sure
new options:
- `jpegxl:use_boxes` - export EXIF, XMP, IPTC, JBIG
- `jpegxl:compress_boxes` - compress meta boxes (disabled for this moment by default)
- jpegxl:exif_box = 1 (enabled)
- jpegxl:xmp_box = 1 (enabled)
- jpegxl:jumb_box = 0 (disabled)
- jpegxl:iptc_box = 0 (disabled)
No tests yet, but if we have I can add.
---------
Signed-off-by: Vlad (Kuzmin) Erium <libalias@gmail.com>
The old name, OIIO_LIBRARY_PATH, was inconsistently named in two ways:
First, it was not clear that it was specific to searching for plugins,
and since it is related to the to the global attribute
"plugin_searchpath", I thought making it mention plugins specifically
would be more clear.
Second, I noticed that all of the publicly documented environment
variables that are "for users" started with OPENIMAGEIO_, except for
this one. There are a bunch of environment variables we check that begin
with OIIO_, but all but this one are intended for developers when
testing or debugging.
So I'm using a new name that is consistent with both principles.
Henceforth, the naming rule about environment variables is:
* `OPENIMAGEIO_FOO` for public interfaces, are fully documented and
intended for users/sites to use to customize behavior.
* `OIIO_FOO` for developers to use when testing and debugging, are
undocumented (except in the code itself), and are not considered part of
the public API/behavior of OpenImageIO -- their meaning may change, they
may disappear without warning, or they may only be functional for debug
builds.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I should not have hidden virtual functions behind
OIIO_DISABLE_DEPRECATED checks. Never make the vtbl possibly vary for
clients of the library from the way we built it!
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Enable the ImageCache for the `iv` tool. It more or less expects it to
be present and certain features, like loading the Image's mip-levels,
requires the ImageCache to be used.
This was breaking CI for us. Hard break for icc/icx tests, since those
REQUIRED a yum install of the intel compilers. Softer undetected break
for all the ASWF <= 2022 containers based on CentOS 7, which were not
failing outright but were failing to install certain optional packages.
The solution (for now) is to configure yum to exclude the missing repo.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
For the last year or so, I have been using git-cliff to generate the
first draft of release notes. It gets me 90% of the way there for the
monthly patch releases, perhaps more like 30% (but a very good starting
point) for the big annual major releases.
In this PR, I document my process and check in the toml file that
provides the git cliff configuration that gets mostly what we need.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Inspired by a question by Vlad Erium, I have added a simpler way for C++
users of OIIO to construct IBA-like functions for simple unary and
binary operations on ImageBufs where each pixel is independent and based
only on the corresponding pixel of the input(s).
The user only needs to supply the contents of the inner loop, i.e. just
doing one pixel's work, and only needs to work for float values. All
format conversion, sizing and allocation of the destination buffer,
looping over pixels, and multithreading is automatic.
If the actual buffers in question are not float-based, conversions will
happen automatically, at about a 2x slowdown compared to everything
being in float all along, which seems reasonable for the extreme
simplicity, especially for use cases where the buffers are fairly likely
to be float anyway.
What you pass is a function or lambda that takes spans for the output
and input pixel values. Here's an example that adds two images channel
by channel, producing a sum image:
// Assume ImageBuf A, B are the inputs, ImageBuf R is the output
R = ImageBufAlgo::perpixel_op(A, B,
[](span<float> r, cspan<float> a, cspan<float> b) {
for (size_t c = 0, nc = size_t(r.size()); c < nc; ++c)
r[c] = a[c] + b[c];
return true;
});
This is exactly equivalent to calling
R = ImageBufAlgo::add(A, B);
and for float IB's, it's just as fast.
To make the not-float case fast and not require the DISPATCH macro
magic, I needed to change the ImageBuf::Iterator just a bit to add
store() and load() method templates to the iterators, and add a field
that holds the buffer type. That might make a slight ABI tweak, so I am
thinking that I will make this for the upcoming OIIO 3.0, and not
backport to the release branch.
I think this is ready to introduce at this time, but I'm also studying
whether more varieties of this approach are needed, whether the
non-float case can be sped up even more, and whether some of the
existing IBA functions should switch to using this internally (good
candidates would be those that are almost always performed on float
buffers, but for which the heavy template expansion of the DISPATCH
approach to handling the full type zoo currently makes them very bloated
and expensive to compile, for very little real-world gain).
We should probably consider this to be experimental for a little while,
just in case the function signature for this changes as I think about it
more or add functionality.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
In master, since we have raised the minimum Imath/OpenEXR to 3.x, no
more 2.x, we no longer need generate our Imath.h and half.h from
templates, they can be turned into regular header files.
Additionally, define IMATH_HALF_NO_LOOKUP_TABLE in half.h before
including Imath/half.h, which ensures that the Imath code we include
converts float <-> half using actual inline code and not the lookup
table. This can reduce or sometimes eliminate the need to link agains
Imath only for the sake of accessing that lookup table.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Track down and fix many places internal to OIIO where we were still
using deprecated API calls.
Add deprecation warnings to some things that have been nominally
deprecated, but weren't already tagged as such.
Guard deprecated things behind OIIO_DISABLE_DEPRECATED so that
downstream projects can hide them to help them find and change any use
of the deprecated calls, and behind !defined(OIIO_INTERNAL) so they are
hidden from the rest of the internals of our code base.
Make some of the deprecated things inline, so that their final removal
later doesn't constitute a link ABI compatibility change. (We do absorb
an ABI compatibility boundary with this PR, so I've bumped the version
number appropriately.) Move all the OIIO-namespace level deprecated
items to the end of imageio.h so they're as out of the way as possible.
A lot of the things that are still present here, just guarded by the
deprecation warnings, will be removed entirely before the final OIIO 3.0
release.
Please note some particular things in imageio that have been deprecated
for a while and that we expect to disappear entirely by 3.0:
* The old style `ImageInput::read_*` functions that don't take explicit
subimage and miplevel paramters, because they aren't thread-safe due to
their implied statefulness. You are expected to use the ones that take
those parameters and are stateless.
* The variety of seek_subimage that takes an `ImageSpec&` to fill in.
* Some obsolete varieties of ImageInput and ImageOutput create() static
methods.
* The `destroy` methods that are no longer needed (plain old `delete` is
fine).
* The various error-setting functions that use the old printf-like
formatting conventions. Use `errorfmt()` and friends that use the
std::format conventions.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
TIFF export with EXIF metadata (e.g. oiio::ColorSpace) creates invalid
TIFF files in the presence of an I/O proxy ("TIFF directory is missing
required "ImageLength" field"). The reasons is that TIFFSetDirectory()
requires to read some of the previously written data, but neither
writer_readproc() nor the available proxy implementations support that.
Note that TIFF output does not yet use this new scheme for I/O proxy
that was used for many other plugins.
I think writer_readproc() should not take any shortcuts and just forward
the call to the I/O proxy.
Is there a reason why IOFile (in write mode) and IOVecOutput do not
allow to re-read written data? This looks like an artificial restriction
of the underlying resources. An alternative not modifiying these two
implementations would be to let the TIFF output use a variant of
IOVecOutput that allows re-reading and forward all data at the very end
to the passed-in proxy.
Is there some policy to add assertions in all methods that are not
supposed to be called? This would have helped here a lot to reduce the
test case.
I did not add an explicit test. I guess the proper way would be to
convert TIFF ouput to use the same I/O proxy scheme that is used by many
other plugins. Small repro case is attached.
[repro.cpp.txt](https://github.com/user-attachments/files/15886829/repro.cpp.txt)
---------
Signed-off-by: Joachim Reichel <43646584+jreichel-nvidia@users.noreply.github.com>
Like the existing Vector2i, but, you know, 3.
Along the way, fixed a bug in the way the Vector2i type is represented
as a string.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fully remove some things that have had deprecation warnings for a long,
long time (mostly since 1.x), so we think it can't possibly break
anyone.
Add deprecation warnings to some things that have been nominally
deprecated for some time. The build warnings will serve as final notice
that they will fully disappear by OIIO 3.0 release.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This continues the adventure from #4082 where the processing of fmt
headers, both locally built and externally referenced, causes large
rebuilds to occur when making small or even zero changes between builds.
Since that previous PR is stuck I'll try again here.
This makes two sets of changes:
- Check if our generated files exist before writing to them (in the
externally referenced case)
- Use `copy_if_different` to prevent redundant copies from occuring, and
triggering timestamp updates (in the local build case)
Note that while this seems to fix the problem, the build is still not
"clean" wrt fmt processing. The following will be be seen when
triggering a build, with zero changes since the last build, within VS:
```
1>------ Build started: Project: ZERO_CHECK, Configuration: Debug x64 ------
1>1>Checking Build System
2>------ Build started: Project: fmt_internal_target, Configuration: Debug x64 ------
3>------ Build started: Project: CopyFiles, Configuration: Debug x64 ------
2>Generating ../../include/OpenImageIO/detail/fmt
2>Building Custom Rule C:/Users/jesse/source/oiio/src/include/CMakeLists.txt
2>Generating ../../include/OpenImageIO/detail/fmt/core.h, ../../include/OpenImageIO/detail/fmt/format-inl.h, ../../include/OpenImageIO/detail/fmt/format.h, ../../include/OpenImageIO/detail/fmt/ostream.h, ../../include/OpenImageIO/detail/fmt/printf.h, ../../include/OpenImageIO/detail/fmt/std.h, ../../include/OpenImageIO/detail/fmt/chrono.h
2>C:\Program Files\Microsoft Visual Studio\2022\Preview\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): warning MSB8065: Custom build for item "C:\Users\jesse\source\oiio\build\CMakeFiles\b19a1b09c2e7eaa53ca90801840dfdd0\fmt.rule" succeeded, but specified output "c:\users\jesse\source\oiio\build\include\openimageio\detail\fmt" has not been created. This may cause incremental build to work incorrectly.
2>C:\Program Files\Microsoft Visual Studio\2022\Preview\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): warning MSB8064: Custom build for item "C:\Users\jesse\source\oiio\build\CMakeFiles\b253238471e54ff57d14ae397110f40e\core.h.rule" succeeded, but specified dependency "c:\users\jesse\source\oiio\build\include\openimageio\detail\fmt" does not exist. This may cause incremental build to work incorrectly.
2>Done building project "fmt_internal_target.vcxproj".
4>------ Build started: Project: OpenImageIO_Util, Configuration: Debug x64 ------
```
Whatever it is, the VS generator doesn't much like the way we're
handling the processing but I don't know enough cmake-fu to ascertain
why that might be.
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
TL;DR: when turning unassociated alpha into associated for
gamma-corrected PNG images, we got the exponent wrong for the
linearization step, and when doing the same for sRGB images, we didn't
linearize at all.
WARNING: will change appearance of PNG files with partial alpha.
Details:
Ugh, two separate problems related to how we associate alpha (i.e.
premultiply the colors) for PNG pixels with partial alpha. The correct
thing to do is linearize the unassociated pixel value first, then
associate, then go back to the nonlinear space.
First problem: PNGs have three possible transfer functions: gamma
correction (with a particular gamma), no gamma correction / linear, and
explicitly sRGB. Guess what? We were neglecting the case of pngs tagged
as sRGB and not doing the linearize/delinearize round trip for those
images.
But if that's not enough, also for the gamma case, we were, ugh,
swapping the gamma and 1/gamma, resulting in those partial alpha pixels
ending up a whole lot darker than they should have been.
None of this affected most ordinary PNGs with no alpha channel, or where
alpha was 1.0. It only affected "edge" or "partially transparent" pixels
with 0 < alpha < 1. But it was definitely wrong before, for which I
apologize and hope you'll understand why those pixels are going to
change now (hopefully, always always for the better).
While I was at it, I also made the color space handling a little more
robust -- instead of just a straight string compare for color space
names, use the ColorConfig to check `equivalent`, which should make us a
lot more robust against aliases and whatnot.
Fixes#4314Closes#4054
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
In several places, we still used OIIO versions of isnan/isinf/isfinite
and is_same. All of there for some time have been aliased to the std
versions (having our own was mostly a pre-C++11 artifact). Also,
there are a couple obsolete function names (pow2roundup and
pow2rounddown were replaced with ceil2 and floor2 to match later
C++ standard names).
Change those OIIO::foo to std::foo or use the modern names
throughout the codebase, and hide the old declarations themselves
behind some guards that:
1. Will eliminate them completely as soon as the version is higher than
2.6.x.
2. Makes them invisible now internally to OIIO (so we aren't
accidentally using them ourselves), and to external projects (like OSL)
that define OIIO_DISABLE_DEPRECATED to hide deprecated things.
3. Add deprecation warnings if they are not hidden by the logic of (2)
above.
Before 3.0 release is final, we will truly remove them entirely, but
these staged warnings will help people with the transition.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Fully remove some things that have had deprecation warnings for a long,
long time (mostly since 1.x), so we think it can't possibly break
anyone.
Add deprecation warnings to some things that have been nominally
deprecated for some time. The build warnings will serve as final notice
that they will fully disappear by OIIO 3.0 release.
This is part of a series of small steps to fully deprecate a whole lot
of cruft between now and the 3.0 release.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Co-authored-by: Thiago Ize <ThiagoIze@users.noreply.github.com>
The primary issue seemed to be that the ChannelInfo width and height
were not filled in for some of the layers, notably the merged image data
layer in most(all?) cases.
A second issue was that there was incorrect nesting of the switch
statement in `read_channel_row`. The last 3 case statements were all
nested under the `if (!bigendian()) {` conditional from the 1st case.
Fixes#4298
## Tests
Use `iconvert` on all PSD files contained in the attached .zip.
Converting to both .exr and .png is a good test to ensure the data is
flowing out to those formats correctly.
[psd.zip](https://github.com/user-attachments/files/15905814/psd.zip)
---------
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
This is the continuation of [4305
](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/4305)(Add
extra step to Makefile config target to force CMake to rescan).
The error occurs when configuring with `make
OpenImageIO_BUILD_MISSING_DEPS=all` option which fails the build as
`libuitil` does not have `./dep/dist/include` on the path.
The original solution was to rerun cmake and let it rescan paths. This
step was only required to run once - after the initial CMake
configuration step. This kind off worked but I'm still unsure why. So
instead of that we can just add the missing location to CMakeLists.
Signed-off-by: kuba <kuba456@gmail.com>
Build-time check for Cuda toolkit as optional dependency. This can be
explicitly disabled with OIIO_USE_CUDA=OFF (though currently, it
defaults to off, you have to turn it on at build time because this is
all still experimental). When this is enabled at build time, C++
preprocessor symbol OIIO_USE_CUDA will be defined within the OIIO
codebase.
OIIO global `attribute("gpu:device", "CUDA")` can be used to enable CUDA
functionality, which at this point doesn't do anything other than
initialize the CUDA device and find out information about it. Several
global `getattribute("cuda:*")` queries let you find out some
information about any device found.
I'm trying to keep a lot of the approaches fairly generic, so that the
interface won't need to change much to allow OpenCL or Metal or whatever
else comes along, though for now I'm trying to flesh out the
implementation mostly with CUDA.
oiiotool adds a new `--cuda` option, reserved for enabling CUDA
functionality. Currently the only way this is used is to print
information about the CUDA device in the `oiiotool --help` messages. An
example of that is:
```
$ oiiotool --cuda --help
oiiotool -- simple image processing operations
...
OIIO 2.6.2.0spi | Linux/x86_64
Build compiler: gcc 9.3 | C++17/201703
HW features enabled at build: sse2,sse3,ssse3,sse41,sse42
CUDA 11.6.0 support enabled at build time
...
Running on 32 cores 187.3GB sse2,sse3,ssse3,sse41,sse42,avx,avx2,avx512f,
avx512dq,avx512cd,avx512bw,avx512vl,fma,f16c,popcnt,rdrand
Compute hardware available: CUDA on Quadro RTX 6000, driver 12020,
runtime 11060, compat 705, memory 23.6 GB
```
Note that at this point, we aren't actually *using* CUDA for anything.
This is just setting up some basics for future expansion. It's all
hidden in the pvt namespace for internal use only, and any of the
functions or nomenclature may change as we continue to add
functionality. I want to iterate on it a bit before it's in any way
exposed in public headers or interfaces.
I'm a big fan of introducing even big strategic changes in the form of a
series of MVP easy to review discrete steps. This step's only goals are:
(a) allowing the build system to find and link against the CUDA Toolkit;
(b) allowing runtime initialization and querying of a few basic facts
about the device found; (c) start noodling around just a bit with
function and attribute interfaces to gain a little experience that will
guide what we eventually want.
LONG TERM goals of this initiative include:
* A full TextureSystem work-alike that can be used from CUDA (and
possibly other compute device APIs) based renderers and with OSL's OptiX
back and, having an analogous (ideally identical) interface and as close
as possible to feature parity with CPU TextureSystem.
* Augment ImageBuf with the ability to store its images in GPU (or
unified) memory and for IBA functions to be able to operate on those
GPU-side buffers with GPU compute kernels, somewhat analogously to how
libraries like PyTorch provides optional GPU support for tensor
operations. (The hope is that this improves impage processing
performance, though it's possible that it'll turn out that typical OIIO
image processing workflows are so I/O dominated that there's not a lot
of potential upside. I think we'll have to try before we really find out
if it's going to be worth it.)
* Allow image reading/writing plugins (or other parts of OIIO
functionality) to leverage a GPU if available, and if there is something
productive they can do with it.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Simplifies the clearing of the tile cache so it no longer allocates a
vector of tile IDs prior to erasing them all.
---------
Signed-off-by: Curtis Black <curtis.w.black@gmail.com>
* List external dependencies found, as well as those not found.
* Sort and deduplicate the lists before printing.
* Print versions of everything
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Split the CI "oldest/hobbled" test into two different tests:
* oldest -- test against the oldest supported versions of all
dependencies.
* hobbled -- test against most optional dependencies missing and
optional feature disabled (including no SSE, no OCIO, etc.).
And add an additional test:
* local-builds -- a test that builds and uses every dependency that we
know how to build locally/automatically.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
freetype 2.10.0 dates from 2019, so we're still supporting 5 years back.
Previously, we enforced no minimum, so we're not sure how far back it
would really work, but the earliest version we tested in CI was 2.8,
which dated from 2018.
Add a build_Freetype.bash for easy setup for CI (and other users).
Add build_freetype.cmake to allow automatic local builds if it can't
be found at build time.
CI many cases need to build freetype, because the old CentOS 7 era
freetype is now too old.
Remove some reference output that was only needed for older freetype.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We're seeing nvcc compilation warnings like the following:
```
ustring.h(869): warning #20012-D: __device__ annotation is ignored on a function("operator=") that is explicitly defaulted on its first declaration
__attribute__((host)) __attribute__((device))
```
According to nvidia's documentation
(https://docs.nvidia.com/cuda/archive/10.1/cuda-c-programming-guide/index.html#compiler-generated-functions),
functions that are explicitly-defaulted should not have execution space
specifiers like \_\_device\_\_ or \_\_host\_\_. Instead the compiler
will determine the annotations from the functions that invoke it.
This patch goes through the ustring and typedesc headers and removes
OIIO_HOSTDEVICE from all defaulted functions.
Signed-off-by: Chris Hellmuth <chellmuth@gmail.com>
Latest versions of cmake are warning about FindPython being a deprecated
module, they recommend FindPython2 or FindPython3 instead. In master,
we're already python3 minimum, so go ahead and switch this to the modern
method.
This also seems to fully fix our recently broken CI on one of the Windows tests!
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
High level TLDR:
* If checked_find_package doesn't find a dependency (or it is not an
acceptable version), it looks for `src/cmake/build_<pkg>.cmake` and
if that exists, includes it. It can do anything, but is expected to
somehow provide the dependency so that a second find_package will
find it and then proceed as if it were a system install.
* I've implemented these scripts so far for Imath, OpenEXR,
OpenColorIO, fmt, and robin-map, that download, build, install the
package in OIIO's build area. More to come later?
* This is really simple with a new build_dependency_with_cmake macro,
much simpler than ExternalProject_Add, as I've seen it used
elsewhere.
* Just look at any of the new build_blah.cmake files to see how simple
it is for each new dependency we set up this way.
* By default, pre-installed packages it can find always take precedent
over building locally. So if you have all the dependencies already
installed, none of this should behave any differntly than before.
But there are variables that let you override on a package by package
basis, giving the option of never building locally, building locally
if the package is missing, or forcing a local build to always happen.
----
Various details:
A bunch of cmake things (including checked_find_package) have been
moved into a new file, dependency_utils.cmake.
build_Imath.cmake, build_OpenColorIO.cmake, build_OpenEXR.cmake,
build_Robinmap.cmake, and build_fmt.cmake implement local builds of
those packages. They're very simple, and lean heavily on common
infrastructure of build_dependency_with_cmake, which also can be found
in dependency_utils.cmake.
Robinmap and fmt are extra simple because we use them as header-only
libraries.
For Imath and OpenEXR, I build them as static libraries, so they will
be incorporated into libOpenImageIO (and/or _Util) libraries to be
totally internal to them, there should be no symbols exposed outside
our libraries. This should mean that the resulting libOpenImageIO
should be perfectly safe to link in an application that also links
against OpenEXR, Imath, or OpenColorIO, even different versions
thereof, without any interference. Note that none of those packages
are used in our public APIs, only internally.
OpenColorIO was a little trickier. It builds its own dependencies as
static libraries that are internalized, but OCIO itself is a dynamic
library. So we end up having to make it part of our install, but I use
OCIO's build system to make a custom symbol namespace and a custom
library name, so it still should not interfere with any other OCIO
linked into the application.
We'll see how it goes for furture dependencies we want to add. The
header only, static libraries incorporated and hidden, and dynamic
library but renamed and with custom namespace, are all techniques that
work well. I'm not sure I'd advocate doing local builds of any
dependency that we can't incorporate in one of these ways, but I guess
we'll cross that bridge when we get to it.
New option_utils.cmake has two handy new utilities: set_cache() is
much like the built-in set() when making a cache variable, and
set_option() is much like build-in option(). The biggest difference is
that both allow an environment variable of the same name, if it
exists, to supply the default value. This is something that cmake does
with many of its own controls, like CMAKE_BUILD_TYPE, but does not
make any provision for built-in set() or option() let users do it.
checked_find_package() has moved to dependency_utils.cmake, and has
been enhanced to take several new options, and also so that if the
enclosed find_package() fails and there is a
src/cmake/build_PKG.cmake, it will run it to build the dependency
itself in the build area. If that build_PKG sets a variable called
PKG_REFIND, it will try find_package again to find the one just built.
build_dependency_with_cmake() is given a git repo and tag, and
basically clones the repo, checks out the tag, configures, builds, and
installs it (all in our own build area).
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This cuts down CI time, as before we were building both release and
debug versions of the packages.
The dependency building step of our CI typically took around 6.5-7
minutes previously, and with this patch, now typically is about 5
minutes and sometimes slightly less.
Thanks so much to @cedrik-fuoco who pointed me at this feature of vcpkg
which had eluded me for a long time!
Signed-off-by: Larry Gritz <lg@larrygritz.com>
A few minor fixes for things I noticed as I started to actually use
these new utilities in other branches I'm working on.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
While making a CI to build python wheels for OIIO on windows I spotted a
few issues in the instructions, these edits made it work for me.
Small adjustment to expected paths for the manual instructions.
Signed-off-by: Mel Massadian <melmassadian@gmail.com>
This file imageio_pvt.h.in was used as the input template for CMake to
create imageio_pvt.h, doing a couple build variable substitutions along
the way. But this was inconvenient (because it doesn't look like a real
.h file to editors or clang-format) and it turns out that there were
only 2 lines that needed the substitution.
So in this patch, we split the file, making 99% of it just plain old
imageio_pvt.h that needs no rewriting, and the couple lines that need
variable substitions becomes buildopts.h.in and is included only from
the one source file that actually cares, imageio.cpp.
Other than cleaving those two definitions off to another header and some
file renaming, I haven't changed any of the other content of the
headers, so there should be no change in code the compiler sees or other
behavior.
The reformatting you see is because now that it's a legit .h file,
clang-format wants its way with it.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
I misunderstood all along -- the add_compile_definitions,
add_compile_options, etc., aren't truly global, they are restricted to
targets in the same directory or below. They aren't really prone to
polluting "surrounding" projects when OIIO is used as a subproject.
So for simplicity, reverting that part and using the usual built-ins.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
New set_utils.cmake has some handy new utilities: set_cache() is much
like the built-in set() when making a cache variable, and set_option()
is much like build-in option(). The biggest difference is that both
allow an environment variable of the same name, if it exists, to supply
the default value. This is something that cmake does with many of its
own controls, like CMAKE_BUILD_TYPE, but does not make any provision for
built-in set() or option() let users do it.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
In PR #4268, I accidentally changed the clause addressing what is
seemingly a bug with older MSVS. I'm just putting things back the way
they were. It's not a new change, just restoring this line closer to the
way it was about 3 commits back.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
* Switch runner type for bleeding edge, use gcc14 and python 3.12
* Various minor gcc-14 + C++20 + Python 3.12 fixes
Notable: gcc14 warns about std::atomic_load for share_ptr as deprecated,
since C++20 provides a true atomic shared pointer.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
We've been plagued on and off for a long time with spurious wrong
results in our SIMD versions of exp and log. It was really squirrelly,
came and went with changes to seemingly unrelated code, and had long
periods of time with no symptoms or ability to reproduce it. It just
started popping up again in CI in the dev-2.5 branch, consistently.
Today I think I finally tracked it down for real. I believe the problem
was actually in the SIMD bitcast_to_int and bitcast_to_float functions,
for the variety that was 8-wide when running on 4-wide hardware, and the
16-wide one when running on 8-wide (or less) hardware.
Background: When real 16-wide HW is not available, we define the 16-wide
vectors as an array of two 8-wide vectors. (Same for 8 vs 4, I'll spare
you explaining it all twice.) So most of the 16 wide functions boil down
to:
vfloat16 op (vfloat16 arg, ...)
{
#if 16 wide HW is available
return real_16_wide_intrinsic(arg);
#else /* recursively define in terms of 8-wide halves */
return vfloat16(op(arg.lo()), op(arg.hi()));
#endif
}
But for the bitcast operators, we did something different, a shade too
clever: we just did a pointer cast, dereference and return:
vint16 bitcast_to_int (const vfloat16& x)
{
#if OIIO_SIMD_AVX >= 512
return _mm512_castps_si512 (x.simd());
#else
return *(vint16 *)&x;
#endif
}
I believe that the problem is that this cast is undefined behavior for
the case where the vector x was broken into two halves, and by being in
the middle of a long code sequence of SIMD ops, the two halves happened
to be in registers at that moment. Basically, we are taking the address,
and it just doesn't seem to always understand that it needs to
materialize the whole thing in memory for the pointer cast to work
properly.
I removed the pointer cast and replaced with
return vfloat16(bitcast_to_float(x.lo()), bitcast_to_float(x.hi()));
basically falling back on the usual recursive definition of 16-wide
operations to being in terms of 8-wide operations when 16-wide HW is not
available. This totally clears up the problem since we're no longer
potentially relying on needing the address of something that has no
address at that moment.
A couple other spots (andnot) also had dubious casts that I fixed in the
same way.
N.B.: The 16-wide cast from bool to int had to do something slightly
different, because it's represented differently and isn't a true
bitcast.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This should address the following 2 issues when using MSVC:
- MSVC does not set defines like `__SSE2__` or `__SSE4_2__` etc. we must
do that ourselves
- MSVC uses the `/arch` flag differently than other compilers and it
should be set only once
The ramifications of not having those defines set basically means that
OIIO won't notice any SIMD is possible.
For MSVC the /arch flag only controls which instruction set the compiler
may use when generating code "on its own". For example, the x64 compiler
by default will use SSE2 whenever possible when its generating code but
when using `/arch:AVX` it will produce AVX code as it sees fit. NOTE:
case matters here too! "avx" does not work but "AVX" does.
However, if the source code uses intrinsics, MSVC will happily use those
instead. In fact it's possible to use `avx512` intrinsics without
specifying any /arch flag at all and it will happily generate
corresponding avx512 code. This is much different than say GCC where, in
order to use intrinsics at all, you must specify the corresponding flags
allowing you to do so for the arch in question.
Fixes#4265
## Tests
Existing simd_test.cpp should be sufficient I think. In addition to
those, I've verified the below locally:
Currently: when using `-DUSE_SIMD="sse4.2,avx2"` to build, we get the
wrong `build:simd` flags:
`sse2`
After: using the same flags as above:
`sse2,sse3,ssse3,sse41,sse42,avx,avx2`
---------
Signed-off-by: Jesse Yurkovich <jesse.y@gmail.com>
A small piece of the ongoing switch to std::format/print style text
output. This part is for error and debugging output when reading/writing
exr images.
I also switched conditional compilation to the DBG macro idiom that I've
used elsewhere, where it takes a combination of DEBUG compile and
setting of environment variable OIIO_DEBUG_OPENEXR to trigger the printf
style debugging (the first use case of this was in our color management
code, which I also modify slightly here to conform).
Also opportunistically simplify gcd declaration now that we're C++17
minimum.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Add proxy support for EXR multipart output by taking the setup code from
the other open() overload and calling the constructor accepting a stream
instead of a filename.
---------
Signed-off-by: Joachim Reichel <43646584+jreichel-nvidia@users.noreply.github.com>
Recent libtiff versions export cmake configs that, if you built libtiff
as a static library, will reference targets it set up when it built, but
which are nonstandard names and won't exist when you build and consume
their config exports. Ugh!
Some can be swept under the rug by turning off some libtiff optional
dependencies, but I can't seem to get rid of a target CMath::CMath that
static libtiff target wants. It just points to the math library. So I'm
just setting that up so it will be found when we do the
find_package(tiff). Oh, the hoops we jump through for badly constructed
packges.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Some weird combination of old libheif library gives a different
oiio:OriginalOrientation result. Need this ref because that's how it
behaves in that version (which I have to build against at work), but
it's not worth tracking down a "fix" because it's so old and works fine
on newer versions.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This is just a code move. There no changes to the code logic, though in
the process, I do change old "local" uses of eval_as_bool to the new
Strutil one.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Background: Some formats that support multi-image files know as soon as
they are opened and read the header what the total subimage count is.
Others (I'm looking at you, TIFF) can't know without reading each
subimage in succession until you hit the end of the file. Format readers
who can easily figure it out open first opening are supposed to set
attribute "oiio:subimages" to the number of subimages, or 0 / unset if
it can't be cheaply or immediately determined.
ImageBuf has a `nsubimages()` method that you'd think would give you the
right count, and it did when all IBs were read through the ImageCache
(which fully inventories the subimages of any file it reads). But when
OIIO 2.5 changed IB behavior to only be backed by IC when explicitly
requested, we effectively lost this functionality most of the time, and
instead returned 0 ("don't know yet"). This was confusing. And also
silly, because it reported 0 even for files whose formats don't permit
multiple images (so you know the count is 1).
This patch does the following:
* Changes the behavior and documentation of `ImageBuf::nsubimages()` to
the following return values:
- 1, for formats that are always a single subimage. (Or for ImageBufs
that are not direct reads of a file.)
- n>=1 for files that permit multiple subimages and the count is known.
- 0 for files that permit multiple subimages, but the count is not known
because it's one of the formats where it can't easily be determined upon
opening the file.
* Add a new ImageInput::supports("multiimage") that says if the input
file format permits multiple images, and ensure that it returns true for
all formats where this is the case.
* Ensure that formats permitting multiimage and can easily know the
count up front set the "oiio:subimages" attribute (not all did).
* Document in the built-in format section of the docs the formats that
are expected to set this attribute.
Fixes#4225
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This is a continuation of PR 4246, had one more spot where we needed to
disable warnings for unity builds involving clang18 and deprecated
unicode conversion functions.
There's some changes included to ci.yml and install_homebrew_deps.bash
that are part of work I did to make it easier for me to force it to use
clang-18 for certain tests while working on this.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Remove some long-ago deprecated printf-like error functions. Mark as
deprecated some other ones that we will remove when it gets closer to
3.0 release. I think these are the very last uses internally of the old
style that we want to remove.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The nullptr at the end of default_font_name was left over from a time
that we iterated by going through array elements until we hit the
nullptr that signalled we had passed the last valid element. At some
point we switched to a range-for, which iterates over every element of
the array without needing an "end sentinel", and we should have
eliminated the nullptr but failed to do so.
Fixes#4248
Signed-off-by: Larry Gritz <lg@larrygritz.com>
clang/llvm 18 (and probably other compilers soon) has made noisy
warnings about the C++17-deprecated unicode conversions. Actually
replacing them enirely with something else is rather hairy, we will
tackle that later.
In the mean time, this patch seems to be the minimal change to disable
the warnings that should be totally easy to backport.
These seem to be the only changes necessary to make a clean build with
clang 18.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Only use the OIIO_LIBRARY_PATH environment variable by default.
Searching through such system paths has some risk loading plugins
installed for another purpose or that may be incompatible.
If the previous behavior is desired, applications can add these
environment variables to the plugin search path manually.
Signed-off-by: Brecht Van Lommel <brecht@blender.org>
The recent switch from boost thread_specific_ptr left a bug: if an
ImageInput (say) is deleted without the caller clearing its error state,
and then later another ImageInput is allocated and happens to get the
same address as the first one, it can "inherit" the unretrieved error
state left behind by the other.
Fix by using a unique ID (initialized by an incrementing atomic
counter) to index the object in the per-thread error store.
I feel like we still don't have a foolproof replacement for the really
nice lifetime management that thread_specific_ptr was doing for us. One
could argue that unretrieved errors in an object are a "bug" in the
user's program, but unretrieved bugs currently just accumulate in the
map, even after the ImageInput is destroyed, until the thread terminates.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
These are build-breaking changes that impose a new minimum OpenEXR/Imath
version supported of 3.1 or higher.
Needless to say, this will not be backported to any release branches,
which guarantee that they will never raise the minimum supported
versions of any dependencies. This is just for future OIIO. There is
already a "developer preview" branch dev-2.6.1 marking the last spot in
master in which the old dependencies are supported.
Fixes#4156
Signed-off-by: Larry Gritz <lg@larrygritz.com>
It was pointed out that oiiotool and maketx's habit of embedding their
exact command line arguments in the Software and ImageHistory metadata
of output images was potentially problematic in that it might
inadvertently leak important internal data (for example, file paths may
reveal secret projects, etc.).
This patch changes the behavior of oiiotool and maketx as follows:
* The default behavior is now to use `"OpenImageIO <version> <hash>"`
where `<hash>` is the SHA-1 hash of the command line arguments (which
allows you to tell if two files use the same command line args, but
doesn't let you know what they were).
* A new `--history` argument (or setting OIIOTOOL_METADATA_HISTORY env
variable to a nonzero integer) instead uses the full command line
arguments with no obfuscation. (This used to be the default behavior but
is now deemed unsafe.) When the env variable is in effect, a
`--no-history` argument explicitly disables it.
* For oiiotool, the existing `--nosoftwareattrib`, as before, causes the
Software and ImageHistory metadata to not be modified at all.
Here is an example of the behavior:
```
$ oiiotool tmp.tif -o tmp.exr && iinfo -v tmp.exr
tmp.exr : 640 x 480, 3 channel, half openexr
channel list: R, G, B
Software: "OpenImageIO 2.6.2.0dev : B897973B86EC13E3F1858AAE5E057EFCCF99889A"
...
$ oiiotool --history tmp.tif -o tmp.exr && iinfo -v tmp.exr
tmp.exr : 640 x 480, 3 channel, half openexr
channel list: R, G, B
Software: "OpenImageIO 2.6.2.0dev : oiiotool --history tmp.tif -o tmp.exr"
Exif:ImageHistory: "oiiotool --history tmp.tif -o tmp.exr"
...
```
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This is an initial code for the RED Digital Cinema support.
CPU only so far.
See the RED section in the docs builtinplugins.rst for more details on what currently works and current limitations.
---------
Signed-off-by: Peter Kovář <peter.kovar@reflexion.tv>
This is just the first step in many for removing deprecated things along
the way to a 3.0 release. I'm doing a little at a time.
I believe that here I'm only removing things that (a) have been
deprecated since 2.0 or earlier, (b) most or maybe all were marked with
OIIO_DEPRECATED and so should have been triggering warnings for a long
time if used downstream (thus, almost certainly nobody is using it), and
(c) are inline definitions, so this change should not appear to break
link ABIs.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
As Marie Kondo would say, Boost no longer sparks joy for us, so we take
one last look, thank it for its many years of service to our project,
and then send it on its way.
Fixes#4158
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Remove the dependency on boost's thread_specific_ptr by using hashmap's
of static duration between the owning object and the target datatype.
After digging a bit into how boost's pointer works behind the scenes, it
is actually a very similar mechanism, with the lookup being into an
std::map. I don't have an easy way to measure performance here, but none
of these are on a hot path (with the possible exception of the
per_thread_info for the image cache, but this one can be managed
externally if you want the best performance.
---------
Signed-off-by: Chris Kulla <ckulla@gmail.com>
We disable fmt execptions and force it to do one of our ASSERT's
instead. But fmt in their main branch recently added `[[noreturn]]`
annotations in a way that cause this to be compiler errors for our
overrides. Needed an extra std::terminate() call to guarantee the
noreturn promise.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
The doc comment said it should initialize, and the analogous structure
for scalar texturing, TextureOpt, initialized. Missed this one.
Fixes#4219
Signed-off-by: Larry Gritz <lg@larrygritz.com>
This change fixes out-of-bounds reads and incorrect scanline ordering
when OpenEXR's decreasingY lineOrder mode is used.
OpenEXR expects to process scanlines in decreasing order when the
decreasingY lineOrder option is enabled. This change fixes the code that
provides the chunks of scanlines to OpenEXR so that the proper scanlines
are available when it accesses the FrameBuffer OIIO created. The old
code was providing incorrect scanlines AND setting up the FrameBuffer
with incorrect pointers so out-of-bounds reads were occurring.
The key things to keep in mind are:
- Chunks of scanlines need to be sent to
OpenEXROutput::write_scanlines() from the bottom of the image to the top
when decreasingY is enabled. If the chunk's scanline range does not
contain the scanlines OpenEXR is expecting to fetch, then it can cause
out-of-bound reads because the wrong information is used to construct
the pointers for the Slices created when constructing a FrameBuffer.
- OpenEXROutput::write_scanlines() does its own chunking of scanlines
and this logic must also properly start from the bottom of the chunk
data passed into this function when decreasingY is enabled.
- The to_native_rectangle() call in OpenEXROutput::write_scanlines() may
return a pointer to m_scratch if a format conversion is needed. This
means the pointers passed to OpenEXR, in this situation, _WILL NOT_ be a
full frame buffer. This caused OpenEXR to do out-of-bounds reads because
the computations used to create the Slices were using the wrong scanline
number to adjust the pointer.
- The progress callback computation needed to be modified to handle
traversing scanlines in decreasing order.
I added an openexr-decreasingY test to verify that the code no longer
crashes because of the out-of-bound reads and it also verifies that the
decreasingY images match the increasingY images.
---------
Signed-off-by: Aaron Colwell <300262+acolwell@users.noreply.github.com>
It seems that two targets `libOpenImageIO`, `libOpenImageIO_Util` were left out
during the [switch to target-based definitions]
(https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/4193)
refactor and have no `-ftest-coverage -fprofile-arcs` options added in
the CODECOV=1 mode.
This was due to #4193 failing to propagate the link options to the dynamic libraries.
Adding the missing options resolves build errors.
Signed-off-by: kuba <kuba456@gmail.com>
To shed the boost dependency, we are disabling stacktrace for now. It's
not vital and probably not worth keeping boost just for that.
C++23 has std::stacktrace, so as compilers add support for this, we will
phase it back in.
We can also add it back if somebody has an alternate suggestion for a
3rd party solution that is a simply drop-in replacement, portable,
lightweight dependency, with compatible license. A good compromise might
be this one: https://github.com/jeremy-rifkin/cpptrace But I think what
I'm proposing is that we accept this PR first, get rid of boost, then we
can come back to looking at cpptrace as potentially a way to restore the
functionality prior to C++23.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
LibRaw 0.20 dates from mid-2020 and in any case, this version or newer
is necesary for use with C++17 which is our new minimum.
Bump the version built by build_scripts/build_LibRaw.bash to the latest.
Remove some old libraw testsuite output references that were only
applicable to the older versions.
This change will not be backported to release branches, the bump will
only be for OIIO 3.0 and later.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Adds support for the following DXGI formats when using the DX10 extended
header on DDS files:
- Compressed BCn and its sRGB variants.
- BGRA8/BGRX8/RGBA8 and its sRGB variants.
- RGB10A2.
- R16.
On the sRGB formats, the attribute `oiio:ColorSpace` is also set to
`sRGB`.
Closes#4045.
Added new DDS test images to the OpenImageIO-images repo and updated the
testsuite expected output.
OpenImageIO-images PR:
https://github.com/AcademySoftwareFoundation/OpenImageIO-images/pull/7
Signed-off-by: Alejandro Aguirre <alexguirre.aa@gmail.com>
Soon we'll raise the minimum OpenEXR to 3.1, and OpenVDB 9.0 is the
first version to support that. And because OpenVDB has Imath types in
its public APIs, it's really hard to mix and match our using an
OpenEXR/Imath 3 internal to OIIO with an OpenVDB that uses an
OpenEXR/IlmBase 2.
---------
Signed-off-by: Larry Gritz <lg@larrygritz.com>
A tiny bit more progress towards #4158.
None of the code was actually dependent on the fact that these
containers are sorted by key. Switching to the robin containers instead
removes one less use of boost in the project.
Signed-off-by: Chris Kulla <ckulla@gmail.com>
These are build-breaking changes that impose a variety of new minimum
dependencies for the toolchain.
The new minimum Python version supported is now 3.7.
Get rid of 'from future' lines in python scripts in testsuite.
Remove python 2.7 specific ref images from testsuite.
Needless to say, this will not be backported to any release branches,
which guarantee that they will never raise the minimum supported
versions of any dependencies. This is just for future OIIO.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
These are build-breaking changes that impose a variety of new minimum
dependencies for the toolchain.
C++17 is the new minimum. All hail C++17.
Bump minimum compiler versions to the respective releases of each one
that is requred to fully support C++17 for us. This means new minimum
changes of gcc 6.1 -> 9.3, clang 3.4 -> 5.0, icc 17 -> 19, and MSVS
v19.14 or higher now.
Remove CI tests for too-old platforms, and adjust some others that
needed particular components upgraded. We now will no longer attempt to
test on "VFX Platform 2019" or 2020. The 2021 platforms are the earliest
we claim to support (reflecting VFXP's guidance about trying to support
up to 3 years back).
By removing the old test cases, we're also removing the last platforms
that test Python 2.7. So while this PR isn't the one that officially
removes the last vestiges of Python 2, it does mean that once this is
merged, we will no longer be testing less than Python 3.7. (A separate
PR removing all Python 2 support will follow and presumably be merged
around the same time.)
With C++17 min, remove traces of boost::filesystem. We can rely on
std::filesystem now.
Clean up pre-C++17 macros in platform.h
Needless to say, this will not be backported to any release branches,
which guarantee that they will never raise the minimum supported
versions of any dependencies. This is just for future OIIO. I won't
merge this until after April 1, at which point I will also tag the last
spot in master that is known to work with C++14/gcc6.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
As mentioned in #4205, #3900 and
https://github.com/AcademySoftwareFoundation/OpenImageIO/discussions/3910#discussioncomment-8913737
OpenImageIO previously did not support reading of 16- and 32-bit files
beyond their merged image data section. I have now implemented this by:
1. Adding support for reading layer info from the "Lr16" and "Lr32"
Tagged Blocks as well as adding additional checks to the regular layer
info section
2. Implemented the Zip and ZipPrediction compression codec which is
required to read these files as that is photoshops preferred compression
codec for those bitdepths
Due to the way the Zip data is stored (per channel but not per scanline)
the current implementation reads the whole channel into a buffer which
is then memcpy'd from by calls to `read_native_scanline()`. The buffer
gets cleared on deletion of the `PSDInput` object
---------
Signed-off-by: EmilDohne <86836589+EmilDohne@users.noreply.github.com>
Signed-off-by: Larry Gritz <lg@larrygritz.com>
Co-authored-by: Larry Gritz <lg@larrygritz.com>
Windows CI was failing because it couldn't get a proper libtiff from
VcPkg, which in turn is because lzma is a dependency of the libtiff
build in VcPkg, and for the time being, it looks like VcPgk is failing
to get lzma which is part of the big xz debacle of 2024.
So work around for now by just building our own libtiff rather than
relying on VcPkg. Seems to work fine.
Signed-off-by: Larry Gritz <lg@larrygritz.com>
description:Generate or update release notes for a patch, minor, or major release, or just to update main. Run git-cliff, organize and edit the output per project conventions, and insert into CHANGES.md.
argument-hint:[patch|minor|major|main] [prev-tag]
---
Generate release notes for an OpenImageIO release.
Arguments: `$ARGUMENTS`
- First argument (optional): release type — `patch` (default), `minor`, `major`, or `main`.
- Second argument (optional): previous release tag (e.g. `v3.1.2.0`). If omitted, find the most recent tag automatically. If the release type is `main`, instead of a tag, look for the last commit at which the CHANGES.md file was updated.
## Steps
1.**Determine the previous tag** if not provided:
```
git describe --tags --abbrev=0
```
or look at recent tags: `git tag --sort=-version:refname | head -10`.
However, if the "release type" is `main`, instead of a tag, just find
### New minimum dependencies and compatibility changes:
### ⛰️ Major new features and public API changes:
* *New image file format support:*
* *oiiotool new features and major improvements*:
* *Command line utilities*:
* *API changes*
* Other notable new feature:
### 🚀 Performance improvements:
### 🐛 Fixes and feature enhancements:
### 🔧 Internals and developer goodies
### 🏗 Build/test/CI and platform ports:
* CMake build system and scripts:
* Dependency and platform support:
* Testing and Continuous integration (CI) systems:
### 📚 Notable documentation changes:
### 🏢 Project Administration
### 🤝 Contributors
```
Note that the section outline may already be present, in which case you
only need to fit items into the existing category outline.
Rules:
- Group commits into sections; within each section, cluster related items together.
- When needed, expand terse one-liners into enough prose that users understand what changed and why it matters.
- For `feat:` commits, make sure the feature is explained sufficiently — don't just copy the commit subject.
- For `api:` or `api!:` commits, clearly call out what changed in the public API.
- Include PR links and author attribution for every entry, in the combined
`(by @githubid / Full Name)` form from step 3.
- The notes should "tell the story" of the release, not just be a dump of commit subjects.
- We aim to make major/minor releases approximately in October 1 of each year. If the anticipted release date is already in the file, don't change it. If it is not present, ask for confirmation of the planned release date.
### For updating release notes in main:
Rules:
- Generally, follow the rules for "major/minor releases", except as noted in other items below.
- Don't apply a new skeleton of category listings unless they are not present for the upcoming release.
5. **Insert the formatted notes** into `CHANGES.md` in the appropriate place (as detailed below). Leave the existing content intact.
- When updating `main` or preparing a `major` or `minor` release, insert the updates in the top section for the upcoming major/minor release. Insert a new set of category sections only if it's not already present.
- When doing a `patch` release, insert the changes immediately above the last patch release of that branch, so that CHANGES.md lists releases in descending numerical (version) order.
- When porting a set of release notes from a release branch into main, or from an older (obsolete) release branch to the current release branch, insert it into the right place to maintain overall descending order.
6. **Double check that the notes are adequately descriptive.** (See more
detailed description of this step below.)
7. **Forward-port release notes from release branches if needed**
8. **Show a summary** of what was inserted and ask the user to review before finalizing.
## Forward-porting release notes
Release notes may have been generated independently in main, and release
branches. When updating one branch, we ensure that any changes from older
branches have been incorporated.
- When preparing `patch` release notes, check the CHANGES.md file in the
"dev-X.(Y-1)" branch for the previous minor release family to identify any
X.(Y-1).Z patch release notes that are not reflected in the current release
notes that we are updating.
- When updating `main` or doing a `major` or `minor` release, check the
CHANGES.md file in both the "dev-X.(Y-1)" and "dev-X.(Y-2)" branches.
Check whichever is newer of the local and remote copy of that branch, since
the local one may have had release notes or its version in CMakeLists.txt
updated, but not yet pushed to GitHub.
- If any patch releases are present in the older dev branches checked, insert
the release notes for those patch releases into the right positions in the
current release notes that we are updating.
- If a change is forward-ported in this manner and the same PR is an update in
the current set of changes we are updating as our main task, document that
in the current set of notes using the following convention: in the line of
the notes, the explanation of the version where it appears should reflect
the first version of all branches where it appeared, for example, `(3.2.0.0,
3.1.3.0, 3.0.8.0)` to indicate that the patch was added to each of those
versions. The versions should be listed in descending order.
## Useful abbreviations for category labels
| Abbrev | Meaning |
|--------|---------|
| IB | ImageBuf |
| IBA | ImageBufAlgo |
| IC | ImageCache |
| TS | TextureSystem |
| oiiotool | the oiiotool command |
| build | CMake/build system |
| deps | Changes to accommodate dependency or toolchain changes |
| ci | CI/GitHub Actions |
| docs | documentation |
| int | internal/refactor |
| test | testsuite or unit tests |
| HEADER.h | Developer utilities in a public header file |
## Combining PRs into single entries
To be more concise and easier to read, within a release's notes, related
PRs/commits can be combined into a single bullet-point line, which would look
@@ -18,10 +18,10 @@ But if you are just asking a question:
* Am I doing something wrong?
* I can't build OpenImageIO
For anything of this nature, the best way to get help using OpenImageIO is
to ask a question on the [oiio-dev developer mail list](https://lists.aswf.io/g/oiio-dev).
For anything of this nature, the best way to get help using OpenImageIO is to
ask a question on the [oiio-dev developer mail list](https://lists.aswf.io/g/oiio-dev) or the [ASWF Slack](https://slack.aswf.io) `#openimageio` channel.
The [documentation](https://openimageio.readthedocs.org)
The [documentation](https://docs.openimageio.org)
is pretty comprehensive, so please check there first; you may find the answer
more quickly than you can get your question answered on the mail list.
YOU MAY DELETE ALL OF THIS IF YOU ALREADY HAVE A DESCRIPTIVE COMMIT MESSAGE!
This is just a template and set of reminders about what constitutes a good PR.
But please look over the checklist at the bottom.
If THIS TEXT is still in your PR description, we'll know you didn't read the
instructions!
### Description
<!-- Please provide a description of what this PR is meant to fix, and -->
<!-- how it works (if it's not going to be very clear from the code). -->
## Tests
### Tests
<!-- Did you / should you add a testsuite case (new test, or add to an -->
<!-- existing test) to verify that this works? -->
## Checklist:
### Checklist:
<!-- Put an 'x' in the boxes as you complete the checklist items -->
- [ ] I have read the [contribution guidelines](https://github.com/AcademySoftwareFoundation/OpenImageIO/blob/master/CONTRIBUTING.md).
- [ ] I have updated the documentation, if applicable.
- [ ] I have ensured that the change is tested somewhere in the testsuite
(adding new test cases if necessary).
- [ ]If I added or modified a C++ API call, I have also amended the
corresponding Python bindings (and if altering ImageBufAlgo functions, also
exposed the new functionality as oiiotool options).
- [ ]My code follows the prevailing code style of this project. If I haven't
already run clang-format before submitting, I definitely will look at the CI
test that runs clang-format and fix anything that it highlights as being
nonconforming.
- [ ]**I have read the guidelines** on [contributions](https://github.com/AcademySoftwareFoundation/OpenImageIO/blob/main/CONTRIBUTING.md) and [code review procedures](https://github.com/AcademySoftwareFoundation/OpenImageIO/blob/main/docs/dev/CodeReview.md).
- [ ]**I have read the [Policy on AI Coding Assistants](https://github.com/AcademySoftwareFoundation/OpenImageIO/blob/main/docs/dev/AI_Policy.md)**
and if I used AI coding assistants, I have an `Assisted-by: TOOL / MODEL`
line in the pull request description above.
- [ ]**I have updated the documentation** if my PR adds features or changes
behavior.
- [ ]**I am sure that this PR's changes are tested in the testsuite**.
- [ ]**I have run and passed the testsuite in CI***before* submitting the
PR, by pushing the changes to my fork and seeing that the automated CI
passed there. (Exceptions: If most tests pass and you can't figure out why
the remaining ones fail, it's ok to submit the PR and ask for help. Or if
any failures seem entirely unrelated to your change; sometimes things break
on the GitHub runners.)
- [ ]**My code follows the prevailing code style of this project** and I
fixed any problems reported by the clang-format CI test.
- [ ] If I added or modified a public C++ API call, I have also amended the
corresponding Python bindings. If altering ImageBufAlgo functions, I also
exposed the new functionality as oiiotool options.
OpenImageIO has been accepted to be an ASWF-administered project, by vote of
the ASWF Technical Advisory Council (TAC) on May 3, 2023.
At this time, the ASWF subdirectory contains *proposed* (and sometimes TBD/WIP
placeholders) project organization and policy documents that will be formally
adopted as the OpenImageIO project is formally turned over to ASWF and moved
to the AcademySoftwareFoundation GitHub account. Please file issues or PRs
or review comments against these documents to make any suggested changes while
we are revising the drafts.
Summary of the current state of documents:
| File | Status | Notes |
| ---- | ------ | ----- |
| CLA-corporate.md | Final | Standard ASWF corporate CLA, approved by the TSC. Will take effect when the repo moves. |
| CLA-individual.md | Final | Standard ASWF individual CLA, approved by the TSC. Will take effect when the repo moves. |
| CODE_OF_CONDUCT.md | Final | Links to the standard LF CoC, approved by the TSC and now in effect. |
| CONTRIBUTING.md | Preliminary | Modified version of our existing CONTRIBUTING guidelines, lightly amended to reflect the changes that will come along with moving the repo, the new CLA/DCO rules, etc.
| GOVERNANCE.md | Final | Approved by the TSC and now in effect. |
| project_intake.md | Ongoing | This is the latest ASWF project intake checklist. We will check off items as we get them done. |
| Technical-Charter.md | Final | Approved by the TSC. |
This is a checklist for TSC's to review as part of the intake process. The TSC should review this entire list during the kickoff meeting. For anything outstanding, create an [issue](../issues) to track and link to it in the list
- Existing Project Governance
- [X] A [LICENSE.md](../LICENSE.md) file in every code repository, with the license chosen an OSI-approved license.
- [X] Any third-party components/dependencies included are listed along with thier licenses ( [THIRD-PARTY.md](../THIRD-PARTY.md) )
- [X] A [README.md](../README.md) file welcoming new community members to the project and explaining why the project is useful and how to get started.
- [X] A [CONTRIBUTING.md](../CONTRIBUTING.md) file explaining to other developers and your community of users how to contribute to the project. The file should explain what types of contributions are needed and how the process works, along with how to disclose security issues responsibly ( may also point to a [SECURITY.md](../SECURITY.md) file ).
- [ ] A [CODEOWNERS](../CODEOWNERS) or [COMMITTERS](../COMMITERS.csv) file to define individuals or teams that are responsible for code in a repository; document current project owners and current and emeritus committers.
- [X] A [CODE_OF_CONDUCT.md](../CODE_OF_CONDUCT.md) file that sets the ground rules for participants’ behavior associated and helps to facilitate a friendly, welcoming environment. By default, projects should leverage the Linux Foundation Code of Conduct unless an alternate Code of Conduct was previously approved.
- [X] A [RELEASING.md](../RELEASING.md) file that provides documentation on the release methodology, cadence, criteria, etc.
- [X] A [GOVERNANCE.md](../GOVERNANCE.md) file that documents the project’s technical governance.
- [ ] A [SUPPORT.md](../SUPPORT.md) file to let users and developers know about ways to get help with your project.
- Infrastructure/Assets
- [X] Technical Charter Approved
- [X] License scan completed and no issues found
- [X] Code repository imported to ASWF GitHub organization or ownership of current GitHub organization given to `thelinuxfoundation` user
- [X] Developer Certificate of Origin past commit signoff done and DCO Probot enabled.
- [X] Issue/feature tracker established (JIRA, GitHub issues)
- [X] Mailing lists ( one of )
- [X] Setup new lists ( -dev@ and -tsc@ ) on [ASWF groups.io](https://lists.aswf.io) ( create [issue on foundation repo](https://github.com/AcademySoftwareFoundation/foundation/issues/new) to setup )
- [X] (In progress) Move to [ASWF groups.io](https://lists.aswf.io) ( create [issue on foundation repo](https://github.com/AcademySoftwareFoundation/foundation/issues/new) to transfer )
- [X] Slack ( create [issue on foundation repo](https://github.com/AcademySoftwareFoundation/foundation/issues/new) to setup project channel on [ASWF Slack](https://slack.aswf.io) )
- [ ] Website
- [X] CI/build environment
- [X] Trademarks/mark ownership rights ( complete 'LF Projects - Form of Trademark and Account Assignment' - create [issue on foundation repo](https://github.com/AcademySoftwareFoundation/foundation/issues/new) - only needed if project using existing name )
- [X] Domain name ( create [issue with the LF IT staff](https://jira.linuxfoundation.org/plugins/servlet/theme/portal/2/group/19) to setup/transfer )
- [X] Zoom account access ( create [issue on foundation repo](https://github.com/AcademySoftwareFoundation/foundation/issues/new) to get access to credentials )
- [X] (N/A) Social media accounts or other project accounts ( create [issue on foundation repo](https://github.com/AcademySoftwareFoundation/foundation/issues/new) to transfer )
- [X] Logo(s) ( create [issue on artwork repo](https://github.com/AcademySoftwareFoundation/artwork/issues/new) to add in SVG and PNG format and color/black/white )
- New Project Governance
- [X] TSC members identified
- [X] First TSC meeting held
- [X] TSC meeting cadence set and added to project calendar
- [X] CLA Approved ( if used ) ( [CCLA](CLA-corporate.md) and [ICLA](CLA-individual.md) )
- Project charter ( [Technical-Charter.md](Technical-Charter.md) )
- [X] [Core Infrastructure Initiative Best Practices Badge](https://bestpractices.coreinfrastructure.org/) achieved as the 'Passing' level.
- Outreach
- [ ] New project announcement done ( create [issue on foundation repo](https://github.com/AcademySoftwareFoundation/foundation/issues/new) to trigger )
- [X] Project added to ASWF website and ASWF landscape
- Incubation Stage requirements
- [X] Have achieved and maintained a [OpenSSF Best Practices Badge at the passing level](https://bestpractices.coreinfrastructure.org/en/criteria).
- [X] Have had a successful license scan with any critical issues remedied.
- [X] Have a defined project mission and scope
- [ ] An overview of the project’s architecture and features defined.
- [ ] A project roadmap defined, which should address the following questions.
- What use cases are possible now?
- What does the next year look like in terms of additional features and use cases covered?
- [ ] Community and contributor growth assessment
- The current number of contributors and committers, and the number of different organizations contributing to the project.
- Demonstrate a sustained flow of commits / merged contributions
- A credible plan for developing a thriving user community, in particular expanding the number of committers and contributors?
- Outline of the plan for the project to complete the requirements for Adopted Stage
- [X] Obtain an affirmative vote of the TAC.
- Adopted Stage graduation requirements
- [ ] Demonstrate a substantial ongoing flow of commits and merged contributions, authored by a healthy number of diverse contributors.
- [ ] Demonstrable roadmap progress.
- [ ] A healthy number of public adopters that are identified within the project ( using an ADOPTERS file or showcased on the project’s website ).
- [ ] Have achieved and maintained an [OpenSSF Best Practices Badge at the gold level](https://bestpractices.coreinfrastructure.org/en/criteria/2).
- [ ] Have a technical lead appointed for voting representation of the project to the TAC.
- [ ] Be deemed by the TAC to add value to the mission of ASWF.
- [ ] 2/3 supermajority vote of the TAC
- [ ] Affirmative majority vote of the Governing Board
guidelines](../CONTRIBUTING.md) to ensure your contribution is in line with
the project's coding and styling guidelines.
* Review the coding standards and [contributing guidelines](CONTRIBUTING.md)
to ensure your contribution is in line with the project's coding and styling
guidelines.
* Sign the Individual CLA, or if you are employed by an organization that
might have any claim to IP you create, have your organization sign the
@@ -79,8 +79,9 @@ request that the TSC restore them to active Committer status.
## Technical Steering Committee
The Technical Steering Committee (TSC) has final authority over this project.
As defined in the project charter, in addition to committer activities, TSC
responsibilities also include, but are not limited to:
As defined in the project [technical charter](https://github.com/AcademySoftwareFoundation/foundation/blob/main/project_charters/openimageio_charter.pdf), in
addition to committer activities, TSC responsibilities also include, but are
not limited to:
* Coordinating technical direction of the Project.
@@ -130,13 +131,12 @@ represents the project at all ASWF TAC meetings.
Current voting members of the TSC are:
* **Chair and TAC representative**: Daniel Greenstein - Sony Pictures Imageworks (possibly interim?)
* **Chair and TAC representative**: Daniel Greenstein - Sony Pictures Imageworks
* **Chief Architect**: Larry Gritz - Sony Pictures Imageworks
* Anton Dukhovnikov - Wētā Digital / Unity
* Jesse Yurkovich - Blender
* Joseph Goldstone - ARRI
* Lewis Wakeland - Disney TV Animation
* Luke Emrose - Animal Logic (interim?)
* Luke Emrose - Animal Logic
* Martin Davies - LAIKA
* Robin Rowe - Heroic Robots
* Thiago Ize - Autodesk
@@ -181,8 +181,8 @@ her absence. Meetings may also be streamed online where appropriate;
connection details will be posted to the oiio-dev mail list in advance of the
scheduled meeting.
The biweekly TSC meetings can be found on the [ASWF Calendar](https://www.aswf.io/meeting-calendar),
just click on the OpenImageIO meeting entries to get the Zoom link.
The biweekly TSC meetings can be found on the [ASWF Calendar](https://calendar.openimageio.org),
just click on the OpenImageIO TSC meeting entries to get the Zoom link.
Items are added to the TSC agenda which are considered contentious or are
modifications of governance, contribution policy, TSC membership, or release
@@ -213,3 +213,6 @@ Due to the challenges of scheduling a global meeting with participants in
several time zones, the TSC will seek to resolve as many agenda items as
possible outside of meetings on the public mailing list or through GitHub
issues, discussions, and pull requests.
Meeting notes (usually very brief) can be found at: https://docs.google.com/document/d/1quSh1K5U2jUDi3cB1eX84i9Y2p7r58w1fZMXM8rnH-w/
| make VERBOSE=1 ... | Show all compilation commands |
| make STOP_ON_WARNING=0 | Do not stop building if compiler warns |
| make EMBEDPLUGINS=0 ... | Don't compile the plugins into libOpenImageIO |
| make USE_OPENGL=0 ... | Skip anything that needs OpenGL |
| make USE_QT=0 ... | Skip anything that needs Qt |
| make MYCC=xx MYCXX=yy ... | Use custom compilers |
| make USE_PYTHON=0 ... | Don't build the Python binding |
| make OIIO_PYTHON_BINDINGS_BACKEND=both ... | For source/CMake builds, build the existing pybind11 bindings and the nanobind (WIP) module |
| make BUILD_SHARED_LIBS=0 | Build static library instead of shared |
| make IGNORE_HOMEBREWED_DEPS=1 | Ignore homebrew-managed dependencies |
| make LINKSTATIC=1 ...| Link with static external libraries when possible |
| make SOVERSION=nn ... | Include the specified major version number in the shared object metadata |
| make NAMESPACE=name | Wrap everything in another namespace |
The command 'make help' will list all possible options.
@@ -278,68 +295,71 @@ Building on Windows
You will need to have Git, CMake and Visual Studio installed.
The minimal set of dependencies for OIIO is: Boost, zlib, libTIFF, OpenEXR, and libjpeg or libjpeg-turbo. If you have them built somewhere then you skip
the section below, and will only have to point OIIO build process so their locations.
The minimal set of dependencies for OIIO is: zlib, libTIFF, Imath, OpenEXR,
OpenColorIO, and libjpeg or libjpeg-turbo. If you have them built somewhere
then if you set the environment variable `CMAKE_PREFIX_PATH` to include those
areas so CMake will find them, the OpenImageIO build will find those pre-built
packages and use them. If you don't have them build, or are unsure, the
`OpenImageIO_BUILD_MISSING_DEPS=all` CMake option will cause the build process
to download the sources of those dependencies, build them, and use them.
* Boost: get the boost source archive, extract into `{BOOST_ROOT}`.
```
cd {BOOST_ROOT}
bootstrap
b2
```
* zlib: this will build it, and then delete the non-static library, so they don't get picked up:
* OpenEXR: you'll have to point it to your `{ZLIB_ROOT}` location from the above. If copy-pasting the multi-line command (with lines ending in `^`) into
cmd.exe prompt, make sure to copy all the lines at once.
and that will build the package and install it into the `{OIIO_ROOT/dist`
area. (If you instead wanted the install to end up elsewhere, the easiest way
is on the earlier config step, to add the option
`-DCMAKE_INSTALL_PREFIX=somewhere_else`.)
On the other hand, if you would prefer to open the generated Visual Studio
solution, the "cmake configure" will have produced
`{OIIO_ROOT}/build/OpenImageIO.sln` that can be opened in Visual Studio IDE.
Note that the solution will be only for the Intel x64 architecture only; and
will only target "min-spec" (SSE2) SIMD instruction set.
Optional packages that OIIO can use (e.g. libpng, Qt) can be build and pointed to OIIO build process in a similar way.
@@ -366,6 +386,57 @@ produce a dynamic-linked version.
3. Execute the PowerShell command from where vcpkg is located in directory. ``vcpkg install openimageio``
**Note: Importing the OpenImageIO Python Module**
As of OpenImageIO 3.0.3.0, the default DLL-loading behavior for Python 3.8+ has changed.
If you've built OIIO from source and ``import OpenImageIO`` is throwing a ModuleNotFound exception, revert to the legacy DLL-loading behavior by setting environment variable
``OPENIMAGEIO_PYTHON_LOAD_DLLS_FROM_PATH=1``.
Python-based Builds and Installs
--------------------------------
**Installing from prebuilt binary distributions**
If you're only interested in the Python module and the CLI tools, you can install with `pip` or `uv`:
> ```pip install OpenImageIO```
**Building and installing from source**
If you have a C++ compiler installed, you can also use the Python build-backend to compile and install
from source by navigating to the root of the repository and running: ```pip install .```
This will download and install CMake and Ninja if necessary, and invoke the CMake build system; which,
in turn, will build missing dependencies, compile OIIO, and install the Python module, the libraries,
the headers, and the CLI tools to a platform-specific, Python-specific location.
See the [scikit-build-core docs](https://scikit-build-core.readthedocs.io/en/latest/configuration.html#configuring-cmake-arguments-and-defines)
for more information on customizing and overriding build-tool options and CMake arguments.
This repo contains python type stubs which are generated from `pybind11` signatures.
The workflow for releasing new stubs is as follows:
- Install [`uv`](https://docs.astral.sh/uv/getting-started/installation/) and `docker`
- Run `make pystubs` locally to generate updated stubs in `src/python/stubs/__init__.pyi`
- Run `make test-pystubs` locally to use mypy to test the stubs against the code in
the python testsuite.
- Commit the new stubs and push to Github
- In CI, the stubs will be included in the wheels built by `cibuildwheel`, as defined in `.github/wheel.yml`
- In CI, one of the `cibuildwheel` Github actions will rebuild the stubs to a
temp location and verify that they match what has been committed to the repo.
This step ensures that if changes to the C++ source code and bindings results
in a change to the stubs, developers are notified of the need to regenerate
the stubs, so that changes can be reviewed and the rules in `generate_stubs.py`
can be updated, if necessary.
Note that if you can't (or don't want to) build the stubs locally, you can
download an artifact containing the wheel and `__init__.pyi` file from any job
that fails the stub validation.
Test Images
-----------
@@ -389,3 +460,7 @@ You do not need any of these packages in order to build or use
OpenImageIO. But if you are going to contribute to OpenImageIO
development, you probably want them, since it is required for executing
OpenImageIO's test suite (when you run "make test").
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
<!-- Copyright Contributors to the OpenImageIO Project. -->
- [User quick start](docs/QuickStart.md) is a quick example of using
OpenImageIO in Python, C++, and the command line.
- [Releases](https://github.com/AcademySoftwareFoundation/OpenImageIO/releases) -- see this for the latest release notes, also found in the [CHANGES.md](CHANGES.md) file.
👷 Building and installing OpenImageIO
--------------------------------------
@@ -118,35 +124,48 @@ it and potentially be able to answer your question quickly (more so than a GH
"issue"). For quick questions, you could also try the [ASWF
| master | :white_check_mark: :construction: ALL fixes immediately, but this is a branch under development with a frequently unstable ABI and occasionally unstable API. |
| 2.5.x | :white_check_mark: All fixes that can be backported without breaking ABI compatibility. New tagged releases monthly. |
| 2.4.x | :warning: Only receives occasional critical fixes, upon request, only if they can be easily backported. |
| <= 2.3.x | :x: No longer receiving patches of any kind. |
| main | :white_check_mark: :construction: ALL fixes immediately, but this is a branch under development with a frequently unstable ABI and occasionally unstable API. |
| 3.1.x | :white_check_mark: All fixes that can be backported without breaking ABI compatibility. New tagged releases monthly. |
| 3.0.x | :warning: Important fixes that can be easily backported without breaking ABI compatibility. New tagged releases as needed, and becoming less frequent over time. |
| <= 2.5.x | :x: No longer receiving patches of any kind. |
## Reporting a Vulnerability
If you think you've found a potential vulnerability in OpenImageIO, please
report it by emailing security@openimageio.org. Only the project administrators
have access to these messages. Include detailed steps to reproduce the issue,
and any other information that could aid an investigation. Our policy is to
respond to vulnerability reports within 14 days.
report it to the maintainers. Include detailed steps to reproduce the issue,
and any other information that could aid an investigation.
Our policy is to address critical security vulnerabilities rapidly and post
patches as quickly as possible.
The best way to report a vulnerability is to file a GitHub [security
If that is not possible, it is also fine to email your report to
security@openimageio.org. Only the project administrators have access to these
reports.
Our policy is to respond to vulnerability reports within 14 days, and to
address critical security vulnerabilities rapidly and post patches quickly,
usually at the next scheduled patch release at the beginning of every month.
For especially severe vulnerabilities, or when a flaw is believed to be
exploited in the wild, we will try to make a special patch release as soon as
possible.
## What do we consider a vulnerability?
A true vulnerability, by our definition, requires the following:
- It allows arbitrary code execution, privilege escalation, exfiltration of
secrets beyond network boundaries, destruction of data or equipment, or
similarly serious outcome.
- It can be plausibly exploited by an untrusted party through normal product
inputs (for example, a maliciously crafted input image).
We would like to hear about these through the confidential GitHub
vulnerability reporting mechanism described above.
The following are examples of problems that we DO NOT consider
vulnerabilities:
- A crash or program termination that does not directly present the
opportunity for further exploits. This includes hardening assertions that
terminate the program rather than allow undefined behavior.
- Out-of-memory errors, as long as the program terminates in a reasonable
amount of time. (But if a tiny crafted input can make it loop infinitely or
spin for minutes, then it might qualify as a DOS vulnerability. Use your
best judgment.)
- Misuse or incorrect use of the APIs. We consider all programs directly using
OIIO's APIs to be trusted.
- Flaws whose root cause lies in a dependency. These should be reported and
fixed upstream; the upstream project owns the CVE when one is warranted.
Please do not use the GitHub vulnerability reporting mechanism for these, and
instead file ordinary Issues and PRs. We do not support requesting a CVE for
these kinds of problems.
Users operating in an environment in which it might encounter untrusted inputs
that could be malicious, and are be running in a situation where real damage
could occur are expected to:
- Enable all the OpenImageIO build-time options for safety checks, hardening,
etc. A problem that only occurs when these are disabled is not considered a
vulnerability.
- Ensure that all the dependencies OIIO builds against are on their latest
patched versions, and not rely on the OS distribution default, or even
OIIO's own "dependency auto-build" scripts. A problem that depends on
having anything but the lastest dependencies is not an OIIO vulnerbility.
## Other security features
### Signed tags
Starting with OpenImageIO 3.0, we cryptographically sign release tags.
To verify a tag, you can use the `git tag -v` command, which will check
the signature against the public key that is included in the repository.
For example,
```bash
git tag -v v3.0.0.3
```
## Outstanding Security Issues
None known
## History of CVE Fixes
Most recent fixes listed first, more or less:
- CVE-2026-63638: Cineon invalid bit depth heap out-of-bounds write [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-9hxv-jvgr-3x8g) / [Fix: PR #5283](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5283) (fixed in 3.1.16.0)
- CVE-2026-63635: PSD RawColor invalid color mode causes global out-of-bounds read and allocation DoS [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-3c8w-9xvm-r6gf) / [Fix: PR #5282](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5282) (fixed in 3.1.16.0)
- CVE-2026-63635: PSD RawColor invalid color mode causes global out-of-bounds read and allocation DoS [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-3c8w-9xvm-r6gf) / [Fix: PR #5282, 3.1.16.0](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5282) (fixed in 3.1.16.0)
- CVE-2026-63420: PSD RawColor indexed image out-of-bounds read in `interleave_row` [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-x877-h4xx-5m5j) / [Fix: PR #5307](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5307) (fixed in 3.1.16.0)
- CVE-2026-63419: OpenImageIO IFF ZBUFFER tile read writes past caller tile buffer [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-w6wc-gcf4-5pj2) / [Fix: PR #5268](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5268) (fixed in 3.1.16.0)
- CVE-2026-59956: Heap-buffer-overread in IffInput::readimg() when ZBUFFER flag is set [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-hjfv-gvxc-qgvh) / [Fix: PR #5251](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5251) (fixed in 3.0.20.0, 3.1.15.0)
- CVE-2026-59181: Stack buffer overflow in OpenImageIO Cineon reader via unchecked numberOfElements [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-xh8r-vmqq-56pp) / [Fix: PR #5250](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5250) (fixed in 3.0.20.0, 3.1.15.0)
- CVE-2026-59156: Unbounded recursion in FITS header parser leads to stack overflow [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-xvwr-x6ch-v2fq) / [Fix: PR #5248](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5248) (fixed in 3.0.20.0, >= 3.1.15.0)
- CVE-2026-50291: Segmentation Fault in BmpInput::read_native_scanline / [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-q3c7-3225-66h7) / [Fix: PR #5030](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5030) (Fixed in 3.0.16.0, 3.1.11.0)
- CVE-2026-43909: Signed integer overflow in SwapRGBABytes loop index leads to out-of-bounds read/write in DPX ABGR decoder / [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-g267-j53j-5258) / [Fix: PR5170](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5170) (Fixed in 3.0.18.1, 3.1.13.1)
- CVE-2026-43908: Signed integer overflow in ConvertCbYCrYToRGB leads to heap out-of-bounds write in DPX 4:2:2 decoder / [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-2jr5-q49v-3858) / [Fix: PR5170](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5170) (Fixed in 3.0.18.1, 3.1.13.1)
- CVE-2026-43907: Integer overflow in QueryRGBBufferSizeInternal leads to heap out-of-bounds write in DPX decoder (kCbYCr and kABGR) / [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-cq46-hp4h-cvfr) / [Fix: PR5170](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5170) (Fixed in 3.0.18.1, 3.1.13.1)
- CVE-2026-43996: Integer wraparound in bounds check of decode_pixel leads to out-of-bounds read in TGA paletted image decoder / [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-mq8j-73c4-cr55) / [Fix: PR5165](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/5165) (Fixed in 3.2.0.1, 3.1.13.0, 3.0.18.0)
- CVE-2024-55194: Broken pgm had memory access error leading to heap overflow / [advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-583r-43f7-cw8w) / [#4559](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/4559) (Fixed in 3.0.2.0, 3.1.4.0)
- CVE-2024-40630: Fixed incorrect image size for certain HEIC files.
[advisory](https://github.com/AcademySoftwareFoundation/OpenImageIO/security/advisories/GHSA-jjm9-9m4m-c8p2) (Fixed in 2.5.13.1)
- CVE-2023-42295: Fix signed integer overflow when computing total number of pixels while reading BMP files. [#3948](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/3948) (by xiaoxiaoafeifei) (Fixed in 2.5.3.0/2.6.0.1)
- CVE-2023-36183: Heap-buffer-overflow while reading ICO files [#3872](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/3872) (by xiaoxiaoafeifei)
- TALOS-2022-1627, CVE-2022-41977: Guard TIFF reads against corrupt files with buffer overflows. [#3628](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/3628) (2.4.5/2.5.0.0)
- TALOS-2022-1626 / CVE-2022-41794, TALOS-2022-1632 / CVE-2022-41684, TALOS-2022-1636 / CVE-2022-41837: Exif (all formats that support it, TIFF/JPEG/PSD) fix bugs where corrupted Exif blocks could overrun memory. [#3627](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/3627) (2.4.5/2.5.0.0)
- TALOS-2022-1634 / CVE-2022-41838, TALOS-2022-1635 / CVE-2022-41999: Fix DDS reading crashes for cubemap files when a cube face was not present, and check for invalid bits per pixel. [#3625](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/3625) (2.4.5/2.5.0.0) #3625 (2.4.5/2.5.0.0)
- [x] libheif 1.11 (from 1.3), which supports many additional features of that format. [#4380](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/4380)
- [x] WebP 1.2 (from 0.6) which lets us use their exported CMake configs and retire FindWebP.cmake. [#4354](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/4354)
- [x] pybind11 2.6 or 2.7 (from 2.4). [#4297](https://github.com/AcademySoftwareFoundation/OpenImageIO/pull/4297)
- [] [#4156](https://github.com/AcademySoftwareFoundation/OpenImageIO/issues/4156) Eliminate the last few places where we use Boost and eliminate it as a dependency.
- [x] [#4158](https://github.com/AcademySoftwareFoundation/OpenImageIO/issues/4158) Eliminate the last few places where we use Boost and eliminate it as a dependency.
<br>
@@ -86,8 +87,8 @@ major release:
Regardless of OCIO availability, version, or contents of any configs, have universal support for the [canonical color spaces](https://github.com/AcademySoftwareFoundation/MaterialX/blob/main/documents/Specification/MaterialX.Specification.md#color-spaces-and-color-management-systems) that seem to be the common consensus of ACES, MaterialX, and USD.
- [ ] Preliminary Rust bindings for most of the OpenImageIO APIs.
- [] Python wheel construction so `pip install openimageio` will be an easy way
for users to install the whole banana.
- [x] Python wheel construction so `pip install openimageio` will be an easy way
for users to install the whole banana. [#4428](https://github.com/AcademySoftwareFoundation/OpenImageIO/issues/4428)
If there is something you think should be on the roadmap for the next major
release but is not, please open an issue or discussion to propose it, or
@@ -97,6 +98,6 @@ bring it up at a TSC meeting, or on the mail list or Slack channel.
## Parking
Here is where we will put things that definitely should be on the roadmap, but
that need not be completed in time for the fall 2024 release of OIIO 3.0.
that need not be completed in time for the fall 2026 release of OIIO 3.2.
Use of "AI coding assistants" is permitted on this project, with the following
guidelines and principles.
Summary of our core values:
- Human must always be in the loop, and is the responsible party for
the contents of a PR (including fully understanding and being able
to explain, defend, and modify it in response to review comments).
- Interact with the project and community yourself, not by agent.
- Disclose what tools you used and how. At a minimum, we require an
"Assisted-by: TOOL/MODEL" line in the commit comments and PR description.
- Don't waste maintainer's time with low quality PRs.
The long version:
### Human authorship and interaction with the project
**The human making a PR is considered the author** and is fully responsible
for the code they submit.
**No PRs should be submitted, reviewed/merged, or deployed without a human
fully in the loop.** You may use agents locally to help you write code (and
docs, tests, etc.), but you may not use agents to autonomously interact with
the project community. Submit your own PRs, write your own discussion
comments, respond to community members in your own words, approve or reject
code in review using your own brain.
Code autonomously submitted without an identifiable human author and
responsible party will be immediately closed, and the account associated with
the submission may be banned from future project participation.
**DO NOT "vibe code."** This term refers to a pure prompt-to-behavior cycle
where the human accepts un-inspected code based on the program behavior alone.
We don't want this. Code that must be maintained by the project should be
truly understood. Project code is ideally the result of discussion / pair
programming between the author and assistant, but even if the machine is doing
most of the work, all code should be approved and fully understood by the
author prior to PR submission.
### Professionalism and quality
**The usual high quality level of PRs should be maintained regardless of tools
used.** This includes not only code, but also the design, testing, PR
preparation, and the author's ability to explain and defend the code and
respond to questions and requests for changes. We never want to hear "but the
AI..." as an excuse for anything.
**PRs should always be [reviewed](CodeReview.md) and approved by someone other
than their author if at all possible.** This is especially true of code with
machine-generated components, because of the additional risk that the
submitter/operator may not fully understand what they did not write
themselves. This is part of the reason for AI tool disclosure -- to ensure
that another pair of eyes is on it. AI code review may be requested for
additional input, but the AI cannot give "approval", nor merge code itself.
Even when using a coding assistant to write the code, you are strongly
encouraged to write your own PR description, in your own words, if for no
other reason than to force yourself to understand the code well enough to
describe it to others.
### Disclosure
**Disclosure is required.** Patches that were written with the aid of AI
coding assistants must have, at a minimum, the following line in the commit
comment and PR description body:
Assisted-by: TOOL / MODEL
Ideally, to the extent practical, the PR description should also have a brief
description of how the tool was used, including the gist of key prompts or
summarizing the direction of the dialog. A full log is not necessary; a short
summary will do.
There are several reasons for this disclosure/documentation, even though it
will sometimes be inconvenient:
- Reproducibility: We want even automated work to be reproducible by others in
principle, just like a scientific paper.
- Education: We want to actively teach each other what tools, prompts, or
methods are successful at making high quality results for the project.
- Metrics: We would like the ability to look back and compare assisted vs
non-assisted code for things like defect rate (how often were new bugs
introduced or commits needed to be reverted), whether people are more likely
to write comprehensive tests when using assistants, etc.
- Compliance: Many key users and developers are in companies where they need
to disclose how AI tools were used in their work, and in other software that
they use. Let's make it easy for them.
- Insurance: Should future legal decisions radically change the IP status of
AI-generated code, or if particular models are implicated in copyright
violations, we want a way to find out which contributions might need to be
revisited in order to bring the project back into compliance.
### Intellectual Property
Regardless of how the code came to be -- from your head alone, from a friend,
from Stack Overflow, from a blog post, from reading other code bases, or from
a coding assistant -- we expect the terms of the
[DCO](https://developercertificate.org/) and CLA to apply, and for the author
to take reasonable care that code is not copied from a source with an
incompatible license.
You may find that your confidence about complying with the DCO+CLA depends on:
- If you provided a detailed specification and how much you edited or guided
the results yourself.
- Whether you are modifying or extending existing code versus writing large
amounts of entirely new code.
- The degree to which the generated code seems to fit into our structure and
idiomatic style, and therefore seems unlikely to be copied from elsewhere.
- Whether your tool has guardrails to prevent answers that are too similar to
existing code, for example as claimed by [Claude](https://privacy.claude.com/en/articles/10023638-why-am-i-receiving-an-output-blocked-by-content-filtering-policy-error) and [Copilot](https://docs.github.com/en/copilot/how-tos/manage-your-account/).
### Extractive submissions
**Maintainer time and attention are precious commodities**, and use of coding
assistants is not an excuse to submit poor PRs or to externalize costs onto
maintainers/reviewers (such as responsibility for understanding, testing, or
fixing PRs that the human author does not have a full understanding of).
Maintainers are free to take counter-measures against submitters of sub-par
PRs, or other violations of this policy, up to and including banning habitual
abusers from future participation in the project.
**We discourage use of AI tools to fix GitHub issues labeled as "good first
issue" or for "Dev Days" work.** Cultivating and educating new contributors is
important, and as such, we do not want people to swoop in and use automated
tools to trivially solve tasks that were curated specifically for somebody to
actually learn from. We expect each developer to implement at most one "good
first issue" per Dev Days occasion, and we encourage you to solve the problem
with your own brain to maximize the learning experience.
### Exceptions
This AI tool use policy is not meant to encompass cases such as:
- "Smart auto-complete", spell-checking, grammar checking, or other uses that
aren't really contributing substantively to authorship.
- Use of LLMs to explain code or learn about the codebase, answer basic
programming questions, help with background research, or audit the code
for bugs that are subsequently confirmed and fixed by people.
- Language translation for non-fluent English speakers or other accessibility
accommodations.
- Trivial or de-minimis fixes such as fixing a typo, obviously wrong variable
use, etc.
- Reviewing your own code for mistakes prior to submitting a PR (as long as it
isn't making the fixes for you).
### Creative contributions -- out of scope
This project repository contains code for tools, and their documentation. We
don't really directly host any artistically creative work here, so we have not
addressed additional policy issues that might come up in such a context.
If you are representing a different project that is using this document for
guidance or inspiration for your own policies, please be aware that you may
have needs that are not covered by these policies but deserve well-crafted
guidance.
### References and inspiration
Our policy has been informed and inspired by the following efforts in other communities:
- [LLVM AI Tool Policy](https://llvm.org/docs/AIToolPolicy.html)
- [Fedora Project policy on AI-assisted contributions](https://communityblog.fedoraproject.org/council-policy-proposal-policy-on-ai-assisted-contributions/)
- [Linux Foundation policy on Generative AI](https://www.linuxfoundation.org/legal/generative-ai)
- [Rust policy on rejecting burdensome PRs](https://github.com/rust-lang/compiler-team/issues/893)
- The METR paper [Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/)
- [GitHub Blog: Rethinking open source mentorship in the AI era](https://github.blog/open-source/maintainers/rethinking-open-source-mentorship-in-the-ai-era/)
Our release numbers are of the form `MAJOR.MINOR.PATCH.TWEAK`, and we follow
an almost-semantic-versioning scheme.
MAJOR changes are those that break backward-compatibility of the public APIs,
MAJOR releases are those that break backward-compatibility of the public APIs,
requiring existing application code to change their source code where it calls
OIIO API functions in order to compile and work correctly.
MINOR changes guarantee API backward-compatibility with prior releases that
MINOR releases guarantee API backward-compatibility with prior releases that
have the same major number, but can introduce ABI or link incompatibilities.
Applications using OIIO need to be recompiled to move to a new minor release,
but should not need to change their source code. Minor releases are allowed
to drop support for sufficiently old compilers and dependencies.
but should not need to change their source code (except, of course, to take
advantage of new features). Minor releases are allowed to drop support for
sufficiently old compilers and dependencies.
PATCH changes guarantee both API and ABI/link backward-compatibility with
PATCH releases guarantee both API and ABI/link backward-compatibility with
prior releases that have the same minor number. Features and new public API
calls may be added (thus deviating from strict semantic versioning), but only
if they preserve ABI/link compatibility. This means that standalone functions
@@ -32,7 +33,7 @@ guarantee that any compilers or dependencies supported for the first release
of a minor series will continue to be supported for all patches to that minor
series.
TWEAK changes are restricted to critical bug fixes or build breaks that do
TWEAK releases are restricted to critical bug fixes or build breaks that do
not alter public APIs in any way.
### Cadence of releases
@@ -51,27 +52,27 @@ Tweak (fully forward and backward compatible) releases are not regularly
scheduled, and only happen sporadically, when a critical fix is needed in
between scheduled patch releases.
### Things are different in "master"
### Things are different in "main"
The "master" branch is where ongoing development occurs without any
The "main" branch is where ongoing development occurs without any
compatibility guarantees. We don't always know if the next year's release from
master will turn out to be major (API breaking) or minor (only ABI breaking).
main will turn out to be major (API breaking) or minor (only ABI breaking).
Some studios or products need the most up-to-date enhancements, and thus tend
to build OIIO from the "master" branch rather than using tagged releases. For
these users, we try to set version numbers and occasionally tag in master, to
send a signal about which points in master seem safe to rely on various
to build OIIO from the "main" branch rather than using tagged releases. For
these users, we try to set version numbers and occasionally tag in main, to
send a signal about which points in main seem safe to rely on various
compatibility levels. However, the rules are slightly different than we use
for releases.
In the "master" branch:
In the "main" branch:
- TWEAK changes within "master" only guarantee ABI back-compatibility, but may
- TWEAK tags within "main" only guarantee ABI back-compatibility, but may
have additions or non-bug-fix behavior changes (akin to patch releases
within a release branch).
- PATCH changes within "master" are allowed to break ABI or API (changes that
would require minor or major releases, if they were not in "master").
- Beware of unmarked breaks in compatibility of commits in master that are
- PATCH tags within "main" are allowed to break ABI or API (changes that
would require minor or major releases, if they were not in "main").
- Beware of unmarked breaks in compatibility of commits in main that are
in between developer preview tags.
@@ -83,10 +84,10 @@ In the "master" branch:
(and thus the branch markers will move over time). Branch names obey the
following conventions:
- `master` is the area for arbitrary changes intended for the next year's
- `main` is the area for arbitrary changes intended for the next year's
major or minor release.
- `dev-1.2` are the areas for staging additions to the next month's patch
release for the designated minor version series.
- `dev-1.2` are the areas for staging additions that will become the next
tweak or patch release for the minor version series.
- `release` is a special branch marker that is always set to the latest tag of
the currently supported, stable release family. People can count on
"release" always being the latest stable release, whatever that may be at
@@ -101,7 +102,7 @@ conventions:
- `v1.2.3.4-alpha`, `v1.2.3.4-beta`, or `v1.2.3.4-RC1` are pre-releases for an
upcoming public stable release, at the alpha, beta, or release candidate
stages, respectively.
- `v1.2.3.4-dev` is a "developer preview" within "master." It is not an official
- `v1.2.3.4-dev` is a "developer preview" within "main." It is not an official
supported release, and while the numbering may indicate compatibility versus
other recent developer previews, they should not be assumed to be compatible
with any prior official stable releases.
@@ -118,7 +119,7 @@ in general should be ignored by anyone who didn't make the tag or branch.
### Soft freeze for monthly tweak releases:
We freely backport *safe*, non-ABI-breaking changes by cherry-picking them
from master to the supported branch continually throughout most of the month.
from main to the supported branch continually throughout most of the month.
In the seven days before the new month, we do a "soft freeze" where only
important bug fixes are backported. Hold on to any other patches you wish to
@@ -133,18 +134,18 @@ project during this soft freeze.
By mid-June (about 1-2 months before the beta of a new annual big release),
you should branch and mark it as an alpha.
At the tip of "master", create a new branch named `dev-MAJOR.MINOR` that will
At the tip of "main", create a new branch named `dev-MAJOR.MINOR` that will
be the staging area for the next release.
Customarily, at that point we add a commit to master (but not to the new
dev-MAJ.MIN branch) that bumps master's version to the next minor level, to
avoid any confusion between builds from master versus what will be the next
release branch. This starts the divergence between master and the release
Customarily, at that point we add a commit to main (but not to the new
dev-MAJ.MIN branch) that bumps main's version to the next minor level, to
avoid any confusion between builds from main versus what will be the next
release branch. This starts the divergence between main and the release
branch, and henceforth, any big or compatibility-breaking changes will only be
committed to master and not backported to the release branch. (Though we are
committed to main and not backported to the release branch. (Though we are
loose about this rule during the alpha period and allow continued breaking
changes in the alpha, but by the time we call it beta, we allow no more
breaking changes.) Also at this stage, the master branch's CHANGES.md should
breaking changes.) Also at this stage, the main branch's CHANGES.md should
have a heading added at the top for the *next* version.
@@ -155,7 +156,34 @@ have a heading added at the top for the *next* version.
pages), and that the [INSTALL.md](../INSTALL.md) file correctly documents
the version ranges that we expect to work and that we actively test.
2. Release notes: [CHANGES.md](../CHANGES.md) should be updated to reflect
2. Update deprecations: Deprecations tend to happen in stages, spread over a
few releases, taking steps forward depending on what kind of release is
occurring:
- Tweak releases (a.b.c.d -> a.b.c.D+1): should not introduce new
deprecations.
- Monthly patch releases (a.b.c.d -> a.b.C+1.0): advertise that a function
is deprecated in its comments, documentation, and release notes. Remember
that these releases are not allowed to break ABI/link compatibility.
- Annual minor releases (a.b.c.d -> a.B+1.0.0): functions previously
marked/documented as deprecated should receive deprecation warnings by
adding `OIIO_DEPRECATED` macros to function or class declarations where
possible, if that has not already been done. Use good judgment about
whether to add these warnings, or postpone to a later release, based on
the amount of disruption that they will likely cause, and how important
it is to rapidly discourage people from using the deprecated function.
For a minor release, we don't allow backwards-incompatible API changes,
so only *remove* a function if there is another function that will accept
the same call signature in the source code. It's allowed to break ABI for
minor releases, so it's ok/encouraged to make a deprecated function from
linked to inline, for example (which might allow it to be removed later
without an ABI change).
- Major releases (a.b.c.d -> A+1.0.0.0): API compatibility breaks are
allowed here, so this is the time to completely remove any functions that
have already had a full minor release where they had been given
deprecation warning macros.
3. Release notes: [CHANGES.md](../CHANGES.md) should be updated to reflect
all the new features and fixes. Looking at the notes from older releases
should provide a good example of what we're aiming for.
@@ -168,7 +196,7 @@ have a heading added at the top for the *next* version.
internals and developer goodies, testing/CI/ports, etc.) and ordered for
readability and relevance.
3. Ensure docs are up to date:
4. Ensure docs are up to date:
- [README.md](README.md): Actually read it! Make sure it all seems
accurate and up to date.
@@ -178,28 +206,28 @@ have a heading added at the top for the *next* version.
sure new contributors are listed.
- [SECURITY.md](SECURITY.md): Make sure it accurately reflects the
status of which branches get updates.
- Skim the [user documentation](https://openimageio.readthedocs.io/) to
- Skim the [user documentation](https://docs.openimageio.org) to
ensure it's building correctly and doesn't have any obvious errors,
especially the parts that describe new features.
4. Make sure the the top-level CMakeLists.txt file is updated:
5. Make sure the the top-level CMakeLists.txt file is updated:
- The `OpenImageIO_VERSION` should be correct.
- The `PROJECT_VERSION_RELEASE_TYPE` variable should be set to "alpha" or
"beta" if appropriate, "RC" for release candidate, or empty string `""`
for most supported releases, and "dev" only for the master branch or
for most supported releases, and "dev" only for the main branch or
developer previews.
- The `${PROJECT_NAME}_SUPPORTED_RELEASE` variable should be `ON` for any
release branch, `OFF` for master.
release branch, `OFF` for main.
5. In the https://github.com/AcademySoftwareFoundation/OpenImageIO-images project, create a branch
6. In the https://github.com/AcademySoftwareFoundation/OpenImageIO-images project, create a branch
`dev-x.y` for the major/minor branch, and in the main oiio repo, update
src/build-scripts/install_test_images.bash to specify `-b dev-x.y` in the
checkout of oiio-images to ensure CI tests are against the set of test
images corresponding to that major/minor release (just in the branch, not
in master!).
in main!).
6. Make sure everything passes the usual CI workflow. Also check the daily or
7. Make sure everything passes the usual CI workflow. Also check the daily or
weekly "analysis" workflows to make sure there aren't any important
warnings that should be fixed.
@@ -236,24 +264,40 @@ truly critical bugs or build breaks that users will encounter.
The following are the steps for making the release:
1. Edit the top-level CMakeLists.txt to remove any RC designation
(i.e., `PROJECT_VERSION_RELEASE_TYPE` should be set to `""`).
1. Double check the top-level CMakeLists.txt to ensure the right version
number and type of version designation (i.e.,
`PROJECT_VERSION_RELEASE_TYPE` should be set appropriately).
2. Edit CHANGES.md to reflect the correct date of the release and ensure it
includes any last-minute changes that were made during beta or release
candidate stages.
candidate stages. Please see the section "Generating release notes" below.
3. Push it to **your** GitHub, make sure it passes CI.
3. Push it to **your** GitHub fork, make sure it passes CI.
4. Tag the release: `git tag v1.2.3.4` (no more beta, RC, or dev suffix).
4. Tag the release with a signed tag:
$ git tag -s v1.2.3.4 # add a signed tag
$ git tag -v v1.2.3.4 # verify the tag
Be sure that the tag also reflects if it's an alpha (v1.2.3.4-alpha),
beta (v1.2.3.4-beta), release candidate (v1.2.3.4-RC1), or developer
preview (v1.2.3.4-dev). If it's a final release, just use the plain
version number (v1.2.3.4).
Note that the tag signing requires you to have a GPG key set up and
associated with your GitHub account. Please see [GitHub's signed tagging instructions](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-tags) for more information.
If you get errors messages "gpg: signing failed: Inappropriate ioctl for device", then the solution is to first:
$ export GPG_TTY=$(tty)
5. If this will now be the recommended stable release, move the `release`
branch marker to the same position.
6. Push it to GitHub: `git push OpenImageIO release --tags`
(This example assumes "OpenImageIO" is the name of the remote for the
sed -i 's,^mirrorlist=,#,; s,^#baseurl=http://mirror\.centos\.org/centos/$releasever,baseurl=https://vault.centos.org/7.9.2009,' /etc/yum.repos.d/CentOS-Base.repo
fi
# time time sudo yum install -y giflib giflib-devel || true
if[["${USE_OPENCV}" !="0"]];then
sudo yum install -y opencv opencv-devel && true
time sudo yum install -y opencv opencv-devel || true
fi
if[["${USE_FFMPEG}" !="0"]];then
sudo yum install -y ffmpeg ffmpeg-devel && true
time sudo dnf install -y ffmpeg ffmpeg-devel || true
fi
if[["${USE_FREETYPE:-1}" !="0"]];then
time sudo yum install -y freetype freetype-devel || true
fi
if[["${USE_LIBRAW:-0}" !="0"]];then
time sudo yum install -y LibRaw LibRaw-devel || true
fi
if[["${EXTRA_DEP_PACKAGES}" !=""]];then
time sudo yum install -y ${EXTRA_DEP_PACKAGES}
time sudo yum install -y ${EXTRA_DEP_PACKAGES}|| true
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Larry Gritz