From 1ab6444304192e0df37a9a6c56a8504e1bea0b96 Mon Sep 17 00:00:00 2001 From: Larry Gritz Date: Tue, 31 Jan 2023 00:51:20 -0800 Subject: [PATCH] Add color management to texture lookups This isn't 100% final -- I'm sure there are some corner cases I missed, maybe things that will eventually need some performance work, but enough works and passes tests that I'm confident in the API and so I want to get this merged and we can iterate on internals later. The use case is as follows: User or shader knows that a particular texture is in something other than the default working color space, this is not necessarily apparent from the file itself and so cannot be fully automatic, but it is presumed that it can be communicated in the texture call. We aim to be efficient, fully converting each tile upon input. Note that we are sweeping under the rug the fact that a non-default color space lookup is not mathematically correct with respect to the MIP-map downsizing math if that was not considered at texture creation time. But by converting texels as they are read, rather than the shader itself trying to do a color transformation on the final texture result, at least we are doing the right math for the filtering within and among MIP levels. The new interface consists of communicating the presumed working color space, finding the colortransform ID of a given from/to (with the "to" defaulting to the working space, if not specified), and then on each texture call, supplying that colortransform ID as one of the settings in TextureOpt (or TextureOptBatch). The outline of changes are as follows: * BREAKING CHANGE: TextureOpt and TextureOptBatch now contain an additional int `colortransformid`. The default, 0, is the old (and, frankly, preferred) behavior. Nonzero values indicate that we want the texels to go through a color transformation as the tiles are read into the cache. * BREAKING CHANGE: New public methods have been added to TextureSystem `get_colortransform_id(fromspace, tospace)` that return a transform ID given the from and to color spaces (ustring or ustringhash). This is the value that goes in the TextureOpt to request a color transformation on the pixels as they are read. * BREAKING CHANGE: IC::get_image_handle / TS::get_texture_handle now take an additional optinal parameter that is a TextureOpt*. This is currently unused, but is reserved for possible future use and/or for alternate TextureSystem implementations that choose a different implementation strategy wherein an entirely different TextureHandle may correspond to each color transformation. That's not how I've done it here, but it's a valid approach (and maybe one we'll switch to in the future). We presume that it's ill-defined to ask for a handle with one explicit color transformation, then use that handle with a TextureOpt that implies a different color transformation. Using those consistently ensure that either TS strategy will work equally well. * New TextureSystem attribute "colorspace" gives the working color space name (defaulting to "scene_linear"), and "colorconfig" can optionally point to a color config file (though this is not yet functional -- it just uses the default color config). * Internally, the tile cache hash has been adjusted so that the same tile in the disk file will appear as a separate tile in the cache for each color transform that is applied to it. Thus, there is no conflict if some different texture calls ask for different color space interpretations of the same texture file. (That said, it's not recommended to be throwing color spaces around willy nilly -- it still takes multiple cache entries, plus extra color conversion math as the tiles are read in.) * A new preprocessor symbol OIIO_TEXTURESYSTEM_SUPPORTS_COLORSPACE in texture.h allows client software can easily tell if this is a version new enough to support color management and has the additional fields in the texture option structures. Because there are breaking changes here to ABI, this is destined only for master (future 2.5) and will not be backported to OIIO 2.4. --- CHANGES.md | 20 ++++ CMakeLists.txt | 2 +- src/cmake/testing.cmake | 1 + src/include/OpenImageIO/imagecache.h | 32 ++++-- src/include/OpenImageIO/texture.h | 41 +++++-- src/libtexture/imagecache.cpp | 105 ++++++++++++++---- src/libtexture/imagecache_pvt.h | 39 ++++--- src/libtexture/texture3d.cpp | 5 +- src/libtexture/texture_pvt.h | 17 ++- src/libtexture/texturesys.cpp | 43 ++++++- src/libutil/ustring_test.cpp | 4 + src/testtex/testtex.cpp | 31 ++++-- testsuite/texture-colorspace/ref/cc.exr | Bin 0 -> 806 bytes testsuite/texture-colorspace/ref/nocc.exr | Bin 0 -> 709 bytes .../texture-colorspace/ref/out-batch.txt | 9 ++ testsuite/texture-colorspace/ref/out.txt | 9 ++ testsuite/texture-colorspace/run.py | 12 ++ 17 files changed, 287 insertions(+), 83 deletions(-) create mode 100644 testsuite/texture-colorspace/ref/cc.exr create mode 100644 testsuite/texture-colorspace/ref/nocc.exr create mode 100644 testsuite/texture-colorspace/ref/out-batch.txt create mode 100644 testsuite/texture-colorspace/ref/out.txt create mode 100755 testsuite/texture-colorspace/run.py diff --git a/CHANGES.md b/CHANGES.md index b97cec5d73..fbeddf0aa1 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -3,6 +3,26 @@ Release 2.5 (summer 2023?) -- compared to 2.4 New minimum dependencies and compatibility changes: New major features and public API changes: +* TextureSystem color management: #3761 (2.5.1.0) + - TextureOpt and TextureOptBatch have a new field, `colortransformid`, + which supplies an integer ID for a requested color space transformation + to be applied as texture tiles are read. The default value 0 means no + transformation because the texture is presumed to be in the working + color space (this is the old behavior, and most performant). Tiles from + the same texture file but using different color transformations are + allowed and will not interfere with each other in the cache. + - New `TextureSystem::get_colortransform_id(from, to)` maps from/to named + color spaces to a color transform ID that can be passed to texture + lookup calls. + - `ImageCache::get_image_handle` and `TextureSystem::get_texture_handle` + now take an optional `TextureOpt*` parameter that can supply additional + constraints (such as color transformation) that TS/IC implementations + may wish to split into separate handles. This is currently not used, but + is reserved so that the API doesn't need to be changed if we use it in + the future. + - texture.h defines symbol `OIIO_TEXTURESYSTEM_SUPPORTS_COLORSPACE` that + can be tested for existence to know if the new fields are in the + TextureOpt structure. * ImageBufAlgo additions: - A new flavor of `ociodisplay()` now contains an inverse parameter. #3650 (2.5.0.0) diff --git a/CMakeLists.txt b/CMakeLists.txt index 323a80bcf8..2b793773ee 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -4,7 +4,7 @@ cmake_minimum_required (VERSION 3.12) -set (OpenImageIO_VERSION "2.5.0.2") +set (OpenImageIO_VERSION "2.5.1.0") set (OpenImageIO_VERSION_OVERRIDE "" CACHE STRING "Version override (use with caution)!") mark_as_advanced (OpenImageIO_VERSION_OVERRIDE) diff --git a/src/cmake/testing.cmake b/src/cmake/testing.cmake index 9406c4ee06..deb78c03ab 100644 --- a/src/cmake/testing.cmake +++ b/src/cmake/testing.cmake @@ -186,6 +186,7 @@ macro (oiio_add_all_tests) texture-stats texture-threadtimes texture-env + texture-colorspace ) oiio_add_tests (${all_texture_tests}) # Duplicate texture tests with batch mode diff --git a/src/include/OpenImageIO/imagecache.h b/src/include/OpenImageIO/imagecache.h index 95c7fd2fb9..2e3de676be 100644 --- a/src/include/OpenImageIO/imagecache.h +++ b/src/include/OpenImageIO/imagecache.h @@ -28,6 +28,9 @@ OIIO_NAMESPACE_BEGIN +// Forward declaration +class TextureOpt; + namespace pvt { // Forward declaration class ImageCacheImpl; @@ -216,6 +219,11 @@ class OIIO_API ImageCache { /// enabled, this reduces the number of file opens, at the /// expense of not being able to open files if their format do /// not actually match their filename extension). Default: 0 + /// - `string colorspace` : + /// The working colorspace of the texture system. Default: none. + /// - `string colorconfig` : + /// Name of the OCIO config to use. Default: "" (meaning to use + /// the default color config). /// /// - `string options` /// This catch-all is simply a comma-separated list of @@ -467,18 +475,22 @@ class OIIO_API ImageCache { /// internals. typedef pvt::ImageCacheFile ImageHandle; - /// Retrieve an opaque handle for fast image lookups. The filename is - /// presumed to be UTF-8 encoded. The opaque `pointer thread_info` is - /// thread-specific information returned by `get_perthread_info()`. - /// Return NULL if something has gone horribly wrong. - virtual ImageHandle* get_image_handle (ustring filename, - Perthread *thread_info=NULL) = 0; + /// Retrieve an opaque handle for fast texture lookups, or nullptr upon + /// failure. The filename is presumed to be UTF-8 encoded. The `options`, + /// if not null, may be used to create a separate handle for certain + /// texture option choices. (Currently unused, but reserved for the future + /// or for alternate IC implementations.) The opaque pointer `thread_info` + /// is thread-specific information returned by `get_perthread_info()`. + virtual ImageHandle* get_image_handle(ustring filename, + Perthread* thread_info = nullptr, + const TextureOpt* options = nullptr) = 0; /// Get an ImageHandle using a UTF-16 encoded wstring filename. - ImageHandle* get_image_handle (const std::wstring& filename, - Perthread *thread_info=NULL) { - return get_image_handle (ustring(Strutil::utf16_to_utf8(filename)), - thread_info); + ImageHandle* get_image_handle(const std::wstring& filename, + Perthread* thread_info = nullptr, + const TextureOpt* options = nullptr) { + return get_image_handle(ustring(Strutil::utf16_to_utf8(filename)), + thread_info, options); } /// Return true if the image handle (previously returned by diff --git a/src/include/OpenImageIO/texture.h b/src/include/OpenImageIO/texture.h index ca4c2c3769..21ecf6c1d6 100644 --- a/src/include/OpenImageIO/texture.h +++ b/src/include/OpenImageIO/texture.h @@ -20,6 +20,7 @@ // Define symbols that let client applications determine if newly added // features are supported. #define OIIO_TEXTURESYSTEM_SUPPORTS_CLOSE 1 +#define OIIO_TEXTURESYSTEM_SUPPORTS_COLORSPACE 1 // Is the getattributetype() method present? (Added in 2.5) #define OIIO_TEXTURESYSTEM_SUPPORTS_GETATTRIBUTETYPE 1 @@ -232,6 +233,7 @@ class OIIO_API TextureOpt { fill(0.0f), missingcolor(nullptr), time(0.0f), rnd(-1.0f), samples(1), rwrap(WrapDefault), rblur(0.0f), rwidth(1.0f), + colortransformid(0), envlayout(0) { } @@ -264,6 +266,8 @@ class OIIO_API TextureOpt { float rblur; ///< Blur amount in the r direction float rwidth; ///< Multiplier for derivatives in r direction + int colortransformid; ///< Color space id of the texture + /// Utility: Return the Wrap enum corresponding to a wrap name: /// "default", "black", "clamp", "periodic", "mirror". static Wrap decode_wrapmode(const char* name) @@ -313,9 +317,7 @@ class OIIO_API TextureOptBatch { alignas(Tex::BatchAlign) float twidth[Tex::BatchWidth]; alignas(Tex::BatchAlign) float rwidth[Tex::BatchWidth]; // Note: rblur,rwidth only used for volumetric lookups -#if OIIO_VERSION_GREATER_EQUAL(2,4,0) alignas(Tex::BatchAlign) float rnd[Tex::BatchWidth]; -#endif // Options that must be the same for all points we're texturing at once int firstchannel = 0; ///< First channel of the lookup @@ -330,6 +332,7 @@ class OIIO_API TextureOptBatch { int conservative_filter = 1; ///< True: over-blur rather than alias float fill = 0.0f; ///< Fill value for missing channels const float *missingcolor = nullptr; ///< Color for missing texture + int colortransformid = 0; ///< Color space id of the texture private: // Options set INTERNALLY by libtexture after the options are passed @@ -551,6 +554,10 @@ class OIIO_API TextureSystem { /// - `int max_errors_per_file` : /// Limits how many errors to issue for each file. (default: /// 100) + /// - `string colorspace` : + /// The working colorspace of the texture system. + /// - `string colorconfig` : + /// Name of the OCIO config to use (default: ""). /// /// Texture-specific settings: /// - `matrix44 worldtocommon` / `matrix44 commontoworld` : @@ -788,16 +795,20 @@ class OIIO_API TextureSystem { class TextureHandle; /// Retrieve an opaque handle for fast texture lookups. The filename is - /// presumed to be UTF-8 encoded. The opaque pointer `thread_info` is - /// thread-specific information returned by `get_perthread_info()`. - /// Return nullptr if something has gone horribly wrong. - virtual TextureHandle* get_texture_handle (ustring filename, - Perthread *thread_info=nullptr) = 0; + /// presumed to be UTF-8 encoded. The `options`, if not null, may be used + /// to create a separate handle for certain texture option choices + /// (currently: the colorspace). The opaque pointer `thread_info` is + /// thread-specific information returned by `get_perthread_info()`. Return + /// nullptr if something has gone horribly wrong. + virtual TextureHandle* get_texture_handle(ustring filename, + Perthread* thread_info = nullptr, + const TextureOpt* options = nullptr) = 0; /// Get a TextureHandle using a UTF-16 encoded wstring filename. - TextureHandle* get_texture_handle (const std::wstring& filename, - Perthread *thread_info=nullptr) { - return get_texture_handle (ustring(Strutil::utf16_to_utf8(filename)), - thread_info); + TextureHandle* get_texture_handle(const std::wstring& filename, + Perthread* thread_info = nullptr, + const TextureOpt* options = nullptr) { + return get_texture_handle(ustring(Strutil::utf16_to_utf8(filename)), + thread_info, options); } /// Return true if the texture handle (previously returned by @@ -810,6 +821,14 @@ class OIIO_API TextureSystem { /// This method was added in OpenImageIO 2.3. virtual ustring filename_from_handle(TextureHandle* handle) = 0; + /// Retrieve an id for a color transformation by name. This ID can be used + /// as the value for TextureOpt::colortransformid. The returned value will + /// be -1 if either color space is unknown, and 0 for a null + /// transformation. + virtual int get_colortransform_id(ustring fromspace, + ustring tospace) const = 0; + virtual int get_colortransform_id(ustringhash fromspace, + ustringhash tospace) const = 0; /// @} /// @{ diff --git a/src/libtexture/imagecache.cpp b/src/libtexture/imagecache.cpp index da8380fa9a..178e8b26fe 100644 --- a/src/libtexture/imagecache.cpp +++ b/src/libtexture/imagecache.cpp @@ -11,10 +11,13 @@ #include #include + +#include #include #include #include #include +#include #include #include #include @@ -791,25 +794,24 @@ ImageCacheFile::init_from_spec() bool -ImageCacheFile::read_tile(ImageCachePerThreadInfo* thread_info, int subimage, - int miplevel, int x, int y, int z, int chbegin, - int chend, TypeDesc format, void* data) +ImageCacheFile::read_tile(ImageCachePerThreadInfo* thread_info, + const TileID& id, void* data) { - OIIO_DASSERT(chend > chbegin); + OIIO_DASSERT(id.chend() > id.chbegin()); // Mark if we ever use a mip level that's not the first + int miplevel = id.miplevel(); if (miplevel > 0) m_mipused = true; - // count how many times this mipmap level was read m_mipreadcount[miplevel]++; + int subimage = id.subimage(); SubimageInfo& subinfo(subimageinfo(subimage)); // Special case for un-MIP-mapped if (subinfo.unmipped && miplevel != 0) - return read_unmipped(thread_info, subimage, miplevel, x, y, z, chbegin, - chend, format, data); + return read_unmipped(thread_info, id, data); std::shared_ptr inp = open(thread_info); if (!inp) @@ -817,10 +819,16 @@ ImageCacheFile::read_tile(ImageCachePerThreadInfo* thread_info, int subimage, // Special case for untiled images -- need to do tile emulation if (subinfo.untiled) - return read_untiled(thread_info, inp.get(), subimage, miplevel, x, y, z, - chbegin, chend, format, data); + return read_untiled(thread_info, inp.get(), id, data); // Ordinary tiled + int x = id.x(); + int y = id.y(); + int z = id.z(); + int chbegin = id.chbegin(); + int chend = id.chend(); + TypeDesc format = id.file().datatype(subimage); + bool ok = true; const ImageSpec& spec(this->spec(subimage, miplevel)); for (int tries = 0; tries <= imagecache().failure_retries(); ++tries) { @@ -852,6 +860,19 @@ ImageCacheFile::read_tile(ImageCachePerThreadInfo* thread_info, int subimage, thread_info->m_stats.bytes_read += b; m_bytesread += b; ++m_tilesread; + if (id.colortransformid() > 0) { + // print("CONVERT id {} {},{} to cs {}\n", filename(), id.x(), id.y(), + // id.colortransformid()); + ImageBuf wrapper(ImageSpec(spec.tile_width, spec.tile_height, + spec.nchannels, format), + data); + ImageBufAlgo::colorconvert( + wrapper, wrapper, + ColorConfig::default_colorconfig().getColorSpaceNameByIndex( + (id.colortransformid() >> 16) - 1), + m_imagecache.colorspace(), true, string_view(), string_view(), + nullptr, ROI(), 1); + } } return ok; } @@ -860,9 +881,7 @@ ImageCacheFile::read_tile(ImageCachePerThreadInfo* thread_info, int subimage, bool ImageCacheFile::read_unmipped(ImageCachePerThreadInfo* thread_info, - int subimage, int miplevel, int x, int y, int z, - int chbegin, int chend, TypeDesc format, - void* data) + const TileID& id, void* data) { // We need a tile from an unmipmapped file, and it doesn't really // exist. So generate it out of thin air by interpolating pixels @@ -875,6 +894,15 @@ ImageCacheFile::read_unmipped(ImageCachePerThreadInfo* thread_info, // N.B. No need to lock the mutex, since this is only called // from read_tile, which already holds the lock. + int subimage = id.subimage(); + int miplevel = id.miplevel(); + int x = id.x(); + int y = id.y(); + // int z = id.z(); + int chbegin = id.chbegin(); + int chend = id.chend(); + TypeDesc format = id.file().datatype(id.subimage()); + // Figure out the size and strides for a single tile, make an ImageBuf // to hold it temporarily. const ImageSpec& spec(this->spec(subimage, miplevel)); @@ -888,7 +916,7 @@ ImageCacheFile::read_unmipped(ImageCachePerThreadInfo* thread_info, // Figure out the range of texels we need for this tile x -= spec.x; y -= spec.y; - z -= spec.z; + // z -= spec.z; int x0 = x - (x % spec.tile_width); int x1 = std::min(x0 + spec.tile_width - 1, spec.full_width - 1); int y0 = y - (y % spec.tile_height); @@ -956,10 +984,18 @@ ImageCacheFile::read_unmipped(ImageCachePerThreadInfo* thread_info, // of reading a "tile" from a file that's scanline-oriented. bool ImageCacheFile::read_untiled(ImageCachePerThreadInfo* thread_info, - ImageInput* inp, int subimage, int miplevel, int x, - int y, int z, int chbegin, int chend, - TypeDesc format, void* data) -{ + ImageInput* inp, const TileID& id, void* data) +{ + int subimage = id.subimage(); + int miplevel = id.miplevel(); + int x = id.x(); + int y = id.y(); + int z = id.z(); + int chbegin = id.chbegin(); + int chend = id.chend(); + TypeDesc format = id.file().datatype(id.subimage()); + int colortransformid = id.colortransformid(); + // Strides for a single tile const ImageSpec& spec(this->spec(subimage, miplevel)); int tw = spec.tile_width; @@ -1021,7 +1057,7 @@ ImageCacheFile::read_untiled(ImageCachePerThreadInfo* thread_info, // tile-row, so let's put it in the cache anyway so // it'll be there when asked for. TileID id(*this, subimage, miplevel, i + spec.x, y0, z, chbegin, - chend); + chend, colortransformid); if (!imagecache().tile_in_cache(id, thread_info)) { ImageCacheTileRef tile; tile = new ImageCacheTile(id, &buf[i * pixelsize], format, @@ -1520,10 +1556,7 @@ ImageCacheTile::read(ImageCachePerThreadInfo* thread_info) // Clear the end pad values so there aren't NaNs sucked up by simd loads memset(m_pixels.get() + size - OIIO_SIMD_MAX_SIZE_BYTES, 0, OIIO_SIMD_MAX_SIZE_BYTES); - m_valid = file.read_tile(thread_info, m_id.subimage(), m_id.miplevel(), - m_id.x(), m_id.y(), m_id.z(), m_id.chbegin(), - m_id.chend(), file.datatype(m_id.subimage()), - &m_pixels[0]); + m_valid = file.read_tile(thread_info, m_id, &m_pixels[0]); file.imagecache().incr_mem(size); if (m_valid) { ImageCacheFile::LevelInfo& lev( @@ -1636,6 +1669,7 @@ ImageCacheImpl::init() m_failure_retries = 0; m_latlong_y_up_default = true; m_Mw2c.makeIdentity(); + m_colorspace = ustring("scene_linear"); m_mem_used = 0; m_statslevel = 0; m_max_errors_per_file = 100; @@ -2196,8 +2230,23 @@ ImageCacheImpl::attribute(string_view name, TypeDesc type, const void* val) do_invalidate = true; } } else if (name == "substitute_image" && type == TypeDesc::STRING) { - m_substitute_image = ustring(*(const char**)val); - do_invalidate = true; + ustring uval(*(const char**)val); + if (uval != m_substitute_image) { + m_substitute_image = uval; + do_invalidate = true; + } + } else if (name == "colorconfig" && type == TypeDesc::STRING) { + ustring uval(*(const char**)val); + if (uval != m_colorconfigname) { + m_colorconfigname = uval; + do_invalidate = true; + } + } else if (name == "colorspace" && type == TypeDesc::STRING) { + ustring uval(*(const char**)val); + if (uval != m_colorspace) { + m_colorspace = uval; + do_invalidate = true; + } } else if (name == "max_mip_res" && type == TypeInt) { m_max_mip_res = *(const int*)val; do_invalidate = true; @@ -2339,6 +2388,14 @@ ImageCacheImpl::getattribute(string_view name, TypeDesc type, void* val) const *(const char**)val = m_substitute_image.c_str(); return true; } + if (name == "colorconfig" && type == TypeDesc::STRING) { + *(const char**)val = m_colorconfigname.c_str(); + return true; + } + if (name == "colorspace" && type == TypeDesc::STRING) { + *(const char**)val = m_colorspace.c_str(); + return true; + } if (name == "all_filenames" && type.basetype == TypeDesc::STRING && type.is_sized_array()) { ustring* names = (ustring*)val; diff --git a/src/libtexture/imagecache_pvt.h b/src/libtexture/imagecache_pvt.h index 2d1b40a598..e12c0afdbb 100644 --- a/src/libtexture/imagecache_pvt.h +++ b/src/libtexture/imagecache_pvt.h @@ -44,6 +44,7 @@ namespace pvt { using boost::thread_specific_ptr; +struct TileID; class ImageCacheImpl; class ImageCachePerThreadInfo; @@ -193,9 +194,8 @@ class OIIO_API ImageCacheFile final : public RefCnt { /// Load new data tile /// - bool read_tile(ImageCachePerThreadInfo* thread_info, int subimage, - int miplevel, int x, int y, int z, int chbegin, int chend, - TypeDesc format, void* data); + bool read_tile(ImageCachePerThreadInfo* thread_info, const TileID& id, + void* data); /// Mark the file as recently used. /// @@ -435,15 +435,13 @@ class OIIO_API ImageCacheFile final : public RefCnt { /// Preconditions: the ImageInput is already opened, and we already did /// a seek_subimage to the right subimage and MIP level. bool read_untiled(ImageCachePerThreadInfo* thread_info, ImageInput* inp, - int subimage, int miplevel, int x, int y, int z, - int chbegin, int chend, TypeDesc format, void* data); + const TileID& id, void* data); /// Load the requested tile, from a file that's not really MIPmapped. /// Preconditions: the ImageInput is already opened, and we already did /// a seek_subimage to the right subimage. - bool read_unmipped(ImageCachePerThreadInfo* thread_info, int subimage, - int miplevel, int x, int y, int z, int chbegin, - int chend, TypeDesc format, void* data); + bool read_unmipped(ImageCachePerThreadInfo* thread_info, const TileID& id, + void* data); // Initialize a bunch of fields based on the ImageSpec. // FIXME -- this is actually deeply flawed, many of these things only @@ -489,7 +487,7 @@ struct TileID { /// Initialize a TileID based on full elaboration of image file, /// subimage, and tile x,y,z indices. TileID(ImageCacheFile& file, int subimage, int miplevel, int x, int y, - int z, int chbegin, int chend) + int z, int chbegin, int chend, int colortransformid = 0) : m_x(x) , m_y(y) , m_z(z) @@ -497,6 +495,7 @@ struct TileID { , m_miplevel(miplevel) , m_chbegin(chbegin) , m_chend(chend) + , m_colortransformid(colortransformid) , m_file(&file) { if (chend < chbegin) { @@ -519,6 +518,7 @@ struct TileID { int chbegin() const { return m_chbegin; } int chend() const { return m_chend; } int nchannels() const { return m_chend - m_chbegin; } + int colortransformid() const { return m_colortransformid; } void x(int v) { m_x = v; } void y(int v) { m_y = v; } @@ -547,7 +547,8 @@ struct TileID { return (a.m_x == b.m_x && a.m_y == b.m_y && a.m_z == b.m_z && a.m_subimage == b.m_subimage && a.m_miplevel == b.m_miplevel && (a.m_file == b.m_file) && a.m_chbegin == b.m_chbegin - && a.m_chend == b.m_chend); + && a.m_chend == b.m_chend + && a.m_colortransformid == b.m_colortransformid); } /// Do the two ID's refer to the same tile? @@ -562,7 +563,8 @@ struct TileID { const uint64_t b = (uint64_t(m_z) << 32) + uint64_t(m_subimage); const uint64_t c = (uint64_t(m_miplevel) << 32) + (uint64_t(m_chbegin) << 16) + uint64_t(m_chend); - const uint64_t d = m_file->filename().hash(); + const uint64_t d = m_file->filename().hash() + + uint64_t(m_colortransformid); return fasthash::fasthash64({ a, b, c, d }); } @@ -575,8 +577,8 @@ struct TileID { { return (o << "{xyz=" << id.m_x << ',' << id.m_y << ',' << id.m_z << ", sub=" << id.m_subimage << ", mip=" << id.m_miplevel - << ", chans=[" << id.chbegin() << "," << id.chend() << ")" - << ' ' + << ", chans=[" << id.chbegin() << "," << id.chend() + << ", cs=" << id.colortransformid() << ") " << (id.m_file ? ustring("nofile") : id.m_file->filename()) << '}'); } @@ -586,6 +588,7 @@ struct TileID { int m_subimage; ///< subimage int m_miplevel; ///< MIP-map level short m_chbegin, m_chend; ///< Channel range + int m_colortransformid; ///< Colorspace id (0 == default) ImageCacheFile* m_file; ///< Which ImageCacheFile we refer to }; @@ -947,9 +950,9 @@ class ImageCacheImpl final : public ImageCache { ImageCachePerThreadInfo* thread_info, bool header_only = false); - ImageCacheFile* - get_image_handle(ustring filename, - ImageCachePerThreadInfo* thread_info = NULL) override + ImageCacheFile* get_image_handle(ustring filename, + ImageCachePerThreadInfo* thread_info, + const TextureOpt* options) override { if (!thread_info) thread_info = get_perthread_info(); @@ -1120,6 +1123,8 @@ class ImageCacheImpl final : public ImageCache { int max_mip_res() const noexcept { return m_max_mip_res; } + ustring colorspace() const noexcept { return m_colorspace; } + private: void init(); @@ -1178,6 +1183,8 @@ class ImageCacheImpl final : public ImageCache { Imath::M44f m_Mw2c; ///< world-to-"common" matrix Imath::M44f m_Mc2w; ///< common-to-world matrix ustring m_substitute_image; ///< Substitute this image for all others + ustring m_colorspace; ///< Working color space + ustring m_colorconfigname; ///< Filename of color config to use mutable FilenameMap m_files; ///< Map file names to ImageCacheFile's ustring m_file_sweep_name; ///< Sweeper for "clock" paging algorithm diff --git a/src/libtexture/texture3d.cpp b/src/libtexture/texture3d.cpp index 59c449d93d..573890e1c3 100644 --- a/src/libtexture/texture3d.cpp +++ b/src/libtexture/texture3d.cpp @@ -362,7 +362,8 @@ TextureSystemImpl::accum3d_sample_closest( int tile_t = (ttex - spec.y) % spec.tile_height; int tile_r = (rtex - spec.z) % spec.tile_depth; TileID id(texturefile, options.subimage, miplevel, stex - tile_s, - ttex - tile_t, rtex - tile_r, tile_chbegin, tile_chend); + ttex - tile_t, rtex - tile_r, tile_chbegin, tile_chend, + options.colortransformid); bool ok = find_tile(id, thread_info, true); if (!ok) error("{}", m_imagecache->geterror()); @@ -592,7 +593,7 @@ TextureSystemImpl::accum3d_sample_bilinear( tile_chend = options.firstchannel + actualchannels; } TileID id(texturefile, options.subimage, miplevel, 0, 0, 0, tile_chbegin, - tile_chend); + tile_chend, options.colortransformid); int startchan_in_tile = options.firstchannel - id.chbegin(); if (onetile && valid_storage.ivalid == all_valid) { diff --git a/src/libtexture/texture_pvt.h b/src/libtexture/texture_pvt.h index 3fd731accc..bb9c96fc37 100644 --- a/src/libtexture/texture_pvt.h +++ b/src/libtexture/texture_pvt.h @@ -114,13 +114,14 @@ class TextureSystemImpl final : public TextureSystem { m_imagecache->destroy_thread_info((ImageCachePerThreadInfo*)threadinfo); } - TextureHandle* get_texture_handle(ustring filename, - Perthread* thread) override + TextureHandle* + get_texture_handle(ustring filename, Perthread* thread, + const TextureOpt* options = nullptr) override { PerThreadInfo* thread_info = thread ? ((PerThreadInfo*)thread) : m_imagecache->get_perthread_info(); - return (TextureHandle*)find_texturefile(filename, thread_info); + return (TextureHandle*)find_texturefile(filename, thread_info, options); } bool good(TextureHandle* texture_handle) override @@ -134,6 +135,11 @@ class TextureSystemImpl final : public TextureSystem { : ustring(); } + int get_colortransform_id(ustring fromspace, + ustring tospace) const override; + int get_colortransform_id(ustringhash fromspace, + ustringhash tospace) const override; + bool texture(ustring filename, TextureOpt& options, float s, float t, float dsdx, float dtdx, float dsdy, float dtdy, int nchannels, float* result, float* dresultds = NULL, @@ -342,10 +348,13 @@ class TextureSystemImpl final : public TextureSystem { /// Find the TextureFile record for the named texture, or NULL if no /// such file can be found. - TextureFile* find_texturefile(ustring filename, PerThreadInfo* thread_info) + TextureFile* find_texturefile(ustring filename, PerThreadInfo* thread_info, + const TextureOpt* options = nullptr) { return m_imagecache->find_file(filename, thread_info); + // FIXME(colorconvert) } + TextureFile* verify_texturefile(TextureFile* texturefile, PerThreadInfo* thread_info) { diff --git a/src/libtexture/texturesys.cpp b/src/libtexture/texturesys.cpp index b0ed6f179d..7c604791ef 100644 --- a/src/libtexture/texturesys.cpp +++ b/src/libtexture/texturesys.cpp @@ -12,6 +12,7 @@ #include +#include #include #include #include @@ -563,6 +564,36 @@ TextureSystemImpl::resolve_filename(const std::string& filename) const +int +TextureSystemImpl::get_colortransform_id(ustring fromspace, + ustring tospace) const +{ + const ColorConfig& cc(ColorConfig::default_colorconfig()); + if (tospace.empty()) + tospace = m_imagecache->colorspace(); + if (fromspace.empty()) + return 0; // null transform + int from = cc.getColorSpaceIndex(fromspace); + int to = cc.getColorSpaceIndex(tospace); + if (from < 0 || to < 0) + return -1; // unknown color space + if (from == to || cc.equivalent(fromspace, tospace)) + return 0; // null transform + return ((from + 1) << 16) | (to + 1); // mash the indices together + // Note: we add 1 to the indices so that 0 can be the null transform +} + + + +int +TextureSystemImpl::get_colortransform_id(ustringhash fromspace, + ustringhash tospace) const +{ + return get_colortransform_id(ustring(fromspace), ustring(tospace)); +} + + + bool TextureSystemImpl::get_texture_info(ustring filename, int subimage, ustring dataname, TypeDesc datatype, @@ -799,7 +830,7 @@ TextureSystemImpl::get_texels(TextureHandle* texture_handle_, tile_chend = chbegin + actualchannels; } TileID tileid(*texfile, subimage, miplevel, 0, 0, 0, tile_chbegin, - tile_chend); + tile_chend, options.colortransformid); size_t formatchannelsize = format.size(); size_t formatpixelsize = nchannels * formatchannelsize; size_t scanlinesize = (xend - xbegin) * formatpixelsize; @@ -1186,7 +1217,8 @@ TextureSystemImpl::texture(TextureHandle* texture_handle_, options.twrap = TextureOpt::WrapPeriodicPow2; if (subinfo.is_constant_image && options.swrap != TextureOpt::WrapBlack - && options.twrap != TextureOpt::WrapBlack) { + && options.twrap != TextureOpt::WrapBlack + && options.colortransformid <= 0) { // Lookup of constant color texture, non-black wrap -- skip all the // hard stuff. for (int c = 0; c < actualchannels; ++c) @@ -1306,6 +1338,7 @@ TextureSystemImpl::texture(TextureHandle* texture_handle, opt.conservative_filter = options.conservative_filter; opt.fill = options.fill; opt.missingcolor = options.missingcolor; + opt.colortransformid = options.colortransformid; // rwrap not needed for 2D texture bool ok = true; @@ -2097,7 +2130,7 @@ TextureSystemImpl::sample_closest( tile_chend = options.firstchannel + actualchannels; } TileID id(texturefile, options.subimage, miplevel, 0, 0, 0, tile_chbegin, - tile_chend); + tile_chend, options.colortransformid); for (int sample = 0; sample < nsamples; ++sample) { float s = s_[sample], t = t_[sample]; float weight = weight_[sample]; @@ -2242,7 +2275,7 @@ TextureSystemImpl::sample_bilinear( tile_chend = options.firstchannel + actualchannels; } TileID id(texturefile, options.subimage, miplevel, 0, 0, 0, tile_chbegin, - tile_chend); + tile_chend, options.colortransformid); float nonfill = 0.0f; // The degree to which we DON'T need fill // N.B. What's up with "nofill"? We need to consider fill only when we // are inside the valid texture region. Outside, i.e. in the black wrap @@ -2599,7 +2632,7 @@ TextureSystemImpl::sample_bicubic( tile_chend = options.firstchannel + actualchannels; } TileID id(texturefile, options.subimage, miplevel, 0, 0, 0, tile_chbegin, - tile_chend); + tile_chend, options.colortransformid); int pixelsize = channelsize * id.nchannels(); imagesize_t firstchannel_offset_bytes = channelsize * (firstchannel - id.chbegin()); diff --git a/src/libutil/ustring_test.cpp b/src/libutil/ustring_test.cpp index 75dde77f91..d7f3257bca 100644 --- a/src/libutil/ustring_test.cpp +++ b/src/libutil/ustring_test.cpp @@ -139,6 +139,8 @@ test_ustring() // from_hash OIIO_CHECK_EQUAL(ustring::from_hash(foo.hash()), foo); + OIIO_CHECK_EQUAL(empty.hash(), 0); + OIIO_CHECK_EQUAL(ustring().hash(), 0); // make_unique, is_unique, from_unique const char* foostr = foo.c_str(); @@ -213,6 +215,8 @@ test_ustringhash() // from_hash OIIO_CHECK_EQUAL(ustringhash::from_hash(hfoo.hash()), hfoo); + OIIO_CHECK_EQUAL(ustringhash("").hash(), 0); + OIIO_CHECK_EQUAL(ustringhash().hash(), 0); // std::hash OIIO_CHECK_EQUAL(std::hash {}(hfoo), hfoo.hash()); diff --git a/src/testtex/testtex.cpp b/src/testtex/testtex.cpp index 252712c141..022c6f7631 100644 --- a/src/testtex/testtex.cpp +++ b/src/testtex/testtex.cpp @@ -47,16 +47,18 @@ static float width = 1; static float widthramp = 0; static float anisoaspect = 1.0; // anisotropic aspect ratio static std::string wrapmodes("periodic"); -static int anisomax = TextureOpt().anisotropic; -static int iters = 1; -static int autotile = 0; -static bool automip = false; -static bool dedup = true; -static bool test_construction = false; -static bool test_gettexels = false; -static bool test_getimagespec = false; -static bool filtertest = false; -static TextureSystem* texsys = NULL; +static std::string texcolorspace; +static int texcolortransform_id = 0; +static int anisomax = TextureOpt().anisotropic; +static int iters = 1; +static int autotile = 0; +static bool automip = false; +static bool dedup = true; +static bool test_construction = false; +static bool test_gettexels = false; +static bool test_getimagespec = false; +static bool filtertest = false; +static TextureSystem* texsys = NULL; static std::string searchpath; static bool batch = false; static bool nowarp = false; @@ -170,6 +172,8 @@ getargs(int argc, const char* argv[]) .help("Set fill value for missing channels"); ap.arg("--wrap %s:MODE", &wrapmodes) .help("Set wrap mode (default, black, clamp, periodic, mirror, overscan)"); + ap.arg("--texcolorspace %s:NAME", &texcolorspace) + .help("Set texture presumed color space"); ap.arg("--anisoaspect %f:ASPECT", &anisoaspect) .help("Set anisotropic ellipse aspect ratio for threadtimes tests (default: 2.0)"); ap.arg("--anisomax %d:MAX", &anisomax) @@ -303,6 +307,7 @@ initialize_opt(TextureOpt& opt) opt.subimage = subimage; else if (!subimagename.empty()) opt.subimagename = ustring(subimagename); + opt.colortransformid = texcolortransform_id; } @@ -332,6 +337,7 @@ initialize_opt(TextureOptBatch& opt) opt.subimage = subimage; else if (!subimagename.empty()) opt.subimagename = ustring(subimagename); + opt.colortransformid = texcolortransform_id; } @@ -1919,6 +1925,11 @@ main(int argc, const char* argv[]) texsys->attribute("gray_to_rgb", gray_to_rgb); texsys->attribute("flip_t", flip_t); texsys->attribute("stochastic", stochastic); + texcolortransform_id + = std::max(0, texsys->get_colortransform_id(ustring(texcolorspace), + ustring("scene_linear"))); + if (texcolortransform_id > 0) + print("Treating texture as if it is in colorspace {}\n", texcolorspace); if (test_construction) { Timer t; diff --git a/testsuite/texture-colorspace/ref/cc.exr b/testsuite/texture-colorspace/ref/cc.exr new file mode 100644 index 0000000000000000000000000000000000000000..158b6acd7c9ba8491e7d739f87b053b9a03b43c7 GIT binary patch literal 806 zcmXTZH)LdDU|>j2EO1FINo6Q5Day=CXAlMo85tNETNxNy85<}V8CjW_S{axpXC&t3 zrREefBxmGg7MC!10+l&3Fak*s0MQ`qgdyUNArgck!jPPwTTql*T%4Johr$LK$()i{ zk{F(umy%!3kd$9xl*u3f6omn6AOnO`GK&jx5-SO+%gM}3^)E_EEn+~hz@8|`tVqpq zEG|e*E(uC3$;@X+%gIkHVPFAU)L>tnT$Gxc2Ql6`HLnC{K$%e*0|(3L_C_hgq}xEE`%3-Sy3ndw z+K-EDU%T;2|H#VtF#Y`fIs0q=J^SWJpc7xa<4D~hJ8Kfd&A q_qvD7(awzffByLTadv;aJcRK+LqkH!Zi9))2ZR_{Qt#P3hX?=*EG44= literal 0 HcmV?d00001 diff --git a/testsuite/texture-colorspace/ref/nocc.exr b/testsuite/texture-colorspace/ref/nocc.exr new file mode 100644 index 0000000000000000000000000000000000000000..fa83e576da714c85140171097e2bd8a4b97acc5c GIT binary patch literal 709 zcmXTZH)LdDU|>j2EO1FINo6Q5Day=CXAlMo85tNETNxNy85<}V8CjW_S{axnXC&t3 zrREefBxmGg7MC!10+l&3Fak*s0MQ`qgdyUNArgck!jPPwTTql*T%4Johr$LK$()i{ zk{F(umy%!3kd$9xl*u3f6omnMAOnO`GK&jx5-SO+%gM}3^)E_EEn+~hz@8|`tVqpq zEG|e*E(uC3$;@X+%gIkHVPFAU)L>tnT$Gxc2Ql6`HLnC{K$%e*0|(3uK)l5 literal 0 HcmV?d00001 diff --git a/testsuite/texture-colorspace/ref/out-batch.txt b/testsuite/texture-colorspace/ref/out-batch.txt new file mode 100644 index 0000000000..f7aebc07b0 --- /dev/null +++ b/testsuite/texture-colorspace/ref/out-batch.txt @@ -0,0 +1,9 @@ +Created texture system +Testing BATCHED 2d texture grey.exr, output = nocc.exr +Created texture system +Treating texture as if it is in colorspace sRGB +Testing BATCHED 2d texture grey.exr, output = cc.exr +Comparing "nocc.exr" and "ref/nocc.exr" +PASS +Comparing "cc.exr" and "ref/cc.exr" +PASS diff --git a/testsuite/texture-colorspace/ref/out.txt b/testsuite/texture-colorspace/ref/out.txt new file mode 100644 index 0000000000..16a67f52a8 --- /dev/null +++ b/testsuite/texture-colorspace/ref/out.txt @@ -0,0 +1,9 @@ +Created texture system +Testing 2d texture grey.exr, output = nocc.exr +Created texture system +Treating texture as if it is in colorspace sRGB +Testing 2d texture grey.exr, output = cc.exr +Comparing "nocc.exr" and "ref/nocc.exr" +PASS +Comparing "cc.exr" and "ref/cc.exr" +PASS diff --git a/testsuite/texture-colorspace/run.py b/testsuite/texture-colorspace/run.py new file mode 100755 index 0000000000..521f8bc5fd --- /dev/null +++ b/testsuite/texture-colorspace/run.py @@ -0,0 +1,12 @@ +#!/usr/bin/env python + +# This test just maps a 50% grey texture, once with default settings, and once +# declaring that it thinks the texture is in sRGB texture space. + +# Note: we deliberately make the two output images different sizes so that +# they can't match against each others' ref images. + +command += oiiotool ("-pattern constant:color=0.5,0.5,0.5 64x64 3 -d uint8 -otex grey.exr") +command += testtex_command ("-res 64 64 --no-gettextureinfo --nowarp grey.exr -o nocc.exr") +command += testtex_command ("-res 60 60 --no-gettextureinfo --nowarp --texcolorspace sRGB grey.exr -o cc.exr") +outputs = [ "nocc.exr", "cc.exr", "out.txt" ]