Add color management to texture lookups

This isn't 100% final -- I'm sure there are some corner cases I
missed, maybe things that will eventually need some performance work,
but enough works and passes tests that I'm confident in the API and so
I want to get this merged and we can iterate on internals later.

The use case is as follows: User or shader knows that a particular
texture is in something other than the default working color space,
this is not necessarily apparent from the file itself and so cannot be
fully automatic, but it is presumed that it can be communicated in the
texture call. We aim to be efficient, fully converting each tile upon
input. Note that we are sweeping under the rug the fact that a
non-default color space lookup is not mathematically correct with
respect to the MIP-map downsizing math if that was not considered at
texture creation time. But by converting texels as they are read,
rather than the shader itself trying to do a color transformation on
the final texture result, at least we are doing the right math for the
filtering within and among MIP levels.

The new interface consists of communicating the presumed working color
space, finding the colortransform ID of a given from/to (with the "to"
defaulting to the working space, if not specified), and then on each
texture call, supplying that colortransform ID as one of the settings
in TextureOpt (or TextureOptBatch).

The outline of changes are as follows:

* BREAKING CHANGE: TextureOpt and TextureOptBatch now contain an
  additional int `colortransformid`. The default, 0, is the old (and,
  frankly, preferred) behavior. Nonzero values indicate that we want
  the texels to go through a color transformation as the tiles are
  read into the cache.

* BREAKING CHANGE: New public methods have been added to TextureSystem
  `get_colortransform_id(fromspace, tospace)` that return a transform
  ID given the from and to color spaces (ustring or ustringhash). This
  is the value that goes in the TextureOpt to request a color
  transformation on the pixels as they are read.

* BREAKING CHANGE: IC::get_image_handle / TS::get_texture_handle now
  take an additional optinal parameter that is a TextureOpt*. This is
  currently unused, but is reserved for possible future use and/or for
  alternate TextureSystem implementations that choose a different
  implementation strategy wherein an entirely different TextureHandle
  may correspond to each color transformation. That's not how I've
  done it here, but it's a valid approach (and maybe one we'll switch
  to in the future). We presume that it's ill-defined to ask for a
  handle with one explicit color transformation, then use that handle
  with a TextureOpt that implies a different color
  transformation. Using those consistently ensure that either TS
  strategy will work equally well.

* New TextureSystem attribute "colorspace" gives the working color
  space name (defaulting to "scene_linear"), and "colorconfig" can
  optionally point to a color config file (though this is not yet
  functional -- it just uses the default color config).

* Internally, the tile cache hash has been adjusted so that the same
  tile in the disk file will appear as a separate tile in the cache
  for each color transform that is applied to it. Thus, there is no
  conflict if some different texture calls ask for different color
  space interpretations of the same texture file. (That said, it's not
  recommended to be throwing color spaces around willy nilly -- it
  still takes multiple cache entries, plus extra color conversion math
  as the tiles are read in.)

* A new preprocessor symbol OIIO_TEXTURESYSTEM_SUPPORTS_COLORSPACE in
  texture.h allows client software can easily tell if this is a
  version new enough to support color management and has the additional
  fields in the texture option structures.

Because there are breaking changes here to ABI, this is destined only
for master (future 2.5) and will not be backported to OIIO 2.4.

Author: lgritz

CHANGES.md

@@ -3,6 +3,26 @@ Release 2.5 (summer 2023?) -- compared to 2.4
 New minimum dependencies and compatibility changes:
 
 New major features and public API changes:
+* TextureSystem color management: #3761 (2.5.1.0)
+    - TextureOpt and TextureOptBatch have a new field, `colortransformid`,
+      which supplies an integer ID for a requested color space transformation
+      to be applied as texture tiles are read. The default value 0 means no
+      transformation because the texture is presumed to be in the working
+      color space (this is the old behavior, and most performant). Tiles from
+      the same texture file but using different color transformations are
+      allowed and will not interfere with each other in the cache.
+    - New `TextureSystem::get_colortransform_id(from, to)` maps from/to named
+      color spaces to a color transform ID that can be passed to texture
+      lookup calls.
+    - `ImageCache::get_image_handle` and `TextureSystem::get_texture_handle`
+      now take an optional `TextureOpt*` parameter that can supply additional
+      constraints (such as color transformation) that TS/IC implementations
+      may wish to split into separate handles. This is currently not used, but
+      is reserved so that the API doesn't need to be changed if we use it in
+      the future.
+    - texture.h defines symbol `OIIO_TEXTURESYSTEM_SUPPORTS_COLORSPACE` that
+      can be tested for existence to know if the new fields are in the
+      TextureOpt structure.
 * ImageBufAlgo additions:
     - A new flavor of `ociodisplay()` now contains an inverse parameter.
       #3650 (2.5.0.0)

CMakeLists.txt

@@ -4,7 +4,7 @@
 
 cmake_minimum_required (VERSION 3.12)
 
-set (OpenImageIO_VERSION "2.5.0.2")
+set (OpenImageIO_VERSION "2.5.1.0")
 set (OpenImageIO_VERSION_OVERRIDE "" CACHE STRING
      "Version override (use with caution)!")
 mark_as_advanced (OpenImageIO_VERSION_OVERRIDE)

src/cmake/testing.cmake

@@ -186,6 +186,7 @@ macro (oiio_add_all_tests)
                     texture-stats
                     texture-threadtimes
                     texture-env
+                    texture-colorspace
                    )
     oiio_add_tests (${all_texture_tests})
     # Duplicate texture tests with batch mode

src/include/OpenImageIO/imagecache.h

@@ -28,6 +28,9 @@
 
 OIIO_NAMESPACE_BEGIN
 
+// Forward declaration
+class TextureOpt;
+
 namespace pvt {
 // Forward declaration
 class ImageCacheImpl;
@@ -216,6 +219,11 @@ class OIIO_API ImageCache {
     ///           enabled, this reduces the number of file opens, at the
     ///           expense of not being able to open files if their format do
     ///           not actually match their filename extension). Default: 0
+    /// - `string colorspace` :
+    ///           The working colorspace of the texture system. Default: none.
+    /// - `string colorconfig` :
+    ///           Name of the OCIO config to use. Default: "" (meaning to use
+    ///           the default color config).
     ///
     /// - `string options`
     ///           This catch-all is simply a comma-separated list of
@@ -467,6 +475,25 @@ class OIIO_API ImageCache {
     /// internals.
     typedef pvt::ImageCacheFile ImageHandle;
 
+#if OIIO_VERSION_GREATER_EQUAL(2, 5, 1)
+    /// Retrieve an opaque handle for fast texture lookups.  The filename is
+    /// presumed to be UTF-8 encoded. The `options`, if not null, may be used
+    /// to create a separate handle for certain texture option choices
+    /// (currently: the colorspace). The opaque pointer `thread_info` is
+    /// thread-specific information returned by `get_perthread_info()`. Return
+    /// nullptr if something has gone horribly wrong.
+    virtual ImageHandle* get_image_handle(ustring filename,
+                                      Perthread* thread_info = nullptr,
+                                      const TextureOpt* options = nullptr) = 0;
+
+    /// Get an ImageHandle using a UTF-16 encoded wstring filename.
+    ImageHandle* get_image_handle(const std::wstring& filename,
+                                  Perthread* thread_info = nullptr,
+                                  const TextureOpt* options = nullptr) {
+        return get_image_handle(ustring(Strutil::utf16_to_utf8(filename)),
+                                thread_info, options);
+    }
+#else
     /// Retrieve an opaque handle for fast image lookups. The filename is
     /// presumed to be UTF-8 encoded. The opaque `pointer thread_info` is
     /// thread-specific information returned by `get_perthread_info()`.
@@ -480,6 +507,7 @@ class OIIO_API ImageCache {
         return get_image_handle (ustring(Strutil::utf16_to_utf8(filename)),
                                  thread_info);
     }
+#endif
 
     /// Return true if the image handle (previously returned by
     /// `get_image_handle()`) is a valid image that can be subsequently read.

src/include/OpenImageIO/texture.h

@@ -20,6 +20,7 @@
 // Define symbols that let client applications determine if newly added
 // features are supported.
 #define OIIO_TEXTURESYSTEM_SUPPORTS_CLOSE 1
+#define OIIO_TEXTURESYSTEM_SUPPORTS_COLORSPACE 1
 
 // Is the getattributetype() method present? (Added in 2.5)
 #define OIIO_TEXTURESYSTEM_SUPPORTS_GETATTRIBUTETYPE 1
@@ -232,6 +233,7 @@ class OIIO_API TextureOpt {
         fill(0.0f), missingcolor(nullptr),
         time(0.0f), rnd(-1.0f), samples(1),
         rwrap(WrapDefault), rblur(0.0f), rwidth(1.0f),
+        colortransformid(0),
         envlayout(0)
     { }
 
@@ -264,6 +266,8 @@ class OIIO_API TextureOpt {
     float rblur;   ///< Blur amount in the r direction
     float rwidth;  ///< Multiplier for derivatives in r direction
 
+    int colortransformid;       ///< Color space id of the texture
+
     /// Utility: Return the Wrap enum corresponding to a wrap name:
     /// "default", "black", "clamp", "periodic", "mirror".
     static Wrap decode_wrapmode(const char* name)
@@ -313,9 +317,7 @@ class OIIO_API TextureOptBatch {
     alignas(Tex::BatchAlign) float twidth[Tex::BatchWidth];
     alignas(Tex::BatchAlign) float rwidth[Tex::BatchWidth];
     // Note: rblur,rwidth only used for volumetric lookups
-#if OIIO_VERSION_GREATER_EQUAL(2,4,0)
     alignas(Tex::BatchAlign) float rnd[Tex::BatchWidth];
-#endif
 
     // Options that must be the same for all points we're texturing at once
     int firstchannel = 0;                 ///< First channel of the lookup
@@ -330,6 +332,7 @@ class OIIO_API TextureOptBatch {
     int conservative_filter = 1;          ///< True: over-blur rather than alias
     float fill = 0.0f;                    ///< Fill value for missing channels
     const float *missingcolor = nullptr;  ///< Color for missing texture
+    int colortransformid = 0;             ///< Color space id of the texture
 
 private:
     // Options set INTERNALLY by libtexture after the options are passed
@@ -551,6 +554,10 @@ class OIIO_API TextureSystem {
     /// - `int max_errors_per_file` :
     ///             Limits how many errors to issue for each file. (default:
     ///             100)
+    /// - `string colorspace` :
+    ///             The working colorspace of the texture system.
+    /// - `string colorconfig` :
+    ///             Name of the OCIO config to use (default: "").
     ///
     /// Texture-specific settings:
     /// - `matrix44 worldtocommon` / `matrix44 commontoworld` :
@@ -788,16 +795,20 @@ class OIIO_API TextureSystem {
     class TextureHandle;
 
     /// Retrieve an opaque handle for fast texture lookups.  The filename is
-    /// presumed to be UTF-8 encoded. The opaque pointer `thread_info` is
-    /// thread-specific information returned by `get_perthread_info()`.
-    /// Return nullptr if something has gone horribly wrong.
-    virtual TextureHandle* get_texture_handle (ustring filename,
-                                            Perthread *thread_info=nullptr) = 0;
+    /// presumed to be UTF-8 encoded. The `options`, if not null, may be used
+    /// to create a separate handle for certain texture option choices
+    /// (currently: the colorspace). The opaque pointer `thread_info` is
+    /// thread-specific information returned by `get_perthread_info()`. Return
+    /// nullptr if something has gone horribly wrong.
+    virtual TextureHandle* get_texture_handle(ustring filename,
+                                      Perthread* thread_info = nullptr,
+                                      const TextureOpt* options = nullptr) = 0;
     /// Get a TextureHandle using a UTF-16 encoded wstring filename.
-    TextureHandle* get_texture_handle (const std::wstring& filename,
-                                       Perthread *thread_info=nullptr) {
-        return get_texture_handle (ustring(Strutil::utf16_to_utf8(filename)),
-                                   thread_info);
+    TextureHandle* get_texture_handle(const std::wstring& filename,
+                                      Perthread* thread_info = nullptr,
+                                      const TextureOpt* options = nullptr) {
+        return get_texture_handle(ustring(Strutil::utf16_to_utf8(filename)),
+                                  thread_info, options);
     }
 
     /// Return true if the texture handle (previously returned by
@@ -810,6 +821,14 @@ class OIIO_API TextureSystem {
     /// This method was added in OpenImageIO 2.3.
     virtual ustring filename_from_handle(TextureHandle* handle) = 0;
 
+    /// Retrieve an id for a color transformation by name. This ID can be used
+    /// as the value for TextureOpt::colortransformid. The returned value will
+    /// be -1 if either color space is unknown, and 0 for a null
+    /// transformation.
+    virtual int get_colortransform_id(ustring fromspace,
+                                      ustring tospace) const = 0;
+    virtual int get_colortransform_id(ustringhash fromspace,
+                                      ustringhash tospace) const = 0;
     /// @}
 
     /// @{