Stochastic mipmap interpolation #3127
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Suggested by Luke Emrose, the idea here is that we introduce two new MIP interpolation modes: StochasticTrilinear and StochasticAniso. These work similarly to Trilinear and Aniso, respectively, except that instead of sampling both of the bracketing MIP levels and blending between them, in stochastic mode we use a random variate to select ONE of the two MIP levels with probability proportional to the computed blending weight. If these mip modes are used, the caller needs to pass a well-stratified value in the new "rnd" field of TextureOpt. Also note that the method only really makes sense in the context of a renderer that is using many samples per pixel. As a single texture lookup, it looks horrible and noisy, as you can see by the testshade-based tests we do of this, which are 1spp and looks pretty rough. My benchmarks (in a hard stress test of the texture system using `testtex --threadtimes 8`) show that this approach has speedups of 35-40% when using between 1-4 threads, dropping to more like 25-30% speedup as threads are in the 12-32 range (that makes sense, it's doing the same amount of total I/O). In this first implementation, in order to preserve ABI compatibility if we decide to backport this feature to the release branch, TextureOpt::rnd is added as a union co-member with the unused "bias" field. For TextureOptBatch, we had to add it outright, but it's guarded by an `#if` and will not be backported to 2.3. (We may change our minds and not backport it at all, I just wanted to leave the option open with this initial patch. Also, there will probably be additional further ABI-breaking changes coming later that will definitely not be backported.) In this first phase, we only use the stochastic component to turn the 2-MIP-level lookup into one level lookup. It has been further suggested that we could apply stochastic techniques to anisotropic filtering (instead of many samples spaced across the major axis of the filter footprint, take one sample distributed across the axis), and even for the ordinary bilinear or bicubic within each level (instead of 4 or 16 weighted texels, just retrieve one texel chosen with the weights as probabilities). But these deserve to be benchmarked and have their quality evaluated separtely and deliberately, and so will be relegated to a later investigation and implementation.
etheory
approved these changes
Oct 13, 2021
There was a problem hiding this comment.
@lgritz this is perfect, thank you! This is exactly how we implemented this on our end, so I'm super happy to test this. I'll try and get this into our renderer branch against the latest OIIO and see how it performs. Thanks again, this is awesome!
|
I'm still in the middle of the OSL side. The simplest thing to do is to add a random variate to the shaderglobals. (Actually, maybe two because we are contemplating using them to stochastically choose closure nodes as well!) This is an ABI break, but maybe it's time for that. Speaking of which, as I was in this code, I saw SO MANY OTHER THINGS to improve, many of which make me want to rearrange a lot about the TextureOpt struct. Do you think it's reasonable to just throw up our hands and say that all of this is going to be a 2.4 feature that has a number of things that change ABI and struct layout? Or should we bend over backwards (and not do it as cleanly) in the interest of allowing at least part of it to be backported to 2.3? |
@lgritz it won't affect us as we're still not (yet....) using the mainline OSL anyway (I've still had no time to test the output placement changes sorry....). Whatever you do I can adapt until our OSL update (breaking OIIO ABI will have no effect for us at all). I'm sure other people would therefore have a stronger feeling on this. |
lgritz
added a commit
to lgritz/OpenImageIO
that referenced
this pull request
Nov 7, 2021
Suggested by Luke Emrose, the idea here is that we introduce two new MIP interpolation modes: StochasticTrilinear and StochasticAniso. These work similarly to Trilinear and Aniso, respectively, except that instead of sampling both of the bracketing MIP levels and blending between them, in stochastic mode we use a random variate to select ONE of the two MIP levels with probability proportional to the computed blending weight. If these mip modes are used, the caller needs to pass a well-stratified value in the new "rnd" field of TextureOpt. Also note that the method only really makes sense in the context of a renderer that is using many samples per pixel. As a single texture lookup, it looks horrible and noisy, as you can see by the testshade-based tests we do of this, which are 1spp and looks pretty rough. My benchmarks (in a hard stress test of the texture system using `testtex --threadtimes 8`) show that this approach has speedups of 35-40% when using between 1-4 threads, dropping to more like 25-30% speedup as threads are in the 12-32 range (that makes sense, it's doing the same amount of total I/O). In this first implementation, in order to preserve ABI compatibility if we decide to backport this feature to the release branch, TextureOpt::rnd is added as a union co-member with the unused "bias" field. For TextureOptBatch, we had to add it outright, but it's guarded by an `#if` and will not be backported to 2.3. (We may change our minds and not backport it at all, I just wanted to leave the option open with this initial patch. Also, there will probably be additional further ABI-breaking changes coming later that will definitely not be backported.) In this first phase, we only use the stochastic component to turn the 2-MIP-level lookup into one level lookup. It has been further suggested that we could apply stochastic techniques to anisotropic filtering (instead of many samples spaced across the major axis of the filter footprint, take one sample distributed across the axis), and even for the ordinary bilinear or bicubic within each level (instead of 4 or 16 weighted texels, just retrieve one texel chosen with the weights as probabilities). But these deserve to be benchmarked and have their quality evaluated separtely and deliberately, and so will be relegated to a later investigation and implementation.
lgritz
added a commit
to lgritz/OpenImageIO
that referenced
this pull request
Nov 7, 2021
Suggested by Luke Emrose, the idea here is that we introduce two new MIP interpolation modes: StochasticTrilinear and StochasticAniso. These work similarly to Trilinear and Aniso, respectively, except that instead of sampling both of the bracketing MIP levels and blending between them, in stochastic mode we use a random variate to select ONE of the two MIP levels with probability proportional to the computed blending weight. If these mip modes are used, the caller needs to pass a well-stratified value in the new "rnd" field of TextureOpt. Also note that the method only really makes sense in the context of a renderer that is using many samples per pixel. As a single texture lookup, it looks horrible and noisy, as you can see by the testshade-based tests we do of this, which are 1spp and looks pretty rough. My benchmarks (in a hard stress test of the texture system using `testtex --threadtimes 8`) show that this approach has speedups of 35-40% when using between 1-4 threads, dropping to more like 25-30% speedup as threads are in the 12-32 range (that makes sense, it's doing the same amount of total I/O). In this first implementation, in order to preserve ABI compatibility if we decide to backport this feature to the release branch, TextureOpt::rnd is added as a union co-member with the unused "bias" field. For TextureOptBatch, we had to add it outright, but it's guarded by an `#if` and will not be backported to 2.3. (We may change our minds and not backport it at all, I just wanted to leave the option open with this initial patch. Also, there will probably be additional further ABI-breaking changes coming later that will definitely not be backported.) In this first phase, we only use the stochastic component to turn the 2-MIP-level lookup into one level lookup. It has been further suggested that we could apply stochastic techniques to anisotropic filtering (instead of many samples spaced across the major axis of the filter footprint, take one sample distributed across the axis), and even for the ordinary bilinear or bicubic within each level (instead of 4 or 16 weighted texels, just retrieve one texel chosen with the weights as probabilities). But these deserve to be benchmarked and have their quality evaluated separtely and deliberately, and so will be relegated to a later investigation and implementation.
lgritz
added a commit
to lgritz/OpenImageIO
that referenced
this pull request
Nov 7, 2021
Suggested by Luke Emrose, the idea here is that we introduce two new MIP interpolation modes: StochasticTrilinear and StochasticAniso. These work similarly to Trilinear and Aniso, respectively, except that instead of sampling both of the bracketing MIP levels and blending between them, in stochastic mode we use a random variate to select ONE of the two MIP levels with probability proportional to the computed blending weight. If these mip modes are used, the caller needs to pass a well-stratified value in the new "rnd" field of TextureOpt. Also note that the method only really makes sense in the context of a renderer that is using many samples per pixel. As a single texture lookup, it looks horrible and noisy, as you can see by the testshade-based tests we do of this, which are 1spp and looks pretty rough. My benchmarks (in a hard stress test of the texture system using `testtex --threadtimes 8`) show that this approach has speedups of 35-40% when using between 1-4 threads, dropping to more like 25-30% speedup as threads are in the 12-32 range (that makes sense, it's doing the same amount of total I/O). In this first implementation, in order to preserve ABI compatibility if we decide to backport this feature to the release branch, TextureOpt::rnd is added as a union co-member with the unused "bias" field. For TextureOptBatch, we had to add it outright, but it's guarded by an `#if` and will not be backported to 2.3. (We may change our minds and not backport it at all, I just wanted to leave the option open with this initial patch. Also, there will probably be additional further ABI-breaking changes coming later that will definitely not be backported.) In this first phase, we only use the stochastic component to turn the 2-MIP-level lookup into one level lookup. It has been further suggested that we could apply stochastic techniques to anisotropic filtering (instead of many samples spaced across the major axis of the filter footprint, take one sample distributed across the axis), and even for the ordinary bilinear or bicubic within each level (instead of 4 or 16 weighted texels, just retrieve one texel chosen with the weights as probabilities). But these deserve to be benchmarked and have their quality evaluated separtely and deliberately, and so will be relegated to a later investigation and implementation.
lgritz
added a commit
to lgritz/OpenImageIO
that referenced
this pull request
Nov 7, 2021
Suggested by Luke Emrose, the idea here is that we introduce two new MIP interpolation modes: StochasticTrilinear and StochasticAniso. These work similarly to Trilinear and Aniso, respectively, except that instead of sampling both of the bracketing MIP levels and blending between them, in stochastic mode we use a random variate to select ONE of the two MIP levels with probability proportional to the computed blending weight. If these mip modes are used, the caller needs to pass a well-stratified value in the new "rnd" field of TextureOpt. Also note that the method only really makes sense in the context of a renderer that is using many samples per pixel. As a single texture lookup, it looks horrible and noisy, as you can see by the testshade-based tests we do of this, which are 1spp and looks pretty rough. My benchmarks (in a hard stress test of the texture system using `testtex --threadtimes 8`) show that this approach has speedups of 35-40% when using between 1-4 threads, dropping to more like 25-30% speedup as threads are in the 12-32 range (that makes sense, it's doing the same amount of total I/O). In this first implementation, in order to preserve ABI compatibility if we decide to backport this feature to the release branch, TextureOpt::rnd is added as a union co-member with the unused "bias" field. For TextureOptBatch, we had to add it outright, but it's guarded by an `#if` and will not be backported to 2.3. (We may change our minds and not backport it at all, I just wanted to leave the option open with this initial patch. Also, there will probably be additional further ABI-breaking changes coming later that will definitely not be backported.) In this first phase, we only use the stochastic component to turn the 2-MIP-level lookup into one level lookup. It has been further suggested that we could apply stochastic techniques to anisotropic filtering (instead of many samples spaced across the major axis of the filter footprint, take one sample distributed across the axis), and even for the ordinary bilinear or bicubic within each level (instead of 4 or 16 weighted texels, just retrieve one texel chosen with the weights as probabilities). But these deserve to be benchmarked and have their quality evaluated separtely and deliberately, and so will be relegated to a later investigation and implementation.
lgritz
added a commit
to lgritz/OpenImageIO
that referenced
this pull request
Nov 7, 2021
Suggested by Luke Emrose, the idea here is that we introduce two new MIP interpolation modes: StochasticTrilinear and StochasticAniso. These work similarly to Trilinear and Aniso, respectively, except that instead of sampling both of the bracketing MIP levels and blending between them, in stochastic mode we use a random variate to select ONE of the two MIP levels with probability proportional to the computed blending weight. If these mip modes are used, the caller needs to pass a well-stratified value in the new "rnd" field of TextureOpt. Also note that the method only really makes sense in the context of a renderer that is using many samples per pixel. As a single texture lookup, it looks horrible and noisy, as you can see by the testshade-based tests we do of this, which are 1spp and looks pretty rough. My benchmarks (in a hard stress test of the texture system using `testtex --threadtimes 8`) show that this approach has speedups of 35-40% when using between 1-4 threads, dropping to more like 25-30% speedup as threads are in the 12-32 range (that makes sense, it's doing the same amount of total I/O). In this first implementation, in order to preserve ABI compatibility if we decide to backport this feature to the release branch, TextureOpt::rnd is added as a union co-member with the unused "bias" field. For TextureOptBatch, we had to add it outright, but it's guarded by an `#if` and will not be backported to 2.3. (We may change our minds and not backport it at all, I just wanted to leave the option open with this initial patch. Also, there will probably be additional further ABI-breaking changes coming later that will definitely not be backported.) In this first phase, we only use the stochastic component to turn the 2-MIP-level lookup into one level lookup. It has been further suggested that we could apply stochastic techniques to anisotropic filtering (instead of many samples spaced across the major axis of the filter footprint, take one sample distributed across the axis), and even for the ordinary bilinear or bicubic within each level (instead of 4 or 16 weighted texels, just retrieve one texel chosen with the weights as probabilities). But these deserve to be benchmarked and have their quality evaluated separtely and deliberately, and so will be relegated to a later investigation and implementation.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Suggested by Luke Emrose, the idea here is that we introduce two new
MIP interpolation modes: StochasticTrilinear and StochasticAniso.
These work similarly to Trilinear and Aniso, respectively, except that
instead of sampling both of the bracketing MIP levels and blending
between them, in stochastic mode we use a random variate to select ONE
of the two MIP levels with probability proportional to the computed
blending weight.
If these mip modes are used, the caller needs to pass a
well-stratified value in the new "rnd" field of TextureOpt. Also note
that the method only really makes sense in the context of a renderer
that is using many samples per pixel. As a single texture lookup, it
looks horrible and noisy, as you can see by the testshade-based tests
we do of this, which are 1spp and looks pretty rough.
My benchmarks (in a hard stress test of the texture system using
testtex --threadtimes 8) show that this approach has speedups of35-40% when using between 1-4 threads, dropping to more like 25-30%
speedup as threads are in the 12-32 range (that makes sense, it's
doing the same amount of total I/O).
In this first implementation, in order to preserve ABI compatibility
if we decide to backport this feature to the release branch,
TextureOpt::rnd is added as a union co-member with the unused "bias"
field. For TextureOptBatch, we had to add it outright, but it's
guarded by an
#ifand will not be backported to 2.3. (We may changeour minds and not backport it at all, I just wanted to leave the
option open with this initial patch. Also, there will probably be
additional further ABI-breaking changes coming later that will
definitely not be backported.)
In this first phase, we only use the stochastic component to turn the
2-MIP-level lookup into one level lookup. It has been further
suggested that we could apply stochastic techniques to anisotropic
filtering (instead of many samples spaced across the major axis of the
filter footprint, take one sample distributed across the axis), and
even for the ordinary bilinear or bicubic within each level (instead
of 4 or 16 weighted texels, just retrieve one texel chosen with the
weights as probabilities). But these deserve to be benchmarked and
have their quality evaluated separtely and deliberately, and so will
be relegated to a later investigation and implementation.