Skip to content

Build: Make the version number be a cmake cache variable #3549

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 13, 2022

Conversation

lgritz
Copy link
Collaborator

@lgritz lgritz commented Sep 12, 2022

Rather than specify the version in the usual project() statement,

project (OpenImageIO VERSION 2.5.0.0 ...)

this patch decouples it

set (OpenImageIO_VERSION "2.5.0.0" CACHE STRING "Version")
project (OpenImageIO VERSION ${OpenImageIO_VERSION} ...)

thus allowing the possibility of overriding the version number at build confuguration time (cmake -DOpenImageIO_VERSION=3.1.4.1).

This should be used with extreme caution. Probably never used by most people. I don't need to describe all the ways that things can go sideways if you make your build of x.y.z.w actually be arbitrarily different from the x.y.z.w that everyone else uses and that is documented.

That said, there are two scenarios I have in mind for which this ability is very useful:

  1. Sometimes, we add or deprecate API features "in advance" by having an #if enable or disable code based on the version, essentially putting in place changes that can't be enabled yet because they would break compatibility rules, but we want to integrate the code now and have it become active upon reaching a particular future version in which that incompatibility will be allowed. The version number override makes it easy to test this by quickly doing a build "as if" we had already bumped the version number to see what will happen when we actually do.

  2. Every once in a while we need to do an internal build that, to conform to version compatibility rules, needs to appear to still be part of a past version family. This lets individual sites play some games with version number overrides rather than have to create new branches and tags for certain one-off internal releases.

Rather than specify the version in the usual project() statement,

    project (OpenImageIO VERSION 2.5.0.0 ...)

this patch decouples it

    set (OpenImageIO_VERSION "2.5.0.0" CACHE STRING "Version")
    project (OpenImageIO VERSION ${OpenImageIO_VERSION} ...)

thus allowing the possibility of overriding the version number at
build confuguration time (`cmake -DOpenImageIO_VERSION=3.1.4.1`).

This should be used with extreme caution. Probably never used by most
people. I don't need to describe all the ways that things can go
sideways if you make your build of x.y.z.w actually be arbitrarily
different from the x.y.z.w that everyone else uses and that is
documented.

That said, there are two scenarios I have in mind for which this
ability is very useful:

1. Sometimes, we add or deprecate API features "in advance" by having
   an `#if` enable or disable code based on the version, essentially
   putting in place changes that can't be enabled yet because they
   would break compatibility rules, but we want to integrate the code
   now and have it become active upon reaching a particular future
   version in which that incompatibility will be allowed. The version
   number override makes it easy to test this by quickly doing a build
   "as if" we had already bumped the version number to see what will
   happen when we actually do.

2. Every once in a while we need to do an internal build that, to
   conform to version compatibility rules, needs to appear to still be
   part of a past version family. This lets individual sites play some
   games with version number overrides rather than have to create new
   branches and tags for certain one-off internal releases.
@lgritz lgritz merged commit 0d9f098 into AcademySoftwareFoundation:master Sep 13, 2022
@lgritz lgritz deleted the lg-version branch September 13, 2022 19:03
lgritz added a commit to lgritz/OpenImageIO that referenced this pull request Sep 14, 2022
…wareFoundation#3549)

Rather than specify the version in the usual project() statement,

    project (OpenImageIO VERSION 2.5.0.0 ...)

this patch decouples it

    set (OpenImageIO_VERSION "2.5.0.0" CACHE STRING "Version")
    project (OpenImageIO VERSION ${OpenImageIO_VERSION} ...)

thus allowing the possibility of overriding the version number at
build confuguration time (`cmake -DOpenImageIO_VERSION=3.1.4.1`).

This should be used with extreme caution. Probably never used by most
people. I don't need to describe all the ways that things can go
sideways if you make your build of x.y.z.w actually be arbitrarily
different from the x.y.z.w that everyone else uses and that is
documented.

That said, there are two scenarios I have in mind for which this
ability is very useful:

1. Sometimes, we add or deprecate API features "in advance" by having
   an `#if` enable or disable code based on the version, essentially
   putting in place changes that can't be enabled yet because they
   would break compatibility rules, but we want to integrate the code
   now and have it become active upon reaching a particular future
   version in which that incompatibility will be allowed. The version
   number override makes it easy to test this by quickly doing a build
   "as if" we had already bumped the version number to see what will
   happen when we actually do.

2. Every once in a while we need to do an internal build that, to
   conform to version compatibility rules, needs to appear to still be
   part of a past version family. This lets individual sites play some
   games with version number overrides rather than have to create new
   branches and tags for certain one-off internal releases.
lgritz added a commit to lgritz/OpenImageIO that referenced this pull request Sep 21, 2022
…wareFoundation#3549)

Rather than specify the version in the usual project() statement,

    project (OpenImageIO VERSION 2.5.0.0 ...)

this patch decouples it

    set (OpenImageIO_VERSION "2.5.0.0" CACHE STRING "Version")
    project (OpenImageIO VERSION ${OpenImageIO_VERSION} ...)

thus allowing the possibility of overriding the version number at
build confuguration time (`cmake -DOpenImageIO_VERSION=3.1.4.1`).

This should be used with extreme caution. Probably never used by most
people. I don't need to describe all the ways that things can go
sideways if you make your build of x.y.z.w actually be arbitrarily
different from the x.y.z.w that everyone else uses and that is
documented.

That said, there are two scenarios I have in mind for which this
ability is very useful:

1. Sometimes, we add or deprecate API features "in advance" by having
   an `#if` enable or disable code based on the version, essentially
   putting in place changes that can't be enabled yet because they
   would break compatibility rules, but we want to integrate the code
   now and have it become active upon reaching a particular future
   version in which that incompatibility will be allowed. The version
   number override makes it easy to test this by quickly doing a build
   "as if" we had already bumped the version number to see what will
   happen when we actually do.

2. Every once in a while we need to do an internal build that, to
   conform to version compatibility rules, needs to appear to still be
   part of a past version family. This lets individual sites play some
   games with version number overrides rather than have to create new
   branches and tags for certain one-off internal releases.
lgritz added a commit to lgritz/OpenImageIO that referenced this pull request Oct 6, 2022
…wareFoundation#3549)

Rather than specify the version in the usual project() statement,

    project (OpenImageIO VERSION 2.5.0.0 ...)

this patch decouples it

    set (OpenImageIO_VERSION "2.5.0.0" CACHE STRING "Version")
    project (OpenImageIO VERSION ${OpenImageIO_VERSION} ...)

thus allowing the possibility of overriding the version number at
build confuguration time (`cmake -DOpenImageIO_VERSION=3.1.4.1`).

This should be used with extreme caution. Probably never used by most
people. I don't need to describe all the ways that things can go
sideways if you make your build of x.y.z.w actually be arbitrarily
different from the x.y.z.w that everyone else uses and that is
documented.

That said, there are two scenarios I have in mind for which this
ability is very useful:

1. Sometimes, we add or deprecate API features "in advance" by having
   an `#if` enable or disable code based on the version, essentially
   putting in place changes that can't be enabled yet because they
   would break compatibility rules, but we want to integrate the code
   now and have it become active upon reaching a particular future
   version in which that incompatibility will be allowed. The version
   number override makes it easy to test this by quickly doing a build
   "as if" we had already bumped the version number to see what will
   happen when we actually do.

2. Every once in a while we need to do an internal build that, to
   conform to version compatibility rules, needs to appear to still be
   part of a past version family. This lets individual sites play some
   games with version number overrides rather than have to create new
   branches and tags for certain one-off internal releases.
@dg0yt
Copy link

dg0yt commented Nov 5, 2022

Note that with this change, the cached version number in a build directory will never increase until it is cleaned explicitly. Not a problem for throw-away CI builds, but for some human workflows. You may get bug reports with a wrong version.

@lgritz
Copy link
Collaborator Author

lgritz commented Nov 5, 2022

That's a good point. That wasn't really what I was aiming for, and I blow away my entire build area including cache 20 times a day (I rely a lot on ccache to make rebuilds inexpensive, so I rm -rf build at the drop of a hat), so this side effect wouldn't be something I would notice.

Maybe I should reconsider this. I was really just aiming for a way to twiddle the version without editing the build scripts, for certain testing purposes.

@lgritz
Copy link
Collaborator Author

lgritz commented Nov 5, 2022

@dg0yt Help me think this through.

I think the scenario that would be a problem is if you did a

rm -rf build ; cmake -B build -S . ; cmake --build build

and then switched branches and kicked off another build

git checkout other-branch
cmake -B build -S . ; cmake --build build

the problem is that if other-branch is a different version, then the version would be cached and the new build wouldn't reflect that (even though it would know to rebuild all the .o files that correspond to different source code between the branches).

I can verify that this does indeed make a build with the wrong version number. I have to admit that I never considered that, and it seems bad, especially because it will happen even if there is no -DOpenImageIO_VERSION=..., in other words, it's not just affecting people who know they are playing with fire by overriding versions.

But... but... isn't it true that all sorts of other things will also be cached that could screw up the build?

I mean, any time you do a checkout that is from a different enough branch that it might have a different version, you are really taking risks if you don't blow away the cache, because any number of things that don't get recomputed from scratch may give you a build that will differ from a fresh checkout, by virtue of being cached from when you configured the other branch.

And certainly, not just for CI, but I would hope that anybody doing a build that might be actually used by somebody else, or deployed in any way, should do a completely fresh build for these reasons. You just don't know what's left over in the cache.

So help me thing through this issue. What is the scenario, the sequence of events, where a user would trigger this behavior in a bad way, where they would NOT clear the cache first, and be surprised by having left over settings from a previous build?

@lgritz
Copy link
Collaborator Author

lgritz commented Nov 5, 2022

What do you think about this solution:

set (OpenImageIO_VERSION "2.5.0.0")
set (OpenImageIO_VERSION_OVERRIDE "" CACHE STRING
     "Version override (use with caution)!")
if (OpenImageIO_VERSION_OVERRIDE)
    set (OpenImageIO_VERSION ${OpenImageIO_VERSION_OVERRIDE})
endif ()

project (OpenImageIO VERSION ${OpenImageIO_VERSION}
         HOMEPAGE_URL "https://openimageio.org"
         LANGUAGES CXX C)

Now with this:

  • You must use the special -DOpenImageIO_VERSION_OVERRIDE=... to override the version.
    • if you do so, you are playing with fire and get what you deserve if you misuse it or change branches and don't remember to clear the cache.
  • If you do a normal build (without trying to override the version),
    • you will get the real version number because it IS NOT cached
    • if you switch branches, you'll always get the right version number
    • ...but switching branches without clearing the cache can still, as always, have other subtle behavior differences versus doing a truly fresh build from scratch.

@lgritz
Copy link
Collaborator Author

lgritz commented Nov 5, 2022

Yeah, I think I like that last proposal better, it's closer to my original intent, which was to allow people (mostly me) to do certain tests with a version override, but it would not affect ordinary users in any way if they weren't going out of their way to do the override.

@dg0yt
Copy link

dg0yt commented Nov 5, 2022

I was less thinking about switching branches then about continuously pulling from (or merging to) the same branch.

I can't say how important it will become in practice. But I think I can say it is unusual and has a chance to create unusual problems. (I just stumbled over a case in vcpkg tool where a dependency URL was a cache variable but not the corresponding SHA512. This caused unexpected errors when the URL was updated in the source. But they only needed a pure input variable for CI, not a long-life cache variable for the URL.)

Your proposal looks good to me.I think it will also play well with existing caches. I would probably add mark_as_advanced.

@lgritz
Copy link
Collaborator Author

lgritz commented Nov 5, 2022

OK, I'll do that, thanks for the feedback.

@lgritz
Copy link
Collaborator Author

lgritz commented Nov 5, 2022

Proposed fix in #3653

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants