Dependency pinning between packages post 1.0

[ocelotl]

Discussed in #1754

^{Originally posted by owais April 6, 2021}
Now that we've hit 1.0 for most packages, I think we should not pin stable opentelemetry-* packages to exact versions. Right now, most opentelemetry-* packages now pin other stable opentelemetry-* packages to 1.0.0. For example,

https://github.com/open-telemetry/opentelemetry-python/blob/v1.0.0/opentelemetry-distro/setup.cfg#L43-L44
https://github.com/open-telemetry/opentelemetry-python/blob/v1.0.0/opentelemetry-sdk/setup.cfg#L44
https://github.com/open-telemetry/opentelemetry-python/blob/v1.0.0/opentelemetry-instrumentation/setup.cfg#L44
https://github.com/open-telemetry/opentelemetry-python/blob/v1.0.0/exporter/opentelemetry-exporter-otlp-proto-grpc/setup.cfg#L44-L46

This can be very annoying for users. For example,

Assuming we continue to release all packages in unison so every release contains a new version of every package and all stable packages for a release have the same version number.

• opentelemetry-sdk v1.2 introduced a new feature in v1.2 along with a bug.
• opentelemetry-distro v1.3 does not really need the new SDK feature but is forced to install SDK 1.3 because of exact dep pinning.
• This forces users to use SDK 1.3 which still contains the bug or downgrade everything to 1.1 not allowing users to take advantage of new features in Distro 1.3.

Instead of pinning exact deps for stable packages, we should pin >=1.N, <2.0 where N is the minimum minor version required by the package. For example, opentelemetry-sdk should continue to depend on opentelemetry-api>=1.0,<2.0 until it needs a specific feature/fix shipped in a later version of the API package. If the API package introduces an additive change in 1.6 that SDK could take advantage of then SDK 1.6 would finally update it's dependency on API to >=1.6<2.0.

We should allow users to mix and match any official and community packages together as long as long as they are API compatible. Users should be able to use the hypothetical versions api 1.5, sdk 1.10,exporter 1.14, distro 1.15 together. However, exporter 1.15 may introduce a change that depends on sdk 1.13. In that case, user would be forced to use sdk >=1.13 only when using exporter >=1.15.

Considerations

Extra maintenance burden

This does introduce extra burden as maintainers will now have to carefully update dependencies based on what actual changes were made since last release instead of blindly bumping the next minor version everywhere. This is a big change in the workflow but I think it's worth it given how much flexibility this will provide to the Python ecosystem for OpenTelemetry. With some workflow changes such as expecting contributors to update dependencies for each PR if required, we should be able to reduce this burden.