Consider bringing back the post-release dev versions

[srikanthccv]

Yeah I think going back to dev versions will be fine. At least until metrics/logs become a bit more stable.

Originally posted by @lzchen in #2583 (comment)

[srikanthccv]

This is done now.