WIP: test/extended/cli/admin: Add 'oc adm upgrade recommend' smoke test #29831
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: wking The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
wking
changed the title
openshift-ci
bot
added
the
do-not-merge/work-in-progress
wking
commented
May 20, 2025
| g.It("runs successfully, even without upstream OpenShift Update Service customization", func() { | ||
| out, err := oc.Run("adm", "upgrade", "recommend").EnvironmentVariables("OC_ENABLE_CMD_UPGRADE_RECOMMEND=true").Output() | ||
| o.Expect(err).NotTo(o.HaveOccurred()) | ||
| o.Expect(out).To(o.MatchRegexp(`FIXME: not sure what to expect so fail`)) |
There was a problem hiding this comment.
We'll want something based on reality in the regexp here before we merge. I'm hoping presubmits will run the new test-case, fail, and give me example reality-based output I can use to craft a regexp.
There was a problem hiding this comment.
While waiting for feedback on openshift/api#2337, I just dropped the feature-gate portion of the string, and hooray, the test case runs :)
: [sig-cli] oc adm upgrade recommend runs successfully, even without upstream OpenShift Update Service customization [Suite:openshift/conformance/parallel]
Run #0: Failed 2s
{ fail [github.com/openshift/origin/test/extended/cli/admin.go:634]: Expected
<string>: warning: Cannot refresh available updates:
Reason: NoChannel
Message: The update channel has not been configured.
No updates available. You may still upgrade to a specific release image with --to-image or wait for new updates to be available.
to match regular expression
<string>: FIXME: not sure what to expect so fail
Ginkgo exit error 1: exit with code 1}
which is getting us into this code. I dunno if I want to assume that all clusters will have their channel unset, although most CI clusters will (openshift/release#40711). I guess we can start building out a case structure here where we try and predict the output based on different cluster configurations, but if we go too far down that road it ends up being "complicated code in the test-case predicts that similar complicated code in the product is generating similar results", and we can miss catching situations that break the test-case logic and the product logic in similar ways.
wking
force-pushed
the
smoke-test-oc-adm-upgrade-recommend
branch
from
e685bb5 to
26183b7
Compare
|
e2e-aws-ovn died in prep work, before getting close to the test-case I'm trying to add: /retest-required |
|
Job Failure Risk Analysis for sha: 26183b7
|
|
/cc |
wking
commented
May 21, 2025
| @@ -945,6 +945,8 @@ var Annotations = map[string]string{ | |||
|
|
|||
| "[sig-cli][Feature:LegacyCommandTests][Disruptive][Serial] test-cmd: test/cmd/volumes.sh [apigroup:image.openshift.io]": "", | |||
|
|
|||
| "[sig-cli][OCPFeatureGate:OCAdminUpgradeRecommend] oc adm upgrade recommend runs successfully, even without upstream OpenShift Update Service customization": " [Suite:openshift/conformance/parallel]", | |||
There was a problem hiding this comment.
Doesn't seem to be running in the stock parallel suite, looking at e2e-aws-ovn, with no sign of either OCAdminUpgradeRecommend or oc adm upgrade recommend:
$ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/29831/pull-ci-openshift-origin-main-e2e-aws-ovn/1924979491902328832/artifacts/e2e-aws-ovn/openshift-e2e-test/build-log.txt | grep '^started:.*sig-cli.*oc adm [ru]'
started: 1/109/565 "[sig-cli] oc adm user-creation [apigroup:user.openshift.io] [Suite:openshift/conformance/parallel]"
started: 2/198/565 "[sig-cli] oc adm role-selectors [apigroup:template.openshift.io] [Suite:openshift/conformance/parallel]"
started: 2/212/565 "[sig-cli] oc adm ui-project-commands [apigroup:project.openshift.io][apigroup:authorization.openshift.io][apigroup:user.openshift.io] [Suite:openshift/conformance/parallel]"
started: 2/467/565 "[sig-cli] oc adm release extract image-references [Suite:openshift/conformance/parallel]"
started: 3/547/565 "[sig-cli] oc adm role-reapers [apigroup:authorization.openshift.io][apigroup:user.openshift.io] [Suite:openshift/conformance/parallel]"There also don't seem to be any tech-preview presubmits?
There was a problem hiding this comment.
I've opened openshift/api#2337, which will hopefully give the test suite a hint that a feature-gated test should run... somewhere...
There was a problem hiding this comment.
Testing against a default-feature set cluster without setting a feature-gate on the test-case works.
|
There's plenty of optional techpreview presubmits /test |
|
@stbenjam: The The following commands are available to trigger optional jobs: Use In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
I hear that folks want something similar to the OpenShift API conditions
[1]:
1. Tests must contain either [OCPFeatureGate:<FeatureGateName>] or
the standard upstream [FeatureGate:<FeatureGateName>].
2. There must be at least five tests for each FeatureGate.
3. Every test must be run on every TechPreview platform we have jobs
for. (Ask for an exception if your feature doesn't support a
variant.)
...
before promoting command-line features to GA. Or maybe that is a
condition before shifting them from alpha to beta, like [2] moving an
upstream gate from IsEnabled to !IsDisabled? Anyhow, I like testing,
and while in the origin suite we don't really know what, if anything,
we'll find in the ClusterVersion status properties the 'recommend'
command is reporting on, this commit at least runs the command to
confirm it doesn't fail to execute.
[1]: https://github.com/openshift/api/blob/9cbdb71c92bbae8d5939379844b8f84904806453/README.md#defining-featuregate-e2e-tests
[2]: https://github.com/kubernetes/kubernetes/pull/131818/files#diff-7e82ada63a2e6664b803ac3ccdb84759ac97338ce807df63be2f81e3c9e0ba9cR362
wking
force-pushed
the
smoke-test-oc-adm-upgrade-recommend
branch
from
26183b7 to
63d080c
Compare
|
@wking: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
Job Failure Risk Analysis for sha: 63d080c
Risk analysis has seen new tests most likely introduced by this PR. New Test Risks for sha: 63d080c
New tests seen in this PR at sha: 63d080c
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hear that folks want something similar to the OpenShift API conditions:
before promoting command-line features to GA. Or maybe that is a condition before shifting them from alpha to beta, like this upstream change moving a commend-line gate from
IsEnabledto!IsDisabled? Anyhow, I like testing, and while in the origin suite we don't really know what, if anything, we'll find in the ClusterVersion status properties therecommendcommand is reporting on, this commit at least runs the command to confirm it doesn't fail to execute.