[oc start-build] numbers builds incorrectly which results in error while creating them

[tnozicka]

oc start-build numbers builds incorrectly which results in error while creating them. It happens only if you use different machines for calling oc start-build.

Jenkins:

[test2] Running shell script
+ ./oc get builds
NAME           TYPE      FROM      STATUS     STARTED       DURATION
helloworld-1   Source    Binary    Complete   2 hours ago   5m23s
+ ./oc start-build helloworld --from-dir=../artifacts/ --follow
Uploading "../artifacts" at commit "HEAD" as binary input for the build ...
Uploading directory "../artifacts" as binary input for the build ...
Error from server: builds "helloworld-1" already exists

I have tried it manually 2 times by running oc start-build helloworld and run Jenkins again:

[test2] Running shell script
+ ./oc get builds
NAME           TYPE      FROM      STATUS     STARTED         DURATION
helloworld-1   Source    Binary    Complete   2 hours ago     5m23s
helloworld-2   Source    Binary    Running    3 minutes ago   3m8s
helloworld-3   Source    Binary    New                        
+ ./oc start-build helloworld --from-dir=../artifacts/ --follow
Uploading "../artifacts" at commit "HEAD" as binary input for the build ...
Uploading directory "../artifacts" as binary input for the build ...
Error from server: builds "helloworld-1" already exists

Every jenkins run uses a new machine (pod).

Version

I have tried running both v1.2.0-2e62fab and v1.3.0-alpha.1.
On the host:

$ oc version
oc v1.3.0-alpha.1-259-g8cd1645
kubernetes v1.3.0-alpha.1-331-g0522e63

Steps To Reproduce

Hard. Needs multiple machines to manifest.

Current Result

Error from server: builds "helloworld-1" already exists

Expected Result

Works. (Always creates deployment with build number +1 to latest.)

[bparees]

what do you mean by "jenkins"? where is jenkins in this scenario?

what is your shell script doing?

start-build does not behave the way you are describing, so there's more to this picture.

@csrwng @gabemontero since i'm likely to be unavailable as this progresses, please keep an eye on it and help @tnozicka

[bparees]

Are you updating the buildconfig object? the buildconfig object stores the build count, so if you are updating it, you're resetting that value. I seem to recall from some discussions with @detiber and @knrc that you might be creating new buildconfig objects for each build.

[tnozicka]

what do you mean by "jenkins"? where is jenkins in this scenario?

Sorry, It was just my set up in which the error manifested and the logs were from Jenkins. I did not meant to imply that it's fault.

Are you updating the buildconfig object?

@bparees Yes, I am and I did not think there would be a counter in it. Thanks, you most probably found the real issue!

I am doing this with every pipeline build (cycle):

./oc process -f https://github.com/tnozicka/openshift-templates/raw/master/binary-build-template.yaml -v "APP_NAME=helloworld,BUILDER_IMAGE=registry.access.redhat.com/jboss-eap-7/eap70-openshift,OUTPUT_IMAGESTREAM_TAG=build" | ./oc apply -f -
./oc start-build helloworld --from-dir=../artifacts/ --follow

Although I would expect it to work this way.

[bparees]

Yeah so you're reverting the build sequence number each time you do that.
Take the sequence number (lastVersion I think) out of your buildconfig in
the template and then hopefully apply will just preserve the existing
value. You might need to use "patch" instead of apply, I can't remember the
nuances right now (on my phone, can't check).

Ben Parees | OpenShift
On Jun 13, 2016 5:08 PM, "Tomáš Nožička" notifications@github.com wrote:

what do you mean by "jenkins"? where is jenkins in this scenario?

Sorry, It was just my set up in which the error manifested and the logs
were from Jenkins. I did not meant to imply that it's fault.

Are you updating the buildconfig object?

@bparees https://github.com/bparees Yes, I am and I did not think there
would be a counter in it. Thanks, you most probably found the real issue!

I am doing this with every pipeline build (cycle):

./oc process -f https://github.com/tnozicka/openshift-templates/raw/master/binary-build-template.yaml -v "APP_NAME=helloworld,BUILDER_IMAGE=registry.access.redhat.com/jboss-eap-7/eap70-openshift,OUTPUT_IMAGESTREAM_TAG=build" | ./oc apply -f -
./oc start-build helloworld --from-dir=../artifacts/ --follow

Although I would expect it to work this way.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#9301 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AEvl3iSyypD_3w8wa_bZjk96HwYzvoA4ks5qLcbIgaJpZM4I0XjL
.

[tnozicka]

@bparees ok, I will try that as an workaround, but I still believe this is a bug. As a user I don't really care about how the counter is implemented and when I run oc start-build on a valid BC, I expect it to work.

So here are some options on top of my mind right now:

1. Delete corresponding builds when deleting BC
2. When running oc start-build skip the build with colliding numbers and start with the first one available
3. Name builds with some unique ID wrt BC so this collisions does not happen at all

[tnozicka]

Btw. similar case: you have DC and 2 deployments. After you do oc apply on DeploymentConfig it will deploy #3 not #1, so the concept already works somewhere although the implementation surely differs.

[bparees]

well it's not a valid BC because it's a BC that's reusing a version number. like i said, if the BC you were applying didn't set the build counter, it would probably be fine. did you try that?

and we do have logic that deletes all builds when you delete a BC, but you aren't deleting the BC, you're updating it.

we do have trello cards in our backlog for better handling build number collisions.

[tnozicka]

So I have been looking into that closer. The value is in status and I just want to point out that I am not setting it in my BC. It is called {{.status.lastVersion}}

Looking into the same thing in DC, there is {{.status.observedGeneration}} but apply works there!

I have a workaround, but it is quite ugly:

export lastBuildVersion=$(oc get bc jenkinsfile-helloworld -o template --template"{.status.lastVersion}}" 2>/dev/null || true) && \
oc apply -f <somefile> && \
if [[ "${lastBuildVersion}" =~ ^[0-9]+$ ]]; then 
    oc patch bc jenkinsfile-helloworld -p "{\\"status\\":{\\"lastVersion\\":${lastBuildVersion}}}"; 
fi

(I know there is a race condition in case BC would have ConfigChange trigger.)

And I believe it is a valid BC. It is not reusing version number because it is not specified in the BC file, from which it gets applied.

[0xmichalis]

So I have been looking into that closer. The value is in status and I just want to point out that I am not setting it in my BC. It is called {{.status.lastVersion}}

Looking into the same thing in DC, there is {{.status.observedGeneration}} but apply works there!

The respective value in DC is {{.status.latestVersion}}

@bparees seems like buildconfigs should have a separate call for updating status, thus disabling status updates from oc

[bparees]

@tnozicka the other workaround would be to delete the BC before creating the new one. Deleting a BC will delete all associated builds, thus resetting the build count to 0.

@Kargakis i'll need to chat w/ you more about your suggestion, i don't know enough about how patch works but i take it it's somehow related to what patch is doing.

[csrwng]

@Kargakis @bparees we have an issue open for that #6886