throttle jenkins status to openshift build updates #157
Conversation
|
[test] |
|
@gabemontero If you have it running, I'd like to take a look. |
|
@spadgett per IRC, here is the plugin change ^^ unzip and then install the hpi file in a running jenkins instance |
|
so if there's a stage that takes, say, 5 minutes to run, you're going to get 30s updates? It would definitely be good for the webconsole to move to calculating the duration itself in that case. (Are we already providing the information the webconsole would need to do that? starttime+completiontime for a stage?) |
|
On Fri, Jul 28, 2017 at 1:05 PM, Ben Parees ***@***.***> wrote:
so if there's a stage that takes, say, 5 minutes to run, you're going to
get 30s updates?
yes
It would definitely be good for the webconsole to move to calculating the
duration itself in that case. (Are we already providing the information the
webconsole would need to do that? starttime+completiontime for a stage?)
The json (provided by jenkins, but tweaked by the plugin to prepend the
jenkins root url to any links) annotated to the build includes start time
and durations for a) when actually running b) for when it sits in the
jenkins queue, c) if the run is paused from the jenkins console
… —
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#157 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADbadKr-zOhOZ3pk-a4qQ9-EBtvIqrL3ks5sShT2gaJpZM4OmwIq>
.
|
but not completion times? |
|
On Fri, Jul 28, 2017 at 1:56 PM, Ben Parees ***@***.***> wrote:
The json (provided by jenkins, but tweaked by the plugin to prepend the
jenkins root url to any links) annotated to the build includes start time
and durations for a) when actually running b) for when it sits in the
jenkins queue, c) if the run is paused from the jenkins console
but not completion times?
The overall job struct from Jenkins has an "endTimeInMillis" but it is not
getting set by Jenkins when we call their api to get the corresponding
json.
The stages do not have and endTime filed.
To be clear, all this job metadata and the json we get from Jenkins is
independent of this change.
The end time of the job can be calculated by adding the 3 durations types
from the job/stage/flownode levels to the start times of each.
From the visualization I saw in the web console, that appears to be what
they are doing, but of course @spadgett can clarify / correct as needed.
… —
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#157 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADbadIioL5t3xrjT7zS6sw7cMqRoGDBgks5sSiDqgaJpZM4OmwIq>
.
|
|
@bparees I'm more concerned about the running stage. If there is clock skew between the client and server, we can't accurately calculate duration from just a start time. |
|
Currently we're looking at |
|
On Fri, Jul 28, 2017 at 2:23 PM, Sam Padgett ***@***.***> wrote:
@bparees <https://github.com/bparees> I'm more concerned about the
running stage. If there is clock skew between the client and server, we
can't accurately calculate duration from just a start time.
There is a choice in the start time used .... the one in the json from
Jenkins (same clock / no skew). Then the is the start/create time in the
build object. Different clock
of course, but aside from clock skew between jenkins and openshift master,
there is a delay between when the build gets created in object, the sync
plugin detects and schedules, etc.
—
… You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#157 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADbadEL7QUwebv7NZoJcb0VLc4W0Cczsks5sSicSgaJpZM4OmwIq>
.
|
|
I'm more worried about the client clock -- the user's computer running the browser -- since it can be wayyyy off. But it might be the best we can do. |
only if we continue to provide the duration. if we wanted to get out of the business of frequently updating the metadata purely for the sake of updating the duration, that would depend on the termination time being available to the web console. @spadgett's clock skew concerns notwithstanding. |
|
On Fri, Jul 28, 2017 at 3:10 PM, Ben Parees ***@***.***> wrote:
The end time of the job can be calculated by adding the 3 durations types
from the job/stage/flownode levels to the start times of each.
only if we continue to provide the duration. if we wanted to get out of
the business of frequently updating the metadata purely for the sake of
updating the duration, that would depend on the termination time being
available to the web console. @spadgett <https://github.com/spadgett>'s
clock skew concerns notwithstanding.
Sure all valid discussion points. To call out what is most likely
obvious: this discussion has evolved to explore both short term changes
(immediately addressing the frequency of update concerns) and longer term
along redesigning (at varying degrees potentially).
I'm inclined to have this PR center on the shorter term flavor of change.
Thoughts?
Also, 1 additional piece of info on what we get from Jenkins since I last
responded ... sync plugin *could* set the end time on the job/run object
when we get the final complete event, such that it would be available in
the json annotated to the build that the console consumes.
… —
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#157 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADbadJ0FD8j4dJQWMP6NeTVZFK35OzlCks5sSjIxgaJpZM4OmwIq>
.
|
agree, definitely did not mean to distract. I think what's here is good for the short term changes, though i'm a little concerned about the "5 minute long stage will only see duration updates every 30s" case, but i leave that mostly up to @spadgett |
|
OK good thanks. So I think that leaves us at deciding when to merge this ... factors:
thoughts everyone ? thanks |
gabemontero
force-pushed
the
nicer-build-polling
branch
from
bbe82ef to
53ce3f4
Compare
|
OK, based on a discussion with @spadgett following some initial testing from @rhamilto I've reverted the polling of Jenkins back to 1 second. This way, stage/flow node transitions should be reported quickly to the build object and console, but we still throttle updates to the build object wrt duration, etc. to 30 seconds when the stages / flow nodes stays the same. Here is the updated patch |
|
Evaluated for jenkins sync plugin test up to 53ce3f4 |
Thanks, Gabe. That indeed fixes the issue with the stage transitions. |
|
On Mon, Jul 31, 2017 at 2:42 PM, Robb Hamilton ***@***.***> wrote:
I've reverted the polling of Jenkins back to 1 second. This way,
stage/flow node transitions should be reported quickly to the build object
and console
Thanks, Gabe. That indeed fixes the issue with the stage transitions.
Cool thanks for the quick update Robb. I'll merge once our PR test
completes.
… —
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#157 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADbadFsIZthVXSXyKjGet2IEAt_xXnpGks5sTiAdgaJpZM4OmwIq>
.
|
|
continuous-integration/openshift-jenkins-sync-plugin/test SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pull_request_jenkins_sync_plugin/4/) (Base Commit: 6ac83f7) (PR Branch Commit: 53ce3f4) |
minor readme cleanup
bug 1475867
https://bugzilla.redhat.com/show_bug.cgi?id=1475867
@openshift/devex @spadgett FYI
adds both time and state based throttling; all of whose details are certainly negotiable
in my testing, the openshift web console still seems to render stages OK, but certainly it is "choppier" than before
there's comments explaining the various conditions, but I'll summarize here:
a) an update has not been made in 30 seconds
b) the number of stages or flow nodes changes
c) the job run reaches a terminal state