feat(events): add events extension #3045

phoban01 · 2025-03-19T18:15:17Z

What type of PR is this?

feature

Which issue does this PR fix:

What does this PR do / Why do we need it:
This PR introduces a new extension: events that enables publishing events in the CloudEvent schema.

The following event types are implemented:

RepositoryCreated
ImageUpdated
ImageDeleted
ImageLintFailed

Two event sinks are implemented as part of this change:

HTTP: a generic http event sink that can be used for webhook style eventing
NATS: a nats event sink that can send events to a nats server

Testing done on this change:

Unit tests added for http and nats event sinks. I have also tested things locally.

Automation added to e2e:

Will this break upgrades or downgrades?
No

Does this PR introduce any user-facing change?:

Yes it introduces a new configuration stanza for events.

The following shows how the extension may be configured:

  "extensions": {
    "events": {
      "enable": true,
      "sinks": [
        {
          "type": "nats",
          "address": "nats://127.0.0.1:4222",
          "timeout": "2s",
          "channel": "alerts.images"
       }, 
        {
          "type": "http",
          "address": "http://127.0.0.1:3333/events/zot/webhook",
          "timeout": "5s",
       }]
    }
  }

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

rchincha · 2025-03-20T02:43:16Z

@phoban01 nice work! thanks for starting this PR.

go.mod

rchincha · 2025-03-20T02:46:26Z

pkg/api/authn_test.go

@@ -954,7 +954,7 @@ func TestCookiestoreCleanup(t *testing.T) {
 		err = os.Chtimes(sessionPath, changeTime, changeTime)
 		So(err, ShouldBeNil)

-		imgStore := local.NewImageStore(rootDir, false, false, log, metrics, nil, nil, nil)
+		imgStore := local.NewImageStore(rootDir, false, false, log, metrics, nil, nil, nil, nil)


We really need to shorten the args list into a struct we can pass around. Not related to your PR thouugh.

pkg/api/config/config.go

pkg/extensions/config/events/config.go

pkg/extensions/events/log_sink.go

phoban01 · 2025-03-20T16:21:46Z

Updated with HTTP and NATs event sinks. Also generalized the sink config.

pkg/extensions/config/events/config.go

examples/config-events.json

phoban01 · 2025-04-02T14:58:53Z

Anything I can do to help bump this along? cc: @rchincha

rchincha · 2025-04-04T05:54:34Z

Anything I can do to help bump this along? cc: @rchincha

There are CI failures that will need fixing.

rchincha · 2025-04-04T16:09:22Z

@phoban01

Let's also add an end-to-end validation with a mock or real CloudEvents server/broker.
Something along these lines.
https://github.com/project-zot/zot/blob/main/.github/workflows/ecosystem-tools.yaml

Easiest to pick something you can package as a container image.
https://github.com/cloudevents/conformance

You are making an investment with this PR, let's also protect it going forward.

phoban01 · 2025-04-04T16:19:33Z

Let's also add an end-to-end validation with a mock or real CloudEvents server/broker.

Yeah, agree; I've been thinking a bit about this. My current dev machine is macos/arm... which is not really compatible with the current Makefile as most tools are targeting amd arch.

I can update the Makefile to address these issues, @rchincha you happy for that to be part of this PR or shall I break it out?

rchincha · 2025-04-05T22:13:50Z

I can update the Makefile to address these issues, @rchincha you happy for that to be part of this PR or shall I break it out?

Either way is fine.

First the CI failures need addressing. Let us know if need help with that. Note that data race detection is enabled for our tests.

scripts/check_logs.sh

phoban01 · 2025-04-10T18:18:58Z

Added blackbox tests for events service covering nats and http.

phoban01 · 2025-05-01T13:38:27Z

Apologies @andaaron; your review got dismissed because of a DCO issue.

rchincha · 2025-05-01T16:35:25Z

{"level":"error","error":"decoding failed due to the following error(s):\n\nerror decoding 'Extensions.Events.Sinks[0]': event sink is not supported","time":"2025-05-01T14:02:39.330842112Z","message":"failed to unmarshal new config"}
Error: decoding failed due to the following error(s):

error decoding 'Extensions.Events.Sinks[0]': event sink is not supported

I believe this PR is very close to merging.

rchincha · 2025-05-01T20:34:54Z

Also, go.mod needs bumping ...
CVEs
Introduced through
github.com/nats-io/[email protected]
Fixed in
github.com/nats-io/nats-server/[email protected], @2.11.1

rchincha · 2025-05-01T21:24:18Z

@phoban01 not sure if you are able to look at coverage profile. If you are, do also look at bumping coverage up.

Signed-off-by: Asgeir Nilsen <[email protected]> Signed-off-by: Piaras Hoban <[email protected]>

Co-authored-by: Andrei Aaron <[email protected]> Signed-off-by: Piaras Hoban <[email protected]>

Signed-off-by: Piaras Hoban <[email protected]>

phoban01 · 2025-05-02T06:52:17Z

@phoban01 not sure if you are able to look at coverage profile. If you are, do also look at bumping coverage up.

I am; spent some time on it earlier this week. Will invest more as time allows.

phoban01 · 2025-05-02T06:56:29Z

{"level":"error","error":"decoding failed due to the following error(s):\n\nerror decoding 'Extensions.Events.Sinks[0]': event sink is not supported","time":"2025-05-01T14:02:39.330842112Z","message":"failed to unmarshal new config"}
Error: decoding failed due to the following error(s):

error decoding 'Extensions.Events.Sinks[0]': event sink is not supported

I believe this PR is very close to merging.

Fix in 524280b

Signed-off-by: Piaras Hoban <[email protected]>

pkg/extensions/config/events/config.go

pkg/extensions/events/extension.go

rchincha

lgtm

pkg/extensions/events/common.go

rchincha · 2025-05-02T19:30:46Z

@phoban01 @asgeirn
PR has been merged.

Pls do consider joining #zot on Slack
https://cloud-native.slack.com/archives/C03EGRE4QGH

rchincha · 2025-05-02T19:58:00Z

https://github.com/project-zot/zot/releases/tag/v2.1.3-rc6
^ now released. @phoban01 enjoy the fruits of your labor, and thanks for sharing with the community.
@asgeirn thx for pitching in.

joejulian · 2025-05-06T13:49:52Z

@rchincha just an FYI. I noticed this introduces a new dependency on helm. The helm project is working on v4 and one of the dependency changes in v4 is oras v2.

You might want to give some thought to upgrading oras at some point.

phoban01 changed the title ~~Feat: support events~~ (feat): support events Mar 19, 2025

phoban01 changed the title ~~(feat): support events~~ feat(events): add events extension Mar 19, 2025

phoban01 force-pushed the feat-support-events branch from 28ab11e to bc805f9 Compare March 19, 2025 18:35

rchincha reviewed Mar 20, 2025

View reviewed changes

go.mod Show resolved Hide resolved

rchincha reviewed Mar 20, 2025

View reviewed changes

pkg/api/config/config.go Outdated Show resolved Hide resolved

rchincha reviewed Mar 20, 2025

View reviewed changes

pkg/extensions/config/events/config.go Outdated Show resolved Hide resolved

rchincha reviewed Mar 20, 2025

View reviewed changes

pkg/extensions/events/log_sink.go Outdated Show resolved Hide resolved

phoban01 marked this pull request as ready for review March 20, 2025 18:19

phoban01 requested review from Andreea-Lupu, shimish2 and andaaron as code owners March 20, 2025 18:19

phoban01 requested a review from rchincha March 21, 2025 16:31

rchincha reviewed Mar 21, 2025

View reviewed changes

pkg/extensions/config/events/config.go Show resolved Hide resolved

rchincha reviewed Mar 21, 2025

View reviewed changes

examples/config-events.json Show resolved Hide resolved

phoban01 force-pushed the feat-support-events branch 2 times, most recently from 6027785 to ba31ccf Compare March 31, 2025 06:47

phoban01 force-pushed the feat-support-events branch from fadab3f to 6a42932 Compare April 4, 2025 08:12

phoban01 force-pushed the feat-support-events branch from 6a42932 to a7632fe Compare April 4, 2025 16:38

phoban01 force-pushed the feat-support-events branch from 1ea5194 to 2b3564b Compare April 7, 2025 07:00

rchincha reviewed Apr 7, 2025

View reviewed changes

scripts/check_logs.sh Show resolved Hide resolved

phoban01 force-pushed the feat-support-events branch from ccc5e45 to 8c0d3f1 Compare April 10, 2025 18:17

phoban01 dismissed andaaron’s stale review via c8c5a68 May 1, 2025 13:32

phoban01 force-pushed the feat-support-events branch 2 times, most recently from c8c5a68 to cc5148a Compare May 1, 2025 13:34

andaaron previously approved these changes May 1, 2025

View reviewed changes

phoban01 dismissed andaaron’s stale review via 30b182a May 1, 2025 18:48

asgeirn and others added 4 commits May 2, 2025 07:48

feat(events): Refactor events to be non-blocking from caller.

1a0fd83

Signed-off-by: Asgeir Nilsen <[email protected]> Signed-off-by: Piaras Hoban <[email protected]>

chore: remove harded-coded linux

fc3124e

Co-authored-by: Andrei Aaron <[email protected]> Signed-off-by: Piaras Hoban <[email protected]>

feat(events): fail to start if incorrect event sink is configured

cba9f60

Signed-off-by: Piaras Hoban <[email protected]>

test: allow cli tests to return errors instead of panic

524280b

Signed-off-by: Piaras Hoban <[email protected]>

phoban01 force-pushed the feat-support-events branch from 30b182a to 524280b Compare May 2, 2025 06:51

andaaron previously approved these changes May 2, 2025

View reviewed changes

chore: bump nats server to v2.11.3

f5ffb76

Signed-off-by: Piaras Hoban <[email protected]>

phoban01 dismissed andaaron’s stale review via f5ffb76 May 2, 2025 07:08

phoban01 force-pushed the feat-support-events branch from 9b640c3 to f5ffb76 Compare May 2, 2025 07:08

rchincha reviewed May 2, 2025

View reviewed changes

pkg/extensions/config/events/config.go Show resolved Hide resolved

rchincha reviewed May 2, 2025

View reviewed changes

pkg/extensions/events/extension.go Show resolved Hide resolved

rchincha approved these changes May 2, 2025

View reviewed changes

rchincha requested a review from andaaron May 2, 2025 19:21

rchincha reviewed May 2, 2025

View reviewed changes

pkg/extensions/events/common.go Show resolved Hide resolved

rchincha merged commit bc5fd1a into project-zot:main May 2, 2025
41 of 43 checks passed

rchincha mentioned this pull request May 2, 2025

[Feat]: add event support #2992

Closed

feat(events): add events extension #3045

feat(events): add events extension #3045

Uh oh!

Conversation

phoban01 commented Mar 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rchincha commented Mar 20, 2025

Uh oh!

Uh oh!

rchincha Mar 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

phoban01 commented Mar 20, 2025

Uh oh!

Uh oh!

Uh oh!

phoban01 commented Apr 2, 2025

Uh oh!

rchincha commented Apr 4, 2025

Uh oh!

rchincha commented Apr 4, 2025

Uh oh!

phoban01 commented Apr 4, 2025

Uh oh!

rchincha commented Apr 5, 2025

Uh oh!

Uh oh!

phoban01 commented Apr 10, 2025

Uh oh!

phoban01 commented May 1, 2025

Uh oh!

rchincha commented May 1, 2025

Uh oh!

rchincha commented May 1, 2025

Uh oh!

rchincha commented May 1, 2025

Uh oh!

phoban01 commented May 2, 2025

Uh oh!

phoban01 commented May 2, 2025

Uh oh!

Uh oh!

Uh oh!

rchincha left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rchincha commented May 2, 2025

Uh oh!

rchincha commented May 2, 2025

Uh oh!

joejulian commented May 6, 2025

Uh oh!

Uh oh!

phoban01 commented Mar 19, 2025 •

edited

Loading