[KS-196] ReportCodec implementation for Streams trigger #13218

bolekk · 2024-05-16T02:11:13Z

Implement Codec, which validates report signatures and decodes needed fields.
Pass report context from Merucry Transmitter, which is needed to validate signatures.
Update fake Syncer to run successful e2e tests.

samsondav · 2024-05-21T13:45:07Z

core/capabilities/streams/codec.go

-func (c Codec) Unwrap(raw values.Value) ([]datastreams.FeedReport, error) {
+var _ datastreams.ReportCodec = &codec{}
+
+func (c *codec) UnwrapValid(wrapped values.Value, allowedSigners [][]byte, minRequiredSignatures int) ([]datastreams.FeedReport, error) {


May I ask, why is the list of reports passed as a wrapped values.Value?

values.Value is a universal wrapper for all data passed around during workflow execution. Trigger Service will wrap it and then the receiver DON will unwrap it here.

samsondav · 2024-05-21T13:45:57Z

core/capabilities/syncer.go

+	s.wg.Add(1)
+	go s.launch(ctx)
+	return nil


Worth implementing a StateMachine here?

This Syncer implementation will soon be replaced with a real one reading stuff from the chain. Right now it's here just to allow easy local and staging tests. Please ignore all shortcomings ;)

samsondav · 2024-05-21T13:46:29Z

core/capabilities/syncer.go

 		}
 		s.subServices = append(s.subServices, triggerCap)
 	}
 	if slices.Contains(triggerDONPeers, myId) {
 		s.lggr.Info("member of a capability DON - starting remote publishers")

-		{
+		/*{


Did you mean to delete this?

I want to leave it for easier local tests - will be replaced within 2 weeks.

samsondav · 2024-05-21T14:29:06Z

core/capabilities/syncer.go

+			count++
+			if count > maxRetrySeconds {
+				s.lggr.Error("failed to get Streams Trigger from the Registry")
+				return


Is this a permanent failure that leaves the node in a broken state?

broken meaning unable to send trigger events but otherwise functional.

samsondav · 2024-05-21T14:29:22Z

core/capabilities/syncer.go

+		count := 0
+		for {
+			count++
+			if count > maxRetrySeconds {


Seems confusingly named, count doesn't appear to be a timer

true, will remove the "seconds" thanks

samsondav · 2024-05-21T14:29:51Z

core/capabilities/syncer.go

+				return
+			}
+			s.subServices = append(s.subServices, triggerCap)
+			break


Would this be cleaner using a backoff.Backoff with permanent retry?

yes but probably not worth it - again this Syncer won't be used in prod.

DeividasK · 2024-05-21T16:24:14Z

core/capabilities/syncer.go

+		"0x9CcE7293a4Cc2621b61193135A95928735e4795F",
+		"0x3c775F20bCB2108C1A818741Ce332Bb5fe0dB925",
+		"0x50314239e2CF05555ceeD53E7F47eB2A8Eab0dbB",
+		"0xd76A4f98898c3b9A72b244476d7337b50D54BCd8",
+		"0x656A873f6895b8a03Fb112dE927d43FA54B2c92A",
+		"0x5d1e87d87bF2e0cD4Ea64F381a2dbF45e5f0a553",
+		"0x91d9b0062265514f012Eb8fABA59372fD9520f56",


Heads up, that this will change to be bytes32 onchain (https://github.com/smartcontractkit/chainlink/pull/13183/files#diff-d94c1796b3544ef8780af7e16b5361c8fb9e4d63f492faeda6a3362a8363f51bR45-R51). We can do decoding once the Syncer is up, but not sure the best place to do that, thread: https://chainlink-core.slack.com/archives/C064418MJHE/p1716308390423929

That's fine. As long as the Syncer passes correct binary addresses to all services, we're good.

DeividasK · 2024-05-21T16:47:07Z

core/capabilities/syncer.go

+			if err2 != nil {
+				// NOTE: it's possible that the jobs are not launched yet at this moment.
+				// If not found yet, Syncer won't add to Registry but retry on the next tick.
+				s.lggr.Infow("trigger not found yet ...", "capabilityId", capId, "error", err)


Suggested change

s.lggr.Infow("trigger not found yet ...", "capabilityId", capId, "error", err)

s.lggr.Infow("trigger not found yet ...", "capabilityId", capId, "error", err2)

fixing, thanks

DeividasK · 2024-05-21T16:51:28Z

core/capabilities/syncer.go

+			workflowDONs := map[string]capabilities.DON{
+				workflowDonInfo.ID: workflowDonInfo,
+			}
+			triggerCap := remote.NewTriggerPublisher(config, underlying, triggerInfo, triggerCapabilityDonInfo, workflowDONs, s.dispatcher, s.lggr)


In the future, this should only happen when DON isPublic, right? Currently, this assumes that the capability is always remote without actually checking for local existence, right?

DeividasK · 2024-05-21T16:59:21Z

core/services/relay/evm/mercury/transmitter.go

@@ -388,15 +388,22 @@ func (mt *mercuryTransmitter) HealthReport() map[string]error {
 	return report
 }

-func (mt *mercuryTransmitter) sendToTrigger(report ocrtypes.Report, signatures []ocrtypes.AttributedOnchainSignature) error {
+func (mt *mercuryTransmitter) sendToTrigger(report ocrtypes.Report, rawReportCtx [3][32]byte, signatures []ocrtypes.AttributedOnchainSignature) error {


Shouldn't this rawReportCtx [3][32]byte, be based on a workflow spec config?

How so? The context encodes OCR round and epoch in which the report was produced.

I guess it's not clear what this context is.

It should be pretty clear in the context of the Transmitter because it already uses it elsewhere.

1. Implement Codec, which validates report signatures and decodes needed fields. 2. Pass report context from Merucry Transmitter, which is needed to validate signatures. 3. Update fake Syncer to run successful e2e tests.

cl-sonarqube-production · 2024-05-21T17:32:46Z

Quality Gate passed

Issues
0 New issues
1 Fixed issue
0 Accepted issues

Measures
0 Security Hotspots
68.8% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

bolekk added the DO_NOT_MERGE label May 16, 2024

bolekk requested review from a team as code owners May 16, 2024 02:11

bolekk temporarily deployed to sdlc May 16, 2024 02:11 — with GitHub Actions Inactive

bolekk force-pushed the deploy_staging branch from 2b04805 to 97c0f8a Compare May 16, 2024 14:32

bolekk temporarily deployed to sdlc May 16, 2024 14:32 — with GitHub Actions Inactive

bolekk force-pushed the deploy_staging branch from 97c0f8a to 212ec09 Compare May 16, 2024 15:18

bolekk requested a review from a team May 16, 2024 15:18

bolekk temporarily deployed to sdlc May 16, 2024 15:18 — with GitHub Actions Inactive

bolekk force-pushed the deploy_staging branch from 212ec09 to b0065c7 Compare May 16, 2024 16:17

bolekk temporarily deployed to sdlc May 16, 2024 16:18 — with GitHub Actions Inactive

bolekk force-pushed the deploy_staging branch from b0065c7 to 7b5c5e5 Compare May 16, 2024 17:17

bolekk temporarily deployed to sdlc May 16, 2024 17:17 — with GitHub Actions Inactive

bolekk force-pushed the deploy_staging branch from 7b5c5e5 to e6dc3d5 Compare May 17, 2024 23:01

bolekk temporarily deployed to sdlc May 17, 2024 23:01 — with GitHub Actions Inactive

bolekk force-pushed the deploy_staging branch from e6dc3d5 to a29642f Compare May 18, 2024 00:16

bolekk temporarily deployed to sdlc May 18, 2024 00:16 — with GitHub Actions Inactive

bolekk force-pushed the deploy_staging branch from a29642f to dec9600 Compare May 20, 2024 04:22

bolekk temporarily deployed to sdlc May 20, 2024 04:22 — with GitHub Actions Inactive

bolekk force-pushed the deploy_staging branch from dec9600 to fb8c8a5 Compare May 20, 2024 04:50

bolekk temporarily deployed to sdlc May 20, 2024 04:50 — with GitHub Actions Inactive

bolekk force-pushed the deploy_staging branch from fb8c8a5 to 0c2da64 Compare May 20, 2024 05:12

bolekk temporarily deployed to sdlc May 20, 2024 05:12 — with GitHub Actions Inactive

bolekk force-pushed the deploy_staging branch from 0c2da64 to c54d126 Compare May 20, 2024 14:39

bolekk temporarily deployed to sdlc May 20, 2024 14:39 — with GitHub Actions Inactive

bolekk force-pushed the deploy_staging branch 2 times, most recently from 49d7904 to db12eea Compare May 21, 2024 01:30

bolekk temporarily deployed to sdlc May 21, 2024 01:30 — with GitHub Actions Inactive

bolekk force-pushed the deploy_staging branch from db12eea to ac1bba1 Compare May 21, 2024 02:52

bolekk temporarily deployed to sdlc May 21, 2024 02:52 — with GitHub Actions Inactive

bolekk removed the DO_NOT_MERGE label May 21, 2024

bolekk requested a review from krehermann May 21, 2024 02:54

bolekk force-pushed the deploy_staging branch from ac1bba1 to 440cf61 Compare May 21, 2024 03:09

bolekk temporarily deployed to sdlc May 21, 2024 03:09 — with GitHub Actions Inactive

bolekk requested a review from samsondav May 21, 2024 03:30

samsondav requested changes May 21, 2024

View reviewed changes

bolekk force-pushed the deploy_staging branch from 440cf61 to e42f85a Compare May 21, 2024 15:03

bolekk temporarily deployed to sdlc May 21, 2024 15:04 — with GitHub Actions Inactive

bolekk force-pushed the deploy_staging branch from e42f85a to aa9659d Compare May 21, 2024 15:20

bolekk temporarily deployed to sdlc May 21, 2024 15:21 — with GitHub Actions Inactive

DeividasK reviewed May 21, 2024

View reviewed changes

DeividasK previously approved these changes May 21, 2024

View reviewed changes

[KS-196] ReportCodec implementation for Streams trigger

28f8e52

1. Implement Codec, which validates report signatures and decodes needed fields. 2. Pass report context from Merucry Transmitter, which is needed to validate signatures. 3. Update fake Syncer to run successful e2e tests.

bolekk dismissed DeividasK’s stale review via 28f8e52 May 21, 2024 17:16

bolekk force-pushed the deploy_staging branch from aa9659d to 28f8e52 Compare May 21, 2024 17:16

bolekk temporarily deployed to sdlc May 21, 2024 17:16 — with GitHub Actions Inactive

DeividasK approved these changes May 21, 2024

View reviewed changes

samsondav approved these changes May 21, 2024

View reviewed changes

bolekk added this pull request to the merge queue May 21, 2024

Merged via the queue into develop with commit 4938ef3 May 21, 2024
107 checks passed

bolekk deleted the deploy_staging branch May 21, 2024 18:11

This was referenced May 21, 2024

[DO NOT MERGE] Changeset Release Preview - v2.19.0 #13148

Draft

[DO NOT MERGE] Changeset Release Preview - v2.16.0 EthanAnro/chainlink#2

Draft

[DO NOT MERGE] Changeset Release Preview - v2.17.0 horberlan/chainlink#1

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[KS-196] ReportCodec implementation for Streams trigger #13218

[KS-196] ReportCodec implementation for Streams trigger #13218

bolekk commented May 16, 2024 •

edited

Loading

samsondav May 21, 2024

bolekk May 21, 2024

samsondav May 21, 2024

bolekk May 21, 2024

samsondav May 21, 2024

bolekk May 21, 2024

samsondav May 21, 2024

bolekk May 21, 2024

samsondav May 21, 2024

bolekk May 21, 2024

samsondav May 21, 2024

bolekk May 21, 2024

DeividasK May 21, 2024 •

edited

Loading

bolekk May 21, 2024

DeividasK May 21, 2024

bolekk May 21, 2024

DeividasK May 21, 2024

bolekk May 21, 2024

DeividasK May 21, 2024

bolekk May 21, 2024

DeividasK May 21, 2024

bolekk May 21, 2024

cl-sonarqube-production bot commented May 21, 2024

	s.lggr.Infow("trigger not found yet ...", "capabilityId", capId, "error", err)
	s.lggr.Infow("trigger not found yet ...", "capabilityId", capId, "error", err2)

[KS-196] ReportCodec implementation for Streams trigger #13218

[KS-196] ReportCodec implementation for Streams trigger #13218

Conversation

bolekk commented May 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DeividasK May 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cl-sonarqube-production bot commented May 21, 2024

Quality Gate passed

bolekk commented May 16, 2024 •

edited

Loading

DeividasK May 21, 2024 •

edited

Loading