Events/msgtype cleanup #9117

drewbailey · 2020-10-16T20:57:28Z

#9013 introduced a few new state store methods that were copies of the originals* with a MsgType explicitly added. This was mainly due to the sheer volume of test case usage. This PR removes the temporary methods and updates all the test usages to use structs.MsgTypeTestSetup

Note to reviewers:

nomad/state/state_store.go removes placholder methods
nomad/job_endpoint uses structs.IgnoreUnknownMsgType as a placeholder for job plan
fsm.go applyDeregisterNodeBatch includes a message type to satisfy the used state store methods, delete events are ignored for now
_test.go files were programmatically updated, I personally wouldn't spend much time reviewing every diff in test file

Placeholder state store methods that have been removed:
UpsertNodeMsgType, DeleteNodeMsgType, UpdateNodeDrainMsgType, UpsertNodeEventsMsgType, UpsertJobMsgType, UpsertEvalsMsgType, UpsertAllocsMsgType

adds message type to signature for upsert node, update tests, remove placeholder method

update test usage of delete node delete placeholder msgtype method

drewbailey · 2020-10-18T17:36:43Z

nomad/fsm.go

@@ -276,7 +276,7 @@ func (n *nomadFSM) Apply(log *raft.Log) interface{} {
 	case structs.SchedulerConfigRequestType:
 		return n.applySchedulerConfigUpdate(buf[1:], log.Index)
 	case structs.NodeBatchDeregisterRequestType:
-		return n.applyDeregisterNodeBatch(buf[1:], log.Index)
+		return n.applyDeregisterNodeBatch(msgType, buf[1:], log.Index)


commenting to highlight non test file changes

I think the applyDeregisterNodeBatch method is always going to have this same msgType. Does passing it as a parameter get us anything here?

Thanks for highlighting the file :).

@tgross mostly just for consistency and explicitness. There are only a few helper methods in fsm that use different msg types but at the time of writing them all I went with passing ¯_(ツ)_/¯

drewbailey · 2020-10-18T17:38:14Z

nomad/job_endpoint.go

@@ -1704,13 +1704,13 @@ func (j *Job) Plan(args *structs.JobPlanRequest, reply *structs.JobPlanResponse)
 		if oldJob.SpecChanged(args.Job) {
 			// Insert the updated Job into the snapshot
 			updatedIndex = oldJob.JobModifyIndex + 1
-			if err := snap.UpsertJob(updatedIndex, args.Job); err != nil {
+			if err := snap.UpsertJob(structs.IgnoreUnknownTypeFlag, updatedIndex, args.Job); err != nil {


commenting to highlight non test file change, a snapshot has a no-op event publisher so this value isn't used IgnoreUnknownTypeFlag is equal to MsgTypeTestSetup

drewbailey · 2020-10-18T17:40:30Z

nomad/state/state_store.go

@@ -782,26 +782,11 @@ func (s *StateStore) ScalingEventsByJob(ws memdb.WatchSet, namespace, jobID stri
 	return nil, 0, nil


commenting to highlight non test file change

…st setup msg type handle snapshot upsert eval outside of FSM and ignore eval event remove placeholder upsertevalsmsgtype handle job plan rpc and prevent event creation for plan msgtype cleanup upsertnodeevents updatenodedrain msgtype msg type 0 is a node registration event, so set the default to the ignore type

tgross

LGTM.

Take a look at the structs2 thing, but my question about the parameter passing is totally a nitpick, so feel free to ignore. 😁

command/job_dispatch_test.go

tgross · 2020-10-19T12:35:40Z

nomad/fsm.go

@@ -276,7 +276,7 @@ func (n *nomadFSM) Apply(log *raft.Log) interface{} {
 	case structs.SchedulerConfigRequestType:
 		return n.applySchedulerConfigUpdate(buf[1:], log.Index)
 	case structs.NodeBatchDeregisterRequestType:
-		return n.applyDeregisterNodeBatch(buf[1:], log.Index)
+		return n.applyDeregisterNodeBatch(msgType, buf[1:], log.Index)


I think the applyDeregisterNodeBatch method is always going to have this same msgType. Does passing it as a parameter get us anything here?

command/job_dispatch_test.go

nomad/client_csi_endpoint_test.go

notnoop · 2020-10-19T12:40:12Z

nomad/fsm.go

@@ -276,7 +276,7 @@ func (n *nomadFSM) Apply(log *raft.Log) interface{} {
 	case structs.SchedulerConfigRequestType:
 		return n.applySchedulerConfigUpdate(buf[1:], log.Index)
 	case structs.NodeBatchDeregisterRequestType:
-		return n.applyDeregisterNodeBatch(buf[1:], log.Index)
+		return n.applyDeregisterNodeBatch(msgType, buf[1:], log.Index)


Thanks for highlighting the file :).

* use msgtype in upsert node adds message type to signature for upsert node, update tests, remove placeholder method * UpsertAllocs msg type test setup * use upsertallocs with msg type in signature update test usage of delete node delete placeholder msgtype method * add msgtype to upsert evals signature, update test call sites with test setup msg type handle snapshot upsert eval outside of FSM and ignore eval event remove placeholder upsertevalsmsgtype handle job plan rpc and prevent event creation for plan msgtype cleanup upsertnodeevents updatenodedrain msgtype msg type 0 is a node registration event, so set the default to the ignore type * fix named import * fix signature ordering on upsertnode to match

github-actions · 2022-12-14T02:18:38Z

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

drewbailey added 3 commits October 16, 2020 15:37

use msgtype in upsert node

825fdda

adds message type to signature for upsert node, update tests, remove placeholder method

UpsertAllocs msg type test setup

86bd246

use upsertallocs with msg type in signature

836217e

update test usage of delete node delete placeholder msgtype method

drewbailey force-pushed the events/msgtype-cleanup branch from 631bdbc to e7d873f Compare October 18, 2020 17:33

vercel bot deployed to Preview October 18, 2020 17:33 View deployment

drewbailey commented Oct 18, 2020

View reviewed changes

drewbailey requested a review from schmichael October 18, 2020 17:41

drewbailey force-pushed the events/msgtype-cleanup branch from e7d873f to 2f2696f Compare October 18, 2020 17:44

vercel bot had a problem deploying to Preview October 18, 2020 17:44 Failure

drewbailey requested review from tgross and notnoop October 19, 2020 12:25

tgross approved these changes Oct 19, 2020

View reviewed changes

notnoop approved these changes Oct 19, 2020

View reviewed changes

fix named import

18951e8

vercel bot deployed to Preview October 19, 2020 12:43 View deployment

fix signature ordering on upsertnode to match

828f73f

vercel bot deployed to Preview October 19, 2020 12:49 View deployment

drewbailey merged commit 7ce0b50 into master Oct 19, 2020

drewbailey deleted the events/msgtype-cleanup branch October 19, 2020 13:30

github-actions bot locked as resolved and limited conversation to collaborators Dec 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Events/msgtype cleanup #9117

Events/msgtype cleanup #9117

drewbailey commented Oct 16, 2020 •

edited

Loading

drewbailey Oct 18, 2020

tgross Oct 19, 2020

notnoop Oct 19, 2020

drewbailey Oct 19, 2020

drewbailey Oct 18, 2020

drewbailey Oct 18, 2020

tgross left a comment

tgross Oct 19, 2020

notnoop Oct 19, 2020

github-actions bot commented Dec 14, 2022

		@@ -782,26 +782,11 @@ func (s *StateStore) ScalingEventsByJob(ws memdb.WatchSet, namespace, jobID stri
		return nil, 0, nil

Events/msgtype cleanup #9117

Events/msgtype cleanup #9117

Conversation

drewbailey commented Oct 16, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tgross left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Dec 14, 2022

drewbailey commented Oct 16, 2020 •

edited

Loading