Backport of volumewatcher: prevent panic on nil volume into release/1.4.x

[hc-github-team-nomad-core]

Backport

This PR is auto-generated from #15101 to be assessed for backporting due to the inclusion of the label backport/1.4.x.

The below text is copied from the body of the original PR.

---

Fixes #15095

If a GC claim is written and then volume is deleted before the volumewatcher enters its run loop, we panic on the nil-pointer access. Simply doing a nil-check at the top of the loop reveals a race condition around shutting down the loop just as a new update is coming in.

Have the parent volumeswatcher send an initial update on the channel before returning, so that we're still holding the lock. Update the watcher's Stop method to set the running state, which lets us avoid having a second context and makes stopping synchronous. This reduces the cases we have to handle in the run loop.

Updated the tests now that we'll safely return from the goroutine and stop the runner in a larger set of cases. Ran the tests with the -race detection flag and fixed up any problems found here as well, and tested this on my local democratic-csi cluster.

[github-actions]

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.