Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix race condition in StreamOperatorService #72

Merged
merged 5 commits into from
May 15, 2024
Merged

Fix race condition in StreamOperatorService #72

merged 5 commits into from
May 15, 2024

Conversation

s-vitaliy
Copy link
Contributor

@s-vitaliy s-vitaliy commented May 14, 2024 •
edited
Loading

Part of #45

The actual problem was with the race condition that happens if the StreamClass was created prior to the CRD.

Scope

Implemented:

  • Add retries to StreamClass event handler.

Additional changes:

  • Fixed PrinterColumns for StreamClass object
  • Create CRDs by default

Checklist

  • GitHub issue exists for this change.
  • Unit tests added and they pass.
  • Line Coverage is at least 80%.
  • Review requested on latest commit.

@github-actions GitHub Actions
Copy link

Coverage after merging fix-race into main will be

63.21%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
src/Configurations
   MetricsPublisherActorConfiguration.cs100%100%100%100%
   MetricsReporterConfiguration.cs100%100%100%100%
src/Extensions
   JsonElementExtensions.cs0%100%0%0%22, 30
   V1JobExtensions.cs85.58%75%90.91%90.16%40, 40, 45, 61–62, 62, 62, 62, 62–65, 83, 83
src/Models
   StreamingClassOperatorResponse.cs100%100%100%100%
src/Models/StreamClass
   V1Beta1StreamClassSpec.cs100%100%100%100%
src/Models/StreamDefinitions/Base
   IStreamDefinition.cs90%83.33%100%100%29
src/Models/StreamStatuses/StreamStatus/V1Beta1
   V1Beta1StreamStatus.cs100%100%100%100%
src/Services/Maintenance
   HostedStreamingJobMaintenanceService.cs0%0%0%0%16–24, 27–29, 29, 29–36, 39–42
   StreamingJobMaintenanceService.cs86.71%72.50%100%91.67%103, 109–112, 120, 125, 131, 131, 134, 134, 134, 134–137, 141, 67, 72
src/Services/Metrics
   DeclaredMetrics.cs88.24%60%100%100%32, 32, 32, 32
   MetricsReporter.cs100%100%100%100%
src/Services/Metrics/Actors
   MetricsPublisherActor.cs85.71%50%100%86.05%90–92, 92, 92–95
src/Services/Models
   CustomResourceApiRequest.cs100%100%100%100%
   ResourceEvent.cs100%100%100%100%
src/Services/Operator
   StreamClassOperatorService.cs74.23%50%77.78%77.63%106–109, 124–127, 82, 82, 82, 85, 87, 92–94, 94, 94–99
   StreamOperatorService.cs87.62%77.59%100%91.30%116–122, 127, 135, 149, 149, 155, 177, 65–66, 66, 66, 66, 66, 66, 66–70
   StreamOperatorServiceWorker.cs77.78%50%75%80%46–50, 55, 63
   StreamOperatorServiceWorkerFactory.cs100%100%100%100%
src/Services/Repositories
   StreamClassRepository.cs80%0%75%85.29%35, 35, 35–39
   StreamDefinitionRepository.cs0%0%0%0%102–104, 104, 104–117, 121–128, 28–35, 38–40, 40, 40–55, 59–61, 61, 61–82, 85–87, 87, 87–99
   StreamingJobTemplateRepository.cs0%0%0%0%20–27, 31–33, 33, 33, 33, 33, 33, 33, 33, 33–36, 39–47
src/Services/Streams
   StreamingJobOperatorService.cs0%0%0%0%100–103, 106–109, 109, 109–111, 114–117, 117, 117–119, 122–125, 125, 125–127, 130–133, 136–145, 28–38, 41, 44–51, 55–56, 56, 56–59, 59, 59–66, 66, 66–86, 86, 86, 86, 86–89, 89, 89–99

@github-actions GitHub Actions
Copy link

Coverage after merging fix-race into main will be

63.60%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
src/Configurations
   MetricsPublisherActorConfiguration.cs100%100%100%100%
   MetricsReporterConfiguration.cs100%100%100%100%
src/Extensions
   JsonElementExtensions.cs0%100%0%0%22, 30
   V1JobExtensions.cs85.58%75%90.91%90.16%40, 40, 45, 61–62, 62, 62, 62, 62–65, 83, 83
src/Models
   StreamingClassOperatorResponse.cs100%100%100%100%
src/Models/StreamClass
   V1Beta1StreamClassSpec.cs100%100%100%100%
src/Models/StreamDefinitions/Base
   IStreamDefinition.cs90%83.33%100%100%29
src/Models/StreamStatuses/StreamStatus/V1Beta1
   V1Beta1StreamStatus.cs100%100%100%100%
src/Services/Maintenance
   HostedStreamingJobMaintenanceService.cs0%0%0%0%16–24, 27–29, 29, 29–36, 39–42
   StreamingJobMaintenanceService.cs86.71%72.50%100%91.67%103, 109–112, 120, 125, 131, 131, 134, 134, 134, 134–137, 141, 67, 72
src/Services/Metrics
   DeclaredMetrics.cs88.24%60%100%100%32, 32, 32, 32
   MetricsReporter.cs100%100%100%100%
src/Services/Metrics/Actors
   MetricsPublisherActor.cs85.71%50%100%86.05%90–92, 92, 92–95
src/Services/Models
   CustomResourceApiRequest.cs100%100%100%100%
   ResourceEvent.cs100%100%100%100%
src/Services/Operator
   StreamClassOperatorService.cs80.22%58.33%87.50%83.10%114–117, 79, 79, 83, 88–90, 90, 90–95
   StreamOperatorService.cs87.62%77.59%100%91.30%116–122, 127, 135, 149, 149, 155, 177, 65–66, 66, 66, 66, 66, 66, 66–70
   StreamOperatorServiceWorker.cs77.78%50%75%80%46–50, 55, 63
   StreamOperatorServiceWorkerFactory.cs100%100%100%100%
src/Services/Repositories
   StreamClassRepository.cs80%0%75%85.29%35, 35, 35–39
   StreamDefinitionRepository.cs0%0%0%0%102–104, 104, 104–117, 121–128, 28–35, 38–40, 40, 40–55, 59–61, 61, 61–82, 85–87, 87, 87–99
   StreamingJobTemplateRepository.cs0%0%0%0%20–27, 31–33, 33, 33, 33, 33, 33, 33, 33, 33–36, 39–47
src/Services/Streams
   StreamingJobOperatorService.cs0%0%0%0%100–103, 106–109, 109, 109–111, 114–117, 117, 117–119, 122–125, 125, 125–127, 130–133, 136–145, 28–38, 41, 44–51, 55–56, 56, 56–59, 59, 59–66, 66, 66–86, 86, 86, 86, 86–89, 89, 89–99

@s-vitaliy s-vitaliy marked this pull request as ready for review May 15, 2024 09:11
@s-vitaliy s-vitaliy requested a review from a team as a code owner May 15, 2024 09:11
@github-actions GitHub Actions
Copy link

Coverage after merging fix-race into main will be

63.60%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
src/Configurations
   MetricsPublisherActorConfiguration.cs100%100%100%100%
   MetricsReporterConfiguration.cs100%100%100%100%
src/Extensions
   JsonElementExtensions.cs0%100%0%0%22, 30
   V1JobExtensions.cs85.58%75%90.91%90.16%40, 40, 45, 61–62, 62, 62, 62, 62–65, 83, 83
src/Models
   StreamingClassOperatorResponse.cs100%100%100%100%
src/Models/StreamClass
   V1Beta1StreamClassSpec.cs100%100%100%100%
src/Models/StreamDefinitions/Base
   IStreamDefinition.cs90%83.33%100%100%29
src/Models/StreamStatuses/StreamStatus/V1Beta1
   V1Beta1StreamStatus.cs100%100%100%100%
src/Services/Maintenance
   HostedStreamingJobMaintenanceService.cs0%0%0%0%16–24, 27–29, 29, 29–36, 39–42
   StreamingJobMaintenanceService.cs86.71%72.50%100%91.67%103, 109–112, 120, 125, 131, 131, 134, 134, 134, 134–137, 141, 67, 72
src/Services/Metrics
   DeclaredMetrics.cs88.24%60%100%100%32, 32, 32, 32
   MetricsReporter.cs100%100%100%100%
src/Services/Metrics/Actors
   MetricsPublisherActor.cs85.71%50%100%86.05%90–92, 92, 92–95
src/Services/Models
   CustomResourceApiRequest.cs100%100%100%100%
   ResourceEvent.cs100%100%100%100%
src/Services/Operator
   StreamClassOperatorService.cs80.22%58.33%87.50%83.10%114–117, 79, 79, 83, 88–90, 90, 90–95
   StreamOperatorService.cs87.62%77.59%100%91.30%116–122, 127, 135, 149, 149, 155, 177, 65–66, 66, 66, 66, 66, 66, 66–70
   StreamOperatorServiceWorker.cs77.78%50%75%80%46–50, 55, 63
   StreamOperatorServiceWorkerFactory.cs100%100%100%100%
src/Services/Repositories
   StreamClassRepository.cs80%0%75%85.29%35, 35, 35–39
   StreamDefinitionRepository.cs0%0%0%0%102–104, 104, 104–117, 121–128, 28–35, 38–40, 40, 40–55, 59–61, 61, 61–82, 85–87, 87, 87–99
   StreamingJobTemplateRepository.cs0%0%0%0%20–27, 31–33, 33, 33, 33, 33, 33, 33, 33, 33–36, 39–47
src/Services/Streams
   StreamingJobOperatorService.cs0%0%0%0%100–103, 106–109, 109, 109–111, 114–117, 117, 117–119, 122–125, 125, 125–127, 130–133, 136–145, 28–38, 41, 44–51, 55–56, 56, 56–59, 59, 59–66, 66, 66–86, 86, 86, 86, 86–89, 89, 89–99

@github-actions GitHub Actions
Copy link

Coverage after merging fix-race into main will be

64.48%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
src/Configurations
   MetricsPublisherActorConfiguration.cs100%100%100%100%
   MetricsReporterConfiguration.cs100%100%100%100%
src/Extensions
   JsonElementExtensions.cs0%100%0%0%22, 30
   V1JobExtensions.cs85.58%75%90.91%90.16%39, 39, 44, 60–61, 61, 61, 61, 61–64, 82, 82
src/Models
   StreamingClassOperatorResponse.cs100%100%100%100%
src/Models/StreamClass
   V1Beta1StreamClassSpec.cs100%100%100%100%
src/Models/StreamDefinitions/Base
   IStreamDefinition.cs90%83.33%100%100%29
src/Models/StreamStatuses/StreamStatus/V1Beta1
   V1Beta1StreamStatus.cs100%100%100%100%
src/Services/Maintenance
   HostedStreamingJobMaintenanceService.cs0%0%0%0%16–24, 27–29, 29, 29–36, 39–42
   StreamingJobMaintenanceService.cs86.71%72.50%100%91.67%103, 109–112, 120, 125, 131, 131, 134, 134, 134, 134–137, 141, 67, 72
src/Services/Metrics
   DeclaredMetrics.cs88.24%60%100%100%32, 32, 32, 32
   MetricsReporter.cs100%100%100%100%
src/Services/Metrics/Actors
   MetricsPublisherActor.cs85.71%50%100%86.05%90–92, 92, 92–95
src/Services/Models
   CustomResourceApiRequest.cs100%100%100%100%
   ResourceEvent.cs100%100%100%100%
src/Services/Operator
   StreamClassOperatorService.cs80.22%58.33%87.50%83.10%114–117, 79, 79, 83, 88–90, 90, 90–95
   StreamOperatorService.cs87.62%77.59%100%91.30%116–122, 127, 135, 149, 149, 155, 177, 65–66, 66, 66, 66, 66, 66, 66–70
   StreamOperatorServiceWorker.cs77.78%50%75%80%46–50, 55, 63
   StreamOperatorServiceWorkerFactory.cs100%100%100%100%
src/Services/Repositories
   StreamClassRepository.cs80%0%75%85.29%35, 35, 35–39
   StreamDefinitionRepository.cs0%0%0%0%102–104, 104, 104–117, 121–128, 28–35, 38–40, 40, 40–55, 59–61, 61, 61–82, 85–87, 87, 87–99
   StreamingJobTemplateRepository.cs0%0%0%0%20–27, 31–33, 33, 33, 33, 33, 33, 33, 33, 33–36, 39–47
src/Services/Streams
   StreamingJobOperatorService.cs0%0%0%0%101–104, 104, 104–106, 109–112, 112, 112–114, 117–120, 123–132, 28–38, 41, 44–51, 55–61, 61, 61–81, 81, 81, 81, 81–84, 84, 84–98

@s-vitaliy s-vitaliy merged commit d31392b into main May 15, 2024
2 checks passed
@s-vitaliy s-vitaliy deleted the fix-race branch May 15, 2024 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adelinag08 adelinag08 adelinag08 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@s-vitaliy @adelinag08