Skip to content

Latest commit

 

History

History
136 lines (95 loc) · 6.7 KB

qa-samples-proposal.md

File metadata and controls

136 lines (95 loc) · 6.7 KB
Raw
title authors reviewers approvers creation-date last-updated status
QA Samples Proposal
@camilamacedo86
TBD
TBD
2019-11-22
2019-11-22
implementable

QA Samples Proposal

Release Signoff Checklist

  • Enhancement is implementable
  • Design details are appropriately documented from clear requirements
  • Test plan is defined
  • Graduation criteria for dev preview, tech preview, GA
  • User-facing documentation is created in operator-sdk/doc

Summary

The Operator SDK has a repo with sample projects. This proposal describes quality improvements in for the sample projects.

Motivation

Goals

  • As a maintainer, I would like to ensure that samples projects continue to work after the changes performed in the PR are applied
  • As an operator developer, I would like to know how to cover the projects with tests
  • As an operator developer, I would like to know how to use the test-frameworkto do unit and integration tests
  • As an operator developer, I would like to know how to call the tests in the CI and integrate them with Travis
  • As an operator developer, I would like to know good practices to ensure the quality of my operators projects

NOTE The above goals are valid for the 3 types/projects Go, Ansible and Helm

Non-Goals

  • Change the code business logic implementation of the projects

Proposal

  • Cover the projects with unit and integration tests.
  • Integrate projects with Travis
  • Integrate projects with Coveralls

User Stories

Unit tests for Go Memcached Sample

  • I as a GO operator developer, I would like to know how to cover the projects with unit tests using the test-framework

Acceptance Criteria

  • The GO project should have a minimum of 70% of its implementation covered by unit tests.
  • The tests should all pass.
  • The project should have a makefile target to call the tests
  • A section with a short info over how to tests and the links for its documents in the README of the project
  • The tests should be done using the default GO testing lib and the test-frameworkprovided by SDK

NOTES

  • See here an example of how to cover the controller.
  • See here an example of how to create the fake client.

e2e tests for Go Memcached Sample

  • I as a Go dev operator user, I'd like to know how to cover the projects with integration tests using the test-framework

Acceptance Criteria

  • The GO project should be covered with a few basic integration tests
  • The tests should all pass.
  • The project should have a makefile target to call the integration test
  • A section with a short info over how to tests and the links for its documents in the README of the project
  • The tests should be done using the default GO testing lib and the test-framework provided by SDK

Unit tests for Ansible Memcached Sample

  • As an Ansible operator developer, I would like to know how to cover the projects with unit tests using molecule

Acceptance Criteria

  • The Ansible project should be covered by tests using molecule which by default is scaffold
  • The tests should all pass.
  • A section with a short info over how to tests and the links for its documents in the README of the project
  • The project should have a makefile target to call the tests

tests for Helm Memcached Sample

  • As an Helm operator developer, I would like to how to cover the projects with tests

Acceptance Criteria

  • The Helm project should have test shell scripts as seen here.- The tests should all pass.
  • A section with a short info over how to tests and the links for its documents in the README of the project
  • The project should have a makefile target to call the tests

e2e tests for Ansible Memcached Sample

  • As an Ansible operator developer, I'd like to how to cover the projects with integration tests using molecule

Acceptance Criteria

  • The Ansible project should have a few integration tests using molecule which by default is scaffold
  • The tests should all pass.
  • A section with a short info over how to tests and the links for its documents in the README of the project
  • The project should have a makefile target to call the tests

Travis Integration

  • As an operator-sdk maintainer, I would like ensure that samples continue to work after the changes performed in the PR are applied.
  • As an operator developer, I would like to know how to call the tests in the CI and integrate them with Travis

Acceptance Criteria

  • The Ansible, Helm and Go project should be integrated with Travis
  • All PRs against the master branch should trigger the CI
  • The unit and integration tests should be checked in the CI
  • The CI should fail if one of the tests do not pass

Coveralls Integration

  • As an operator developer, I'd like to know good practices to ensure the quality of my operators projects

Acceptance Criteria

  • Ansible and GO Memcached samples should be integrated with Coveralls
  • All above stories should be achieved successfully

Risks and Mitigations

Coveralls may not work well with molecule, if this is the case we can just not integrate with it or we can find a similar tool.