FI-3338: Create common test kit specs

[Jammjammjamm]

Summary

This branch creates a set of shared examples which test kits can import to ensure that they meet the requirements to be deployed on inferno.healthit.gov.

Testing Guidance

Check out the fi-3338-test branch in the US Core Test Kit, bundle, and bundle exec rake to run these tests.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.30%. Comparing base (068ac08) to head (2b3d657).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #586   +/-   ##
=======================================
  Coverage   84.30%   84.30%           
=======================================
  Files         275      275           
  Lines       11686    11688    +2     
  Branches     1307     1307           
=======================================
+ Hits         9852     9854    +2     
  Misses       1824     1824           
  Partials       10       10           

Flag	Coverage Δ
backend	92.62% <100.00%> (+<0.01%)	⬆️
frontend	79.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[arscan]

We should also add in the requirement to have standard links (smart health cards forgot that in this first release).

[Shaumik-Ashraf]

bundle exec rake on us core branch fails:

An error occurred while loading ./spec/us_core/us_core_test_kit_spec.rb. - Did you mean?
                    rspec ./spec/us_core/search_test_spec.rb

Failure/Error: require_relative '../../../inferno/spec/shared/test_kit_examples'

LoadError:
  cannot load such file -- /Users/shaumikashraf/workspace/inferno/inferno/spec/shared/test_kit_examples
# ./spec/us_core/us_core_test_kit_spec.rb:1:in `require_relative'
# ./spec/us_core/us_core_test_kit_spec.rb:1:in `<top (required)>'

[Jammjammjamm]

bundle exec rake on us core branch fails:

This should be fixed now.

[Shaumik-Ashraf]

If I try adding in metadata, I get the following errors:

Failures:

  1) USCoreTestKit behaves like platform_deployable_test_kit TestKit defines test kit in the Metadata class
     Failure/Error: DatabaseCleaner.cleaning { example.run }
       expected USCoreTestKit::Metadata to be a kind of Inferno::Entities::TestKit
     Shared Example Group: "platform_deployable_test_kit" called from ./spec/us_core/us_core_test_kit_spec.rb:2
     # ./spec/spec_helper.rb:113:in `block (3 levels) in <top (required)>'
     # ./spec/spec_helper.rb:113:in `block (2 levels) in <top (required)>'

  2) USCoreTestKit behaves like platform_deployable_test_kit TestKit relies on the test kit version rather than defining the version in the suites
     Failure/Error: DatabaseCleaner.cleaning { example.run }
     
     TypeError:
       no implicit conversion of Array into String
     Shared Example Group: "platform_deployable_test_kit" called from ./spec/us_core/us_core_test_kit_spec.rb:2
     # ./spec/spec_helper.rb:113:in `block (3 levels) in <top (required)>'
     # ./spec/spec_helper.rb:113:in `block (2 levels) in <top (required)>'

Finished in 1.1 seconds (files took 3.88 seconds to load)
166 examples, 2 failures

I tried the following metadata and gemspec which I think are as intended, you can see the full diff at inferno-framework/us-core-test-kit#222:

# metadata.rb
require_relative 'version'

module USCoreTestKit
  class Metadata < Inferno::TestKit
    id :us_core_test_kit
    title 'US Core Test Kit'
    description <<~DESCRIPTION
      The US Core Test Kit validates the conformance of a server implementation to a specific
      version of the [US Core IG](http://hl7.org/fhir/us/core). Currently, Inferno can test
      against implementations of following versions of the US Core IG:
      [v3.1.1](http://hl7.org/fhir/us/core/STU3.1.1/),
      [v4.0.0](http://hl7.org/fhir/us/core/STU4/),
      [v5.0.1](http://hl7.org/fhir/us/core/STU5.0.1/),
      [v6.1.0](https://hl7.org/fhir/us/core/STU6.1/),
      and
      [v7.0.0](https://hl7.org/fhir/us/core/STU7/).
      <!-- break -->

      TODO grab full us core description 
    DESCRIPTION
    suite_ids [:us_core_v311, :us_core_v400, :us_core_v501, :us_core_v610, :us_core_v700]
    tags ['SMART App Launch', 'US Core']
    last_updated '2024-12-05'
    version VERSION
    maturity 'Low'
    authors 'inferno-gang@mitre.org'
    repo 'https://github.com/inferno-framework/us-core-test-kit'
  end
end

# us_core_test_kit.gemspec
require_relative 'lib/us_core_test_kit/version'

Gem::Specification.new do |spec|
  spec.name          = 'us_core_test_kit'
  spec.version       = USCoreTestKit::VERSION
  spec.authors       = ['Stephen MacVicar']
  spec.email         = ['inferno@groups.mitre.org']
  spec.summary       = 'US Core Inferno tests'
  spec.description   = 'US Core Inferno tests'
  spec.homepage      = 'https://github.com/inferno-framework/us-core-test-kit'
  spec.license       = 'Apache-2.0'
  # spec.add_runtime_dependency 'inferno_core', '>= 0.5.0'
  spec.add_runtime_dependency 'smart_app_launch_test_kit', '>= 0.4.6'
  spec.add_runtime_dependency 'tls_test_kit', '~> 0.2.0'
  spec.add_development_dependency 'database_cleaner-sequel', '~> 1.8'
  spec.add_development_dependency 'factory_bot', '~> 6.1'
  spec.add_development_dependency 'rspec', '~> 3.10'
  spec.add_development_dependency 'webmock', '~> 3.11'
  spec.required_ruby_version = Gem::Requirement.new('>= 3.1.2')
  spec.metadata['inferno_test_kit'] = 'true'
  spec.metadata['homepage_uri'] = spec.homepage
  spec.metadata['source_code_uri'] = 'https://github.com/inferno-framework/us-core-test-kit'
  spec.files         = `[ -d .git ] && git ls-files -z lib config/presets LICENSE`.split("\x0")

  spec.require_paths = ['lib']
end

[Jammjammjamm]

Ok, those should be fixed. Note that the second failure isn't related to the gemspec. It's saying you should get rid of the version whatever calls in the suites.

[Shaumik-Ashraf]

Thanks for the tip. It works!