CCQ (Check if your client qualifies for legal aid)

This is a calculator for legal aid providers to quickly check if a client is eligible for legal aid.

CCQ is effectively a front end for CFE Civil, which contains all eligibility logic. CCQ only knows about the specifics of the eligibility ruleset to the extent that this knowledge is needed in order to be able to ask the right questions.

Currently CCQ only enables checks relating to civil legal aid; however a future roadmap ambition is to also provide checks for criminal legal aid.

Documentation for developers.

Dependencies

• Ruby version

  • Ruby version 3.3.4
  • Rails 7.1.x

• System dependencies

  • postgres
  • yarn
  • pdftk

Setting up the app

You can use Homebrew to install any dependencies. The Homebrew documentation has lots of useful commands.

If using Homebrew:

Install PostgreSQL. You will need to select a version if using brew. Run the below command, changing the version number as appropriate:

brew install postgresql@14

You will be prompted on the command line to start the server with something like:

brew services start postgresql@14

You will also need pdftk. There is a Mac installer for convenience.

bundle install

Create the development and test databases and run migrations

bundle exec rails db:create
bundle exec rails db:migrate:with_data
bundle exec rails db:migrate:with_data RAILS_ENV=test

Install Yarn (you can use Homebrew for this) and run the below:

brew install yarn
yarn install
yarn build
yarn build:css

Running locally

To change settings for your local development environment, copy your .env.sample file to a new file and rename it to .env.development.

To run the server locally, you can use bundle exec rails s or bin/dev. The latter will automatically rebuild JS and CSS every time they change, although it also has more verbose console output that can make debugging with binding.p harder.

Tests

We test with RSpec and enforce 100% line and branch coverage with SimpleCov.

You can run tests with the command:

bundle exec rspec

Unit tests

Form tests

Form test files are held in spec/forms. Form tests are RSpec feature specs, with each test file describing the behaviour of a given form screen. Every form has form tests.

The purpose of form tests is to demonstrate that a given form screen performs the correct validation on data entered, and if data entered passes validation, that the correct information is stored in the session. If the structure or copy of a form is affected by the content of the session, we test that too in these specs.

Since feature specs don't normally provide session access, we use the rack_session_access gem.

CfeService tests

CfeService test files are held in spec/services, and there is one for each of the various SubmitXService classes. These tests comprehensively describe the behaviour of these classes, by providing session data as input and setting expectations on what methods get called on CfeConnection and what arguments are passed to it.

CfeConnection tests

CfeConnection is tested in spec/services/cfe_connection_spec.rb. It validates that for each method on CfeConnection, whatever gets passed in gets turned into an appropriate HTTP request. We set expectations with stub_request calls.

Result screen tests

Result screen tests are held in spec/views/results/ mock a response payload from CFE and set expectations for what appears on the results screen accordingly. They comprehensively test what content gets displayed based on the eligibility outcome.

Integration tests

The version of chromedriver has to be constantly updated to keep pace with chrome - we use chrome to generate PDFs, so switching to firefox is not an option. After performing 'brew upgrade chromedriver' the first call will produce an warning dialog about trusting the binary. This can be suppressed with:

xattr -d com.apple.quarantine /opt/homebrew/bin/chromedriver

UI flow tests

UI flow tests are held in spec/flows, and are RSpec feature specs. Each test describes a different journey from the start page to the check answers page, making explicit what screens are reached. These flows do not explore validation errors (which are covered in form tests), or data passed to CFE. They do specify explicitly which screens are filled out in what order, although they do not specify how the screens are filled out (instead this is delegated to helper functions) except to the extent that it affects the flow. There are flows for:

• Similar parallel pathways such as controlled vs certificated
• Collections of screens that are toggled or skipped in response to an answer given on a previous screen, such as partner questions, passporting questions etc
• The effect of specific feature flags
• Looping flows like the 'add another' benefits journey
• The check answers journey, both simple loops and more complex ones caused by changing answers in the check answers flow
• The effect of using the back button and changing answers

End-to-end tests

End-to-end tests are held in spec/end_to_end and are RSpec feature specs. Each test describes a journey from start page to result screen, describing the values that are filled on each screen along the way. Some tests test, via HTTP stubs, the values that are sent to CFE when loading the result screen. Some actually send the payload to CFE and test what appears on the results screen, but these must be explicitly enabled by calling bundle exec rspec -t end2end

There are end-to-end tests to cover the main categories of journey through the site, but the end-to-end tests are not intended to be comprehensive.

System tests

"System" tests are held in /spec/system, and are where we hold tests that involve running our javascript.

The above is not a comprehensive list of tests, and we are pragmatic about how best to test any given piece functionality. Classes not comprehensively exercised by the above tests get their own unit tests.

Significant libraries we use

Sentry

Sentry is a realtime application monitoring and error tracking service. The service has separate monitoring for UAT/dev, Staging and Production.

New error messages can be added using the Sentry.capture_exception() or Sentry.capture_message() methods:

def test_sentry
  begin
    1 / 0
  rescue ZeroDivisionError => exception
    Sentry.capture_exception(exception)
   end
end

or

Sentry.capture_message("This is the error message that is sent to Sentry")

Feature flags

For "static" feature flags we set the flag values in env vars. To add a new feature flag, set a "#{flag_name.upcase}_FEATURE_FLAG" env var with value "ENABLED" in all environments where you want the flag enabled. Then add flag_name to the list of flags in app/lib/feature_flags.rb.

When adding a flag_name to the list of static flags, you will need to decide if this is a "global" flag i.e. always taken from the env var and not the session, or a "session" flag i.e. taken from the session_data of the check.

We introduced this as a way of making our feature flags 'backwards compatible' - if a user check is underway during the switch-on of a flag, their user journey will not be affected by any flag-related changes. This is because we use their session_data to determine the value of the flag that was set at the start of their check.

To use the feature flag in your code, call FeatureFlags.enabled?(:flag_name, session_data). For cases where you are not able to pass in session_data e.g. on the start page, call FeatureFlags.enabled?(:flag_name, without_session_data: true).

In tests, you can temporarily enable a feature flag by setting the ENV value. However, flags are not reset between specs, so to avoid polluting other tests use an around block and change the ENV value back once the test has run.

We also have time-dependent flags, defined in app/lib/feature_flags.rb, which default to disabled but also have a date associated. They will be enabled on the associated date.

When the FEATURE_FLAG_OVERRIDES env var is set to enabled, it is possible to use the /feature-flags endpoint to set database values that override env values for both static and time-based feature flags. The username is "flags" and the password is a secret stored alongside our other secrets in K8s.

Saving as PDF

We use Grover to save pages as PDF files for download, which in turn uses Puppeteer. For it to work, the app needs to make HTTP requests to the app, to load assets. This means that it only works in a multi-threaded environment. To run the app multithreadedly in development mode, set the MULTI_THREAD environment variable, e.g.:

MULTI_THREAD=1 bundle exec rails s

PDF Accessibility

When generating PDFs from an eligibility check, we found that the iOS screenreader, was having difficulty focussing on <h2> and <p> html elements on a mobile/tablet screen.

To combat this we replaced <h2> & <p> html elements, with <li> elements and nested them either in a <h2> or <ul>structure. Helper methods have been created in results_helper.rb, to construct these new html elements, remove stylings and only show them when a PDF is generated.

The iOS screenreader, was also having difficulty announcing the numbers in our tables. To combat this, we created the pdf_friendly_numeric_table_cell method which uses the govuk-!-text-align-right override class, instead of using the numeric: true class (for a cell with a number in it).

Call this method (with the relevant arguments, for this method’s parameters) anytime you want to create a table cell with a number in it, for the use in a PDF.

Retrieving user-entered data for a given check

User-entered values are stored in the session. However, rather than retrieve values directly from the session, most places retrieve them from associated model objects and helpers, of which there is a hierarchy:

Steps::Logic contains methods that directly interrogate a session object for a few specific attributes that affect navigation flow through the form. It knows how answers to certain questions affect the relevance of certain other questions.

Steps::Helper uses Steps::Logic to determine which screens, or steps, should be displayed for a given check, based on the answers provided so far.

Flow::Handler knows, for any given step, which Form object to populate to back the HTML form displayed on screen

Check provides access to all relevant data for a check. For any attribute it uses Steps::Helper and Flow::Handler to determine whether, given the other answers supplied, the attribute is relevant. If not, when asked for that attribute it will return nil. Otherwise it will return that attribute.

I18n

We keep all user-facing content strings in locale files. In particular, config/locales/en.yml contains nearly every piece of text on the site. We have a utility to help identify obsolete content built into our test suite. You can run CHECK_UNUSED_KEYS=true bundle exec rspec and it will print out, at the end of the test run, all keys found in en.yml that don't get looked up by the test suite. Using this periodically can help remove stale content from that file.

Deploying to UAT/Staging/Production

The service uses helm to deploy to Cloud Platform Environments via CircleCI. This can be installed using:

brew install helm

To view helm deployments in a namespace the command is:

helm -n <namespace> ls --all

e.g. helm -n laa-check-client-qualifies-uat ls --all

Deployments can be deleted by running:

helm delete <name-of-deployment>

e.g. helm -n laa-check-client-qualifies-uat delete el-351-employment-questions-pa

It is also possible to manually deploy to an environment from the command line, the structure of the command can be found in bin/uat_deployment

Secrets

We keep secrets in AWS Secrets Manager. To edit them, visit the AWS web console. The following secrets are currently stored in a secret called "aws-secrets" in each namespace we use:

• NOTIFICATIONS_API_KEY
• SECRET_KEY_BASE
• BASIC_AUTH_PASSWORD
• GOOGLE_OAUTH_CLIENT_SECRET

The current values for these are available as secure notes in 1Password for each environment, should they be lost from Kubernetes.

Portal integration

Secrets

There is only 1 of these (the secret key) which is kept in k8s 'portal_secrets' secret This is a single key X509_KEY which contains the secret key for the certificate used to authenticate with Portal.

Last parameter is 'no DES' (Data Encryption Standard) (rather than nodes) which means don't store a passphrase against it
https://en.wikipedia.org/wiki/Data_Encryption_Standard

    openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 3650 -nodes

    kubectl -n <namespace> create secret generic portal-secrets --from-file=X509_KEY=./private-key.pem

where private-key.pem is the private key creeated via openssl: 

Metadata files

The metadata files are required by the portal team before they can do their integration - this is a little chicken-and-egg situation as the metadata files are produced by our Rails code at the url :/providers/auth/saml/metadata

Once you have the xml metadata (from visiting :/providers/auth/saml/metadata), copy and paste the metadata into appropriate file (uat.xml or stg.xml etc). Remove the errant "..." from the file and format it with your IDE.

Then let the portal team know that you branch the new metadata for portal to "point back to". Once that is done, you can test out on this UAT branch: https://portal.uat.legalservices.gov.uk

We also store the portal metadata files in our repository - they are in the Portal GitHub, but their repo is private so we can't easily get to them without creating a github access key - it was easier just to copy and paste their config files although this could be a potential future option - the code we inherited from crime apply does have the capability of loading a Portal metadata file from a URL.

Manual Puppeteer upgrade

The application uses puppeteer as part of its testing pipeline - namely as part of the browser tools dockerfile. This is pinned to a specific puppeteer version, but because Chrome updates quite regularly, we have to manually update this when a new release comes out.

Here is an example PR of what the update looks like: https://github.com/ministryofjustice/laa-check-client-qualifies/pull/1482/files

Note we use a custom image inside browser tools dockerfile - when you create the branch with the puppeteer upgrade, you'll also need to add the branch name inside the YAML file that pushes the changes to Docker (browser_tools_docker_image.yml), and update the CircleCI config accordingly.

You can see our custom Docker image here - this will update once you've pushed a new image: https://hub.docker.com/r/checkclientqualifiesdocker/circleci-image/tags

Steps to follow are:

1. create an appropriately named branch referencing the puppeteer version upgrade i.e. puppeteer-22**
2. update Dockerfile_browser_tools.dockerfile & package.json with the new puppeteer version
3. run yarn install to update yarn.lock
4. add your branch name to .github/workflows/browser_tools_docker_image.yml so the new image gets pushed to Dockerhub
5. update .circleci/config.yml to reference the new image

Branch naming

We name our branches to start with the Jira ticket ID, followed by a short description of the work.

Due to case-sensitivity in the integration between CircleCi and Jira, the Jira ticket ID needs to be uppercase so that it exactly matches how it is on Jira. For example: ❌ el-123-add-new-feature ✅ EL-123-add-new-feature