Home

Price Paid Standard Reports UI

This project provides a straighforward user experience for users who want to define and download a Price Paid Standard Report. Standard Reports aggregate various slices of Price Paid (PPD) data, according to property type, region and date. The basic workflow is fairly simple: users step through a wizard interface making selections for the various report criteria, then at the end they click to download the selected report which will kick off a background batch job, using our report generator API. This batch job will either return with the report download link, if a report with those options has already been generated and cached, or a queue position as the job moves up the batch queue.

Please see the other repositories in the HM Land Registry Open Data project for more details.

Operator Notes

Runtime Configuration environment variables

We use a number of environment variables to determine the runtime behaviour of the application:

name	description	default value
API_SERVICE_URL	The base URL from which data is accessed, including the HTTP scheme eg.	None
	http://localhost:8081 if running a standard-reports-manager service locally
	http://standard-reports-manager:8080 if running a standard-reports-manager docker image locally
SECRET_KEY_BASE	See description.
	For development mode a acceptable value is already configured, in production mode this should be set to the output of rails secret.
	This is handled automatically when starting a docker container, or the server make target
SENTRY_API_KEY	The DSN for sending reports to the PPD Sentry account	None

Prometheus monitoring

Prometheus is set up to provide metrics on the /metrics endpoint. The following metrics are recorded:

• api_status (counter) Response from data API, labelled by response status code
• api_requests (counter) Count of requests to the data API, labelled by result success/failure
• api_connection_failure (counter) Could not connect to back-end data API, labelled by message
• api_service_exception (counter) The response from the back-end data API was not processed, labelled by message
• internal_application_error (counter) Unexpected events and internal error count, labelled by message
• memory_used_mb (gauge) Process memory usage in megabytes
• process_threads (gauge) The number of currently threads, labelled by status:
  • "aborting": If this thread is aborting
  • "sleep": Returned if this thread is sleeping or waiting on I/O
  • "run": When this thread is executing
  • "false": When this thread is terminated normally
  • "nil": If the thread is terminated with an exception
• api_response_times (histogram) Histogram of response times of successful API calls.

Internally, we use ActiveSupport Notifications to emit events which are monitored and collected by the Prometheus store. Relevant pieces of the app include:

• config/initializers/prometheus.rb Defines the Prometheus counters that the app knows about, and registers them in the metrics store (see above)
• config/initializers/load_notification_subscribers.rb Some boiler-plate code to ensure that all of the notification subscribers are loaded when the app starts
• app/subscribers Folder where the subscribers to the known ActiveSupport notifications are defined. This is where the transform from ActiveSupport::Notification to Prometheus counter or gauge is performed.

In addition to the metrics we define, there is a collection of standard metrics provided automatically by the Ruby Prometheus client

To test Prometheus when developing locally, there needs to be a Prometheus server running. Tip for Linux users: do not install the Prometheus apt package. This starts a locally running daemon with a pre-defined configuration, which is useful when monitoring the machine on which the server is running. A better approach for testing Prometheus is to download the server package and run it locally, with a suitable configuration. A basic config for monitoring a Rails application is provided in test/prometheus/dev.yml.

Using this approach, and assuming you install your local copy of Prometheus into ~/apps, a starting command line would be something like:

~/apps/prometheus/prometheus-2.32.1.linux-amd64/prometheus \
  --config.file=test/prometheus/dev.yml \
  --storage.tsdb.path=./tmp/metrics2

Something roughly equivalent should be possible on Windows and Mac as well.

Developer Notes

• Additional Information
  • Coding standards
  • Tests
  • Issues
  • Deployment
    • entrypoint.sh features
• Running the app
  • Locally
    • Running Rails as a server with a sub-directory via Makefile
  • As a docker container
  • Development and Production mode
• Running the Standard Reports Manager
  • Building and running from standard reports manager repository
  • Running an existing ECR image
  • Running as a docker container