Skip to content

Dev environment tips

Jump to bottom
LachlanStuart edited this page Jan 13, 2022 · 4 revisions

SSH Settings

Create a file ~/.ssh/config:

AddKeysToAgent  yes
ServerAliveInterval 120 # This prevents the connection from timing out when there's no activity

Host prod
    Hostname metaspace2020.eu
    User ubuntu

# Repeat the "Host" section for other environments as needed.

Now you can type ssh prod instead of ssh ubuntu@metaspace2020.eu.

Protect your dev environment with a firewall

Docker Dev Environments use default credentials for postgres, redis, etc. which is a significant security issue. The best solution is to set up a firewall so that the Docker-exposed ports aren't publicly accessible by default. UFW is a great option on Linux.

sudo apt install ufw
sudo ufw enable
sudo ufw default deny incoming
sudo ufw allow 22 comment SSH
sudo ufw allow from 172.19.0.0/24 comment "Allow docker connections to host"
sudo ufw status verbose # Show rules

PyCharm file watcher for Black formatting

In Settings -> Tools -> File Watchers create a new watcher with these details:

  • Name: Black
  • File type: Python
  • Scope: Create a new scope called engine that recursively includes these directories:
    • /metaspace/metaspace/engine/migrations
    • /metaspace/metaspace/engine/tests
    • /metaspace/metaspace/engine/scripts
    • /metaspace/metaspace/engine/sm
    • /metaspace/metaspace/recal
    • /metaspace/metaspace/python-client/metaspace
  • Program: (path to the black executable. Run which black inside your conda environment to find it)
  • Arguments: --line-length 100 --skip-string-normalization --target-version py38 $FilePath$
  • Output paths to refresh: $FilePath$
  • Working directory: $ProjectFileDir$
    • Auto-save edited files to trigger the watcher
    • Trigger the watcher on external changes
    • Trigger the watcher regardless of syntax errors
    • Create output file from stdout
  • Show console: On error

Set up DataGrip to connect to the production databases

This requires you to have SSH access to the desired server. Our servers are configured to block external access to most ports, including the postgres port. However, DataGrip can be configured to first SSH to a proxy server (in this case the same server as the target) before connecting to the target server.

To do this, create a PostgreSQL data source with these values:

  • Host: localhost
  • Port: 5432
  • Authentication: User & Password
  • User and Password: (The easiest way to get these is just SSH in to the server and grab them from /opt/dev/metaspace/metaspace/engine/conf/config.json. They can also be found in the Ansible Vault files)
  • Database: sm (by default)

Then go to the SSH/SSL page, enable Use SSH tunnel, click the ... button and create a new SSH configuration with these values:

Host: (target server e.g. metaspace2020.eu) Port: 22 User name: ubuntu Local port: (leave blank so that it says <dynamic>) Authentication type: Key pair OpenSSH or PuTTY Private key file: (Select your key file)

Test Connection should now work. Note that you will need to set up a separate SSH configuration for each server you wish to connect to.

Additionally, for connections to production, it's a good idea to create two connections - one read-only (can be set on the Options tab) and one not read-only. This way you can use the read-only connection by default without having to worry about breaking anything, and only use the non-read-only connection when you're ready to make a modification.

Clone this wiki locally