Skip to content

Latest commit

 

History

History
230 lines (172 loc) · 13.4 KB

SETUP.md

File metadata and controls

230 lines (172 loc) · 13.4 KB
Raw

Server Architecture

The Server is divided into a number of services. Each service is a Visual Studio project in the Server solution. These are:

  • Admin
  • Api
  • Icons
  • Identity
  • Notifications
  • SQL

Each service is built and run separately. The Bitwarden clients can use different servers for different services.

This means that you don't need to run all services locally for a development environment. You can run only those services that you intend to modify, and use Bitwarden.com or a self-hosted instance for all other services required.

Local Development Environment Setup

This guide will show you how to set up the Api, Identity and SQL projects for development. These are the minimum projects for any development work. You may need to set up additional projects depending on the changes you want to make.

We recommend using Visual Studio.

Database setup

Docker setup

We use SQL Server Developer Edition as a Docker Container to run the local development database. More information about the container image is available on its Docker Hub page (this is especially useful if you're having issues).

Steps:

  1. Make sure you have already run git clone for the server repo, so that you have the migrator scripts required

  2. Install the Docker desktop runtime

  3. Create a Docker account if prompted (optional)

  4. Organize your folders (important for the below scripts to work properly):

    • Create a folder to house Docker information (e.g. docker) - this should be in the same folder as your cloned repositories (e.g. server, web, browser etc)
    • Create sub-folder to house this specific container (ex: docker/SQLServer)

  5. Create docker-compose.yml file in docker/SQLServer with the following contents:

    version: '3.1'

services:

  db:
    image: mcr.microsoft.com/mssql/server:2017-CU14-ubuntu
    container_name: mssql-dev
    restart: always
    environment:
      ACCEPT_EULA: Y
      SA_PASSWORD: SET_A_PASSWORD_HERE
      MSSQL_PID: Developer
    volumes:
      - mssql_dev_data:/var/opt/mssql/data
      - ../../server/util/Migrator:/mnt/migrator/
    ports:
      - '1433:1433'

volumes:
  mssql_dev_data:

  6. Update the SA_PASSWORD field with a password you want to use. It must include at least 8 characters of at least three of these four categories: uppercase letters, lowercase letters, numbers and non-alphanumeric symbols.

  7. Open up a terminal window and cd into your folder with the .yml file and execute the following command

    • docker-compose up -d (omit the -d switch if you want to see the output for debugging)
  • You should now have a container up and running the SQL Server

salserver

Running Migrator scripts

You now have an empty SQL server instance. The instructions below will automatically create your vault_dev database and run the migration scripts located in server/util/Migrator/DbScripts to populate it.

Note: you must have followed the steps above to set up your folder structures and the docker-compose file for this to work.

  1. Open the Docker Desktop GUI

  2. Click the CLI button to open a new terminal in your mssql-dev service Screen Shot 2021-03-18 at 11 12 30 am

  3. Run sh /mnt/migrator/createVaultDev.sh 'SA_PASSWORD'. Replace SA_PASSWORD with the password you set in your docker-compose.yml file. You should put your SA_PASSWORD in single quotes to avoid special bash characters being caught by the shell.

  4. (Optional) To confirm this worked correctly, you can connect to the SQL database with an SQL client (such as Azure Data Studio). You should see that the vault_dev database has been created and has been populated with tables.

  5. Troubleshooting: if your login details for sa are being rejected:

    • Make sure your SA_PASSWORD is meeting the complexity requirements above. If these requirements are not met, the password may not be set properly (without any warning) and your login attempts will be rejected for having an incorrect password. If this is happening and you're sure you're using the right password, try increasing the complexity of SA_PASSWORD.
    • If you change SA_PASSWORD in docker-compose.yml, you may need to delete the Docker container and volume for it to take effect. (This will obviously delete all of your container files/setup.) Stop and delete the running container, then delete the volume with docker volume ls and docker volume rm <volume name>. Then update docker-compose.yml and run docker compose up -d again.
    • Make sure you are wrapping your SA_PASSWORD in single quotes when executing the createVaultDev.sh script.

Your database is now ready to go!

Setting up User Secrets

User secrets are a method for managing application settings on a per-developer basis. They are stored outside of the local git repository so that they are not pushed to remote.

User secrets override the settings in appsettings.json of each project. Your user secrets file should match the structure of the appsettings.json file for the settings you intend to override.

For more information, see: Safe storage of app secrets in development in ASP.NET Core.

Open the server solution file (bitwarden-server.sln) in Visual Studio before proceeding.

Editing user secrets - Visual Studio on Windows

Right-click on the project in the Solution Explorer and click Manage User Secrets.

Editing user secrets - Visual Studio on macOS

Open a terminal and navigate to the project directory. Once there, initiate and create the blank user secrets file by running:

dotnet user-secrets init

Add a user secret by running:

dotnet user-secrets set "<key>" "<value>"

View currently set secrets by running:

dotnet user-secrets list

By default, user secret files are located in ~/.microsoft/usersecrets/<project name>/secrets.json. After the file has been created, you can edit this directly with VSCode, which is much easier than using the command line tool.

Editing user secrets - Rider

  • Navigate to Preferences -> Plugins and Install .NET Core User Secrets
  • Right click on the a project and click Tools > Open project user secrets

User Secrets - Certificates

Once you have your user secrets files set up, you'll need to generate 3 of your own certificates for use in local development.

This guide uses OpenSSL to generate the certificates. If you are using Windows, pre-compiled OpenSSL binaries are available via Cygwin.

  1. Open a terminal.
  2. Create an Identity Server (Dev) certificate file (.crt) and key file (.key): 
    openssl req -x509 -newkey rsa:4096 -sha256 -nodes -keyout identity_server_dev.key -out identity_server_dev.crt -subj "/CN=Bitwarden Identity Server Dev" -days 3650
  3. Create an Identity Server (Dev) .pfx file based on the certificate and key you just created. You will be prompted to enter a password - remember this because you’ll need it later: 
    openssl pkcs12 -export -out identity_server_dev.pfx -inkey identity_server_dev.key -in identity_server_dev.crt -certfile identity_server_dev.crt
  4. Create a Data Protection (Dev) certificate file (.crt) and key file (.key): 
    openssl req -x509 -newkey rsa:4096 -sha256 -nodes -keyout data_protection_dev.key -out data_protection_dev.crt -subj "/CN=Bitwarden Data Protection Dev" -days 3650
  5. Create a Data Protection (Dev) .pfx file based on the certificate and key you just created. You will be prompted to enter a password - remember this because you’ll need it later: 
    openssl pkcs12 -export -out data_protection_dev.pfx -inkey data_protection_dev.key -in data_protection_dev.crt -certfile data_protection_dev.crt
  6. Install the .pfx files by double-clicking on them and entering the password when prompted.
    • On Windows, this will add them to your certificate stores. You should add them to the "Trusted Root Certificate Authorities" store.
    • On MacOS, this will add them to your keychain. You should update the Trust options for each certificate to always trust.
  7. Get the SHA1 thumbprint for the Identity and Data Protection certificates
    • On Windows
      • press Windows key + R to open the Run prompt
      • type "certmgr.msc" and press enter. This will open the system tool used to manage user certificates
      • find the "Bitwarden Data Protection Dev" and "Bitwarden Identity Server Dev" certificates in the Trusted Root Certificate Authorities > Certificates folder
      • double click on the certificate
      • click the "Details" tab and find the "Thumbprint" field in the list of properties.
    • On MacOS
      • press Command + Spacebar to open the Spotlight search
      • type "keychain access" and press enter
      • find the "Bitwarden Data Protection Dev" and "Bitwarden Identity Server Dev" certificates
      • select each certificate and click the "i" (information) button
      • find the SHA-1 fingerprint in the list of properties
  8. Add the SHA1 thumbprints of both certificates to your user secrets for the Api and Identity projects. (See the example user secrets file below.)

User Secrets - Other

selfhosted: It is highly recommended that you use the selfHosted: true setting when running a local development environment. This tells the system not to use cloud services, assuming that you are running your own local SQL instance.

Alternatively, there are emulators that allow you to run local dev instances of various Azure and/or AWS services (e.g. local-stack), or you can use your own Azure accounts for provisioning the necessary services and set the connection strings accordingly. These are outside the scope of this guide.

sqlServer__connectionString: this provides the information required for the Server to connect to the SQL instance. See the example connection string below.

licenseDirectory: this must be set to avoid errors, but it can be set to an aribtrary empty folder.

installation__key and installation__id: request your own private Installation Id and Installation Key for self-hosting: https://bitwarden.com/host/.

Example User Secrets file

This is an example user secrets file for both the Api and Identity projects.

{
  "globalSettings": {
    "selfHosted": true,
    "identityServer": {
      "certificateThumbprint": "<your Identity certificate thumbprint with no spaces>"
    },
    "dataProtection": {
      "certificateThumbprint": "<your Data Protection certificate thumbprint with no spaces>"
    },
    "installation": {
      "id": "<your Installation Id>",
      "key": "<your Installation Key>"
    },
    "licenseDirectory": "<full path to licence directory>",
    "sqlServer": {
      "connectionString": "Server=localhost;Database=vault_dev;User Id=sa;Password=<your SA_PASSWORD>"
    }
  }
}

Running and Debugging

After you have completed the above steps, you should be ready to launch your development environment for the Api and Identity projects.

Visual Studio

To debug:

  • On Windows, right-click on each project > click Debug > click Start New Instance
  • On MacOS, right-click each project > click Start Debugging Project

To run without debugging, open a terminal and navigate to the location of the .csproj file for that project (usually in src/ProjectName). Start the project with:

dotnet run

NOTE: check the output of the running project to find the port it is listening on. If this is different to the default in appsettings.json, you may need to update your user secrets to override this (typically the Api user secrets for the Identity URL).

Rider

From within Rider, launch both the Api project and the Identity project by clicking the "play" button for each project separately.

Testing your deployment

Troubleshooting

  • If you get a 404 error, the projects may be listening on a non-default port. Check the output of your running projects to check the port they are listening on.