Techie Relationship Management

The Techie Relationship Management (TRM) project at TechLabs provides a data platform to store & analyze all information we have about our Techies (participants). We want to know who they are and how they're doing throughout their journey to become a Digital Shaper.

This project aims to provide a standardized platform across TechLabs locations. It integrates with existing tools within the TechLabs organization, such as TypeForm, GSuite, and Slack. The TRM Dashboard is a web application for the local journey teams to view, edit, and analyze the data for their Techies to get actionable insights for improving the TechLabs experience.

Resources

• Techie Relationship Management page on Notion – used for scope & project planning, to-dos, and meeting notes
• Slack Channel on the TechLabs MGMT Slack workspace

Architecture

As for the main components, we have Hasura, the TRM Dashboard, and some serverless functions for integrating external providers.

• The Hasura GraphQL engine provides a GraphQL API used by the TRM dashboard and provider integrations.
• The TRM Dashboard is a React single-page web application (SPA) based on the react-admin framework. See /frontend/ for the dashboard source code.
• The serverless functions in /functions/ integrate the TRM project with external providers, such as for authentification with Google, retrieving form responses from TypeForm, and fetching activity data from edyoucated & Slack.
• PostgreSQL is used for data storage & retrieval

Development

We're happy that you want to work with us to make the TRM even more awesome! 🎉

Try to work your way to the documentation & setup steps outlined below. If you have any questions or are stuck with the setup, do not hestitate to call for help on Slack! 😌

Technical Onboarding

You should familiarize yourself with the architecture as described in the Architecture section. In particular, have a look what Hasura and react-admin are, and how they integrate with the GraphQL API.

This is a list of relevant resources like documentation, books, and conference talks:

• Introduction to Hasura with React & Apollo

  The TRM uses this exact stack in combination with react-admin as a user interface & frontend data management framework.

• Introduction to GraphQL with Hasura

  This introduction to GraphQL and how to use it with Hasura will familiarize yourself with the way React & Hasura play together behind the scenes. You will create a basic todo app and see how Hasura works with foreign key relationships, authorization, authentication and custom business logic.

  The TRM uses Google Auth (provided by the /functions/auth serverless function) instead of Auth0, but the JSON Web Token (JWT) issuance & verification is the same.

  We use remote schemas and event triggers to integrate with Google Sheets, Typeform, and edyoucated.

• React-Admin 30-minute tutorial

  Marmelab's 30-minute react-admin tutorial will show you how react-admin works in combination with a simple placeholder data source. You'll see how to map API resources to the frontend, and how to customize the frontend.

  The TRM will use Apollo with the Hasura GraphQL API for data storage & retrieval, but working with a simple JSON API first may be an easier way with less moving parts to learn react-admin.

• YouTube Guides

  If you're more into watching intro videos, have a look at these tutorials on YouTube:

  • Scott Tries Hasura - A Realtime GraphQL API Builder

  • React Admin Crash Course

  • Hasura, GraphQL, React, Apollo – Part 1 and Part 2

If you feel like you have a general understanding how it works together, proceed to setup the development environment on your local machine as explained below.

Requirements

Must-Have:

• Docker (Desktop) – Docker for Mac / Docker for Windows

May-Have:

• NodeJS v13.9.0 or higher
• Yarn v1.22.4 or higher
• NPM v6.13.7 or higher

Local NodeJS, Yarn and NPM installations are required in case you want to develop on your local machine instead of in a container.

Setup

1. Clone repository

2. Build images for Hasura & functions

   docker-compose build --parallel
   

3. Start Postgres, Hasura & functions:

   docker-compose up -d hasura
   

4. Start the frontend development server:

   docker-compose up frontend
   

   After 30s or so it should say You can now view trm-frontend in the browser..

5. Verify everything is running and healthy:

   docker-compose ps
   

   It should say Up (healthy) for every service.

6. (Only required if developing locally) Setup frontend for development:

   • The following commands are assumed to be run in the /frontend directory (cd frontend)

   • yarn install downloads the dependencies

   • Run cp config.example.js config.js for an example config which is set up to work with the local database & Hasura instance you brought up with docker-compose.

Usage

• Generate Sample Data

  • It is not much fun to work with an empty database. This is why there is an included data generator which you can use to populate the database with some near-realistic data. This generator is also used on the staging environment.

  • Generate some techies:

    1. Create a Semester in the frontend and copy its UUID (shown in the location bar like http://localhost:3000/#/semesters/UUID)

    2. Run the techie generator:

       docker-compose run --rm data-generator -- generate techies 100 --semester UUID --location LOCATION
       

       (location is BERLIN, for example)

• Frontend Development

  • yarn start starts a development server on port 3000 with support for live code reloading.

    Run this in the /frontend directory (cd frontend).

    It should open the browser at http://localhost:3000 automatically.

• Hasura Development

  • Run docker-compose exec hasura hasura-cli console --address 0.0.0.0 --no-browser and open http://localhost:9695 for the Hasura console.

  • You can find the Admin Secret in docker-compose.yml (hasura service, environment variables, HASURA_GRAPHQL_ADMIN_SECRET)

  • Authorization works with the Admin Secret, but you may want to use token authorization (e.g. for testing roles & permissions).

    Generate a token with docker-compose run --rm data-generator -- generate token --location BERLIN --team-member-id ID --functional-team JOURNEY --roles user,hr,journey.

    Then add the Authorization header in the Hasura GraphQL Explorer with Bearer TOKEN as its value (replace TOKEN with the output of the generate token command).

  • (Optional, for setup) Generate new JWT Secret Key: openssl rand 256 | base64

Deployment

The project is deployed on Google Cloud Platform (GCP). Follow the guide for setting up GCP.

The infrastructure (see /infrastructure/) is managed with Terraform in two projects:

• /infrastructure/common for all shared resources between environments: the Cloud SQL PostgreSQL database and a Google Artifact Registry Repository
• /infrastructure/app provides resources for a specific environment: the Hasura instance as a Cloud Run service, and the serverless functions in Cloud Functions

The deployment process runs the GitHub Actions deploy workflow for every push to the deployment branches:

• staging branch for the staging environment
• production branch for the production environment

The TRM Dashboard is deployed on GitHub pages. GitHub Actions builds the frontend and commits the build in the following repositories:

• TechLabs-Berlin/trm-frontend-staging – for the staging environment
• TechLabs-Berlin/trm-frontend-production – for the production environment

Contributing

We appreciate any contribution and are happy to get in touch. If you see a problem, please open an issue in this repository. For write access to this repository, please send a message on our Slack Channel

To contribute code, we use a Pull Request workflow:

1. Create a branch with your proposed changes and open a Pull Request
2. We work with you for review and merge your code into the main branch.

Setup on Gitpod

Gitpod just recently announced they're now supporting Docker in their workspaces. The following guide will guide you through the setup process. It works quite well, though be aware they may be some quirks left.

Watch a Screencast of @Shark setting up the frontend on Gitpod.

Or follow these steps:

1. Go to gitpod.io/settings, login and enable "Feature Preview". Choose "Code" as Default IDE.

2. Go to https://gitpod.io/#https://github.com/TechLabs-Berlin/trm to create a new workspace for the TRM project.

3. Download and install newest docker-compose (https://docs.docker.com/compose/install/) in the workspace.

4. Run sudo docker-up. You need to leave the command running, thus open a new terminal by clicking on the plus icon.

5. In docker-compose.yml, replace the keys under depends_on: with an array of the container names.

   Currently, the healthchecks don't work on Gitpod. This is why you need to tell docker-compose to bring up dependent services even though they are unhealthy. This step probably won't be necessary in the future.

6. Follow normal Setup guide

   Ignore warnings about services being unhealthy.

7. In /frontend, copy config.example.js to config.js

8. Go to Remote Explorer Tab in the Sidebar

   1. For Port 8000: Allow public access, copy URL & set as authFnURL
   2. For Port 8080: Allow public access, copy URL & set as graphqlApiURL

9. Open Port 3000 in the browser (click on the globe icon in the Remote Explorer Tab)

License

The TechLabs-Berlin/trm project is licensed MIT. Learn more about MIT license on choosealicense.com.