Digital Paper Edit - Firebase

An application to make it faster, easier and more accessible to edit audio and video interviews using automatically generated transcriptions form STT service.

See intro for more info on the project. And user journey for a high level overview of the user journey.

Development

Fork: API and Client

A fork of the client and API is still being developed in the open, in separate forks: client and API, thanks to @pietrop.

We don't want the opensource collaboration to stop, so we will be pulling in changes from the forks. If there's a valuable feature that we haven't yet implemented from the fork, please open an issue in this repo or get in touch with us!

Collaborating across forks

We will look at the changes in the fork and manually add them to our firebase version.

When manually adding changes to the changes from a fork - there are challenges of pulling in dependencies, such as:

[
  ("@bbc/aes31-adl-composer": "^1.0.1"),
  ("@bbc/digital-paper-edit-storybook": "^1.3.2"),
  ("@bbc/fcpx-xml-composer": "^1.0.0"),
  ("@bbc/react-transcript-editor": "^1.4.0"),
];

These dependencies (non-exhaustive) they might've been forked also. In this case, this is a matter of updating the BBC's version with the newest changes from the dependency's forks and making sure everything works together.

For understanding the approach see this PR.

Current project board

• BBC News Labs - Digital Paper Edit - Sprint Board
• BBC News Labs - Digital Paper Edit 2WOT - Sprint Board

The project is divided into concurrent milestones as described here with UX being an overarching milestone that cuts across these different parts. See UX Approach in docs guides for more information on the UX development process.

Project Architecture

This is a simplified version of the bbc/digital-paper-edit-client application, using Firebase. There are many moving parts in this project, but Firebase is self-contained. If you want to see all the projects you can also look at them as a list here. Most React Components are developed, maintained and imported from bbc/digital-paper-edit-storybook. There is a Storybook demo website in the repo that you access to view the components.If you want to develop with a local version of the React Components, see here.

See docs in ADR for an understanding of the architectural decisions made. If you'd like some information on Firebase, please read Firebase ADR and the Modular Architecture ADR for more info on the implementation

Versioning

The projects use npm semantic versioning

Setup

This project uses npm.

Configuration

.env contains environment variables setting up against the dev environment. You can copy over the .env.example to start. REACT_APP_NAME is used in browser title and navbar component.

REACT_APP_NAME="digital-paper-edit"
REACT_APP_NODE_ENV="development"
REACT_APP_PROTOTYPE_BY="BBC News Labs"
REACT_APP_PROTOTYPE_BY_LINK="http://bbcnewslabs.co.uk"

Firebase specific configuration

This doc from Firebase is handy to understand the variables

let firebaseConfig = {
  apiKey: "api-key",
  authDomain: "project-id.firebaseapp.com",
  databaseURL: "https://project-id.firebaseio.com",
  projectId: "project-id",
  storageBucket: "project-id.appspot.com",
  messagingSenderId: "sender-id",
  appId: "app-id",
  measurementId: "G-measurement-id",
};

If you do change this and you want Travis to run deployment, you will need to encrypt it using Travis CLI's encrypt-file feature and recommit .env.enc.

1. Make changes to the .env file
2. Generate the encrypted .env file by running travis encrypt-file .env
3. Then Git add and commit .env.enc

CORS

The cors.json file in root has been used to update the setting for GCP Storage to get the media file for playback.

gsutil cors set cors.json gs://dev-digital-paper-edit

Local Development

You must setup the Firebase credentials in order to develop the project - as mentioned in above section. If you are BBC staff, please sign into the corporate GCP account. Firebase can be free, but some parts of the app may not work. In root of the project (cd digital-paper-edit-firebase):

npm run start

which maps to "run-p --race dev:firebase dev:react",

This will start two servers: proxy (3000) and Firebase server (4000). You should have an entry point app running in port 3000. Note that this is accessing real data, rather than a dummy one. We are trying to resolve this in #31

{
    "dev:firebase": "firebase serve -p 4000",
    "dev:react": "cross-env REACT_APP_NODE_ENV=development react-scripts start",
}

Local version of React Components (DPE-Storybook)

If you want to develop with a local version of the React Components, see here.

Production and Deployment

See Configuration step above and configure firebase.json, .firebaserc to change the sitename and environment. The deployment will be done via Travis CI

To deploy to development environment:

npm run deploy:dev:hosting

To deploy to production environment:

npm run deploy:prod:hosting

Both steps will remove the build folder, rebuild and deploy, using firebase cli tools. You must ensure that Firebase is installed globally (npm i -g firebase-tools).

Read more about Firebase and initialising here.

System Architecture

React is setup using Create React App.

You can learn more in the Create React App documentation.

To learn React, check out the React documentation.

Development env

• npm > 6.11.3
• node v 10 - lts/dubnium
• see .eslintrc in the various packages for linting rules

Node version is set in node version manager .nvmrc

Functions

Use node v8 in Functions directory.

Developing is a lot easier if you have your local emulator set up.

Running Firebase functions locally:

1. Set up admin credentials for emulated functions via the service account
2. Make sure you are in the newslab-dev project and click Add key
3. You need to save this as gcp-credentials.json and keep it in your digital-paper-edit-firebase/functions folder.
4. cd to your digital-paper-edit-firebase/functions folder.
5. run the following commands

export GOOGLE_APPLICATION_CREDENTIALS="gcp-credentials.json"
firebase functions:config:get > .runtimeconfig.json

3. Then to start the emulator in your terminal window, run

firebase functions:shell

4. BEWARE!!! Running functions locally will really affect files in the Firestore, so tread carefully!
5. Call the function you would like to test e.g. run dpeCronExpiredMediaChecker()

See here for more info on running functions locally.

Function deployment

From the functions directory, you can

Deploy to dev:

npm run deploy:dev

Deploy to prod:

npm run deploy:prod

Documentation

See docs folder

• docs/features-list overview of main features of the app.
• docs/user-journey/user-journey.md overview of main features of the app.
• docs/notes/ contains unsorted dev notes on various aspects of the project (think of it as drafts).
• docs/guides/ contains good to know/how to on various aspects of the project.
  • guides/admin-view contains instructions on the Admin View
  • guides/create-new-users contains instructions for adding new users to DPE
• docs/adr/ contains Architecture Decision Record.

An architectural decision record (ADR) is a document that captures an important architectural decision made along with its context and consequences.

We are using this template for ADR

Build

npm run build

Build of react client side will be in build

Builds the app for production to the build folder.
It correctly bundles React in production mode and optimizes the build for the best performance.

The build is minified and the filenames include the hashes.
Your app is ready to be deployed!

Tests

Test coverage using jest, to run tests

npm run test

During development you can use

npm run test:watch

Launches the test runner in the interactive watch mode.<br/ > See the section about running tests for more information.

Contributing

See CONTRIBUTING.md guidelines and CODE_OF_CONDUCT.md guidelines.

Licence

See LICENCE

Legal Disclaimer

Despite using React and Firebase, the BBC is not promoting any Facebook products or other commercial interest.