Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename DataHub to Querybook for docs and docs site #339

Merged
merged 1 commit into from
Dec 7, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/admin_guide/customize_html.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: Customize HTML
sidebar_label: Customize HTML
---

DataHub allows for some basic customization in the frontend. You can use the web plugin to inject custom javascript into DataHub frontend. Please check [Plugins Guide](./plugins.md) to see how to get started.
Querybook allows for some basic customization in the frontend. You can use the web plugin to inject custom javascript into Querybook frontend. Please check [Plugins Guide](./plugins.md) to see how to get started.

Right now there are four use cases for custom javascript:

Expand All @@ -18,7 +18,7 @@ script.innerHTML = `
document.body.appendChild(script);
```

2. Customize some of the messages in DataHub. Currently DataHub allows the plugin to set the following messages:
2. Customize some of the messages in Querybook. Currently Querybook allows the plugin to set the following messages:

```typescript
interface Window {
Expand Down Expand Up @@ -46,6 +46,6 @@ interface Window {
}
```

The renderer should be treated as the entry point into your customized react view. You can add any default HTML elements you want or import UI elements from DataHub.
The renderer should be treated as the entry point into your customized react view. You can add any default HTML elements you want or import UI elements from Querybook.

If the mode variable is not provided, then DataHub would show this custom content in the middle section (currently empty) of the default landing page. However, if you specify the mode to be "replace", then the entire landing page will be replaced with your custom content.
If the mode variable is not provided, then Querybook would show this custom content in the middle section (currently empty) of the default landing page. However, if you specify the mode to be "replace", then the entire landing page will be replaced with your custom content.
8 changes: 4 additions & 4 deletions docs/admin_guide/deployment_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ title: Deployment Guide
sidebar_label: Deployment Guide
---

While there are many ways to deploy datahub to production, there are some general principles
While there are many ways to deploy Querybook to production, there are some general principles
that are recommended when setting up your own production deployment.

1. Please make sure the web server, the celery beat scheduler, and the celery worker are using the production Docker image. Please refer to `containers/docker-compose.prod.yml` to see how to launch these images for production.
2. To get logs, make sure /var/log/datahub/ is mounted as a Docker volume so that the logs can be moved to the host machine.
2. To get logs, make sure /var/log/querybook/ is mounted as a Docker volume so that the logs can be moved to the host machine.
3. Use the /ping/ endpoint for health checks.
1. During deployments, you can create a file that has the path `/tmp/datahub/deploying` to make the health check endpoint /ping/ return 503 and remove it after completion.
1. During deployments, you can create a file that has the path `/tmp/querybook/deploying` to make the health check endpoint /ping/ return 503 and remove it after completion.
4. Please make sure celery worker is ran with concurrent mode as it is the only mode that can have a memory limit per worker.
5. During worker deployments, you can run the following first to make the celery worker stop receving new tasks and exit once all current tasks are finished: `celery multi stopwait datahub_worker@%h -A tasks.all_tasks --pidfile=/opt/celery_%n.pid`. This will make deployment time take much longer but users' running queries won't be killed.
5. During worker deployments, you can run the following first to make the celery worker stop receving new tasks and exit once all current tasks are finished: `celery multi stopwait querybook_worker@%h -A tasks.all_tasks --pidfile=/opt/celery_%n.pid`. This will make deployment time take much longer but users' running queries won't be killed.
50 changes: 28 additions & 22 deletions docs/admin_guide/embedded_iframe.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,43 @@
---
id: embedded_iframe
title: Embed DataHub As Iframe
title: Embed Querybook As Iframe
sidebar_label: Embed Iframe
---

You can embed DataHub as an iframe in other tools so users can edit query directly on DataHub, test it out, and then update it back. Here are the steps to implement this feature:
You can embed Querybook as an iframe in other tools so users can edit query directly on Querybook, test it out, and then update it back. Here are the steps to implement this feature:

1. Include the following HTML

```html
<iframe
id="datahub-iframe"
src="https://{DATAHUB_URL}/data/_/embedded_editor/"
id="querybook-iframe"
src="https://{QUERYBOOK_URL}/data/_/embedded_editor/"
/>
```

2. When the iframe loads, there will be 2 kinds of events that can be send back, please handle them by adding an event listener to window:
```js
window.addEventListener('message', (e) => {
// Determine the event type here
const type = e?.data?.type;

```js
window.addEventListener(
'message',
(e) => {
// Determine the event type here
const type = e?.data?.type;

if (type === 'SEND_QUERY') {
// If it is a send query event, please send the
// datahub iframe the query being edited
iframeEl?.contentWindow?.postMessage(
{ type: 'SET_QUERY', value: query },
'*'
);
} else if (type === 'SUBMIT_QUERY') {
// If it is a submit query event, it means the user
// has finished editing the query in datahub, please
// update your query in your tool accordingly
updateQuery(e?.data?.value);
}
}, false);
if (type === 'SEND_QUERY') {
// If it is a send query event, please send the
// querybook iframe the query being edited
iframeEl?.contentWindow?.postMessage(
{ type: 'SET_QUERY', value: query },
'*'
);
} else if (type === 'SUBMIT_QUERY') {
// If it is a submit query event, it means the user
// has finished editing the query in querybook, please
// update your query in your tool accordingly
updateQuery(e?.data?.value);
}
},
false
);
```
12 changes: 6 additions & 6 deletions docs/admin_guide/general_config.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ sidebar_label: General Config

## Overview

Other than the basic infrastructure setup such as rdbms and redis, all configurations can be done in the admin tools on the DataHub website directly. Note that it is only accessible for users under the "Admin" role.
Other than the basic infrastructure setup such as rdbms and redis, all configurations can be done in the admin tools on the Querybook website directly. Note that it is only accessible for users under the "Admin" role.

The first user that installs DataHub gets the "Admin" role. Admins can add other admins and modify anyone's role.
The first user that installs Querybook gets the "Admin" role. Admins can add other admins and modify anyone's role.

In the next section we will go over different things that can be configured in the Admin tools.

Expand All @@ -18,12 +18,12 @@ Checkout [Sharing & Security](./sharing_and_security.md) to learn how to configu

#### Environment

Environment ensures users on DataHub are only allowed to access to information/query they have permission to. All DataDocs, Query Engines are attached to some environments.
Environment ensures users on Querybook are only allowed to access to information/query they have permission to. All DataDocs, Query Engines are attached to some environments.

Here are all the fields needed when creating an environment:

- Name (**required**): The name must be all letters since it will appear in the url.
- Public: A public environment allows any users of DataHub to use.
- Public: A public environment allows any users of Querybook to use.
- Archived: Once archived, environments cannot be accessed on the website.
- Environment description: A short description of the environment which will appear as a tooltip on the environment picker.
- Logo Url: By default an environment's icon is shown as an square button with the first letter of the environment's name in it. You can also supply a custom image.
Expand All @@ -46,7 +46,7 @@ Query engine configures the endpoints that users can query. Each query engine ne

### Announcement

DataHub Admins can use the announcement feature to send quick updates to users on DataHub.The announcement will appear as a top banner on DataHub's main site. DataHub actively polls the announcement end point five minutes so any change to the announcements are quickly reflected.
Querybook Admins can use the announcement feature to send quick updates to users on Querybook.The announcement will appear as a top banner on Querybook's main site. Querybook actively polls the announcement end point five minutes so any change to the announcements are quickly reflected.
Here are all the fields needed when creating an announcement:

- url prefix: admins can enter a regex so the announcement will only show up if the url matches the regex
Expand All @@ -55,7 +55,7 @@ Here are all the fields needed when creating an announcement:

### User Role

The user role feature allows admin to add other admins or remove admin permission of other admins. At least 1 admin needs to exist on DataHub.
The user role feature allows admin to add other admins or remove admin permission of other admins. At least 1 admin needs to exist on Querybook.

### Job Status

Expand Down
26 changes: 13 additions & 13 deletions docs/admin_guide/infra_config.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,21 +10,21 @@ sidebar_label: Infra Config
THIS GUIDE IS ONLY FOR INFRASTRUCTURE SETUP, PLEASE READ THE [GENERAL CONFIG](../admin_guide/general_config.md) TO CONFIGURE ENTITIES SUCH AS QUERY ENGINE & ACCESS PERMISSION.
:::

Eventhrough DataHub can be launched without any configuration, it is absolutely required for more powerful infrastructure/flexible customization. In this guide we will walkthrough different kind of environment settings you can set for DataHub. You can see all possible options and default values in this repo by checking out `datahub/config/datahub_default_config.yaml`.
Eventhrough Querybook can be launched without any configuration, it is absolutely required for more powerful infrastructure/flexible customization. In this guide we will walkthrough different kind of environment settings you can set for Querybook. You can see all possible options and default values in this repo by checking out `querybook/config/querybook_default_config.yaml`.

### Making custom configs

There are two ways to pass custom configs to DataHub. The first way is using a custom config yaml file. You can write out the file and then pass it through datahub using docker volumes. For example you can add this in the docker-compose file:
There are two ways to pass custom configs to Querybook. The first way is using a custom config yaml file. You can write out the file and then pass it through querybook using docker volumes. For example you can add this in the docker-compose file:

```yaml
- path_to_my_custom_config.yaml:/opt/datahub/datahub/config/datahub_config.yaml
- path_to_my_custom_config.yaml:/opt/querybook/querybook/config/querybook_config.yaml
```

Otherwise you can also pass the environment variable directly when launching the docker image. The order of precedence for a config settings is as the follows:

1. Environment variables (highest priority)
2. datahub_config.yaml
3. datahub_default_config.yaml (lowest priority)
2. querybook_config.yaml
3. querybook_default_config.yaml (lowest priority)

## Infrastructure

Expand Down Expand Up @@ -59,21 +59,21 @@ The following settings are only relevant if you are using `db`, note that all un
The following settings are only relevant if you are using `s3` or `gcs` (Google Cloud Storage), note that all units are in bytes:

- `STORE_BUCKET_NAME` (optional): The Bucket name
- `STORE_PATH_PREFIX` (optional, defaults to **''**): Key/Blob prefix for DataHub's stored results/logs
- `STORE_PATH_PREFIX` (optional, defaults to **''**): Key/Blob prefix for Querybook's stored results/logs
- `STORE_MIN_UPLOAD_CHUNK_SIZE` (optional, defaults to **10485760**): The chunk size when uploading
- `STORE_MAX_UPLOAD_CHUNK_NUM` (optional, defaults to **10000**): The number of chunks that can be uploaded, you can determine the maximum upload size by multiplying this with chunk size.
- `STORE_READ_SIZE` (optional, defaults to 131072): The size of chunk when reading from store.
- `STORE_MAX_READ_SIZE` (optional, defaults to 5242880): The max size of file DataHub will read for users to view.
- `STORE_MAX_READ_SIZE` (optional, defaults to 5242880): The max size of file Querybook will read for users to view.

### Logging

`LOG_LOCATION` (optional): By default server logs goes to stderr. Supply a log path if you want the log to appear in a file.

## Authentication

`AUTH_BACKEND` (optional, defaults to **app.auth.password_auth**): Python path to the authentication file. By default DataHub provides:
`AUTH_BACKEND` (optional, defaults to **app.auth.password_auth**): Python path to the authentication file. By default Querybook provides:

- app.auth.password_auth: Username/password based on registering on DataHub.
- app.auth.password_auth: Username/password based on registering on Querybook.
- app.auth.oauth_auth: Oauth based authentication.

You can also supply any custom authentication added in the auth plugin. See "Add Auth" and "Plugins" guide for more details.
Expand All @@ -97,15 +97,15 @@ If you want to force the user to login again after a certain time, you can the f

## Communication

By default DataHub supports email and slack notifications for sharing DataDocs and Query completions.
By default Querybook supports email and slack notifications for sharing DataDocs and Query completions.

`PUBLIC_URL` (optional, defaults to localhost): The public url to access DataHub's website, used in oauth, email and slack.
`PUBLIC_URL` (optional, defaults to localhost): The public url to access Querybook's website, used in oauth, email and slack.

### Slack

`DATAHUB_SLACK_TOKEN` (**optional**): Put the Bot User OAuth Access Token from slack here. See https://api.slack.com/docs/oauth#bots for more details.
`QUERYBOOK_SLACK_TOKEN` (**optional**): Put the Bot User OAuth Access Token from slack here. See https://api.slack.com/docs/oauth#bots for more details.

### Email

`EMAILER_CONN` (optional, defaults to localhost:22): Location of the emailer server
`DATAHUB_EMAIL_ADDRESS` (optional, required for email): Origin address when sending emails
`QUERYBOOK_EMAIL_ADDRESS` (optional, required for email): Origin address when sending emails
Loading