From f6e22a4f2ac72ca83456b2b908920a7def53198a Mon Sep 17 00:00:00 2001 From: Brian D Date: Fri, 28 Apr 2023 15:14:57 -0500 Subject: [PATCH 1/9] docs: github pages is not supported --- sites/docs/docs/deployment/github-pages.md | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 sites/docs/docs/deployment/github-pages.md diff --git a/sites/docs/docs/deployment/github-pages.md b/sites/docs/docs/deployment/github-pages.md new file mode 100644 index 0000000000..9cd201cef7 --- /dev/null +++ b/sites/docs/docs/deployment/github-pages.md @@ -0,0 +1,7 @@ +--- +sidebar_position: 1 +hide_table_of_contents: false +title: Github Pages +--- + +Evidence does not currently support Github Pages. This is because Github Pages sites are hosted with a root path other than `/`, which is not well supported by SvelteKit. For more information you can see some of the discussion on [GitHub](https://github.com/evidence-dev/evidence/issues/603). From aae98dca1c2639dda283aa4dddefb5abebdcf9b6 Mon Sep 17 00:00:00 2001 From: Brian D Date: Mon, 1 May 2023 14:34:09 -0500 Subject: [PATCH 2/9] docs: note sql file aliases --- sites/docs/docs/core-concepts/queries/index.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/sites/docs/docs/core-concepts/queries/index.md b/sites/docs/docs/core-concepts/queries/index.md index db570e2b11..f7f64b01c9 100644 --- a/sites/docs/docs/core-concepts/queries/index.md +++ b/sites/docs/docs/core-concepts/queries/index.md @@ -118,3 +118,14 @@ In your evidence file, you can now reference `my_query` and `some_category_my_ca ### Advanced Usage SQL file queries can [depend on other query files](/core-concepts/queries/#query-chaining), but they will all need to be referenced in the files you use them in. For example, if `my_query` depends on `some_category_my_category_query`, then you will have to have them both in your [frontmatter](/markdown/#frontmatter), as shown above. + +If the name of your query file is long, or doesn't match how it is being used on the page; it is easy to change how it is refernced on the page. Instead of importing your query as ` - query.sql`, you can import is as ` - renamed: query.sql`. Now instead of the data being `{query}`, it is `{renamed}`. + +```markdown +--- +sources: + - default_query_name.sql # This will be referenced as default_query_name + - alias: default_query_name.sql # This will be referenced as alias +# ^ This is the key +--- +``` \ No newline at end of file From ce0a5bdf4ebd97209106d4c62dfbfe74d3e8c3f2 Mon Sep 17 00:00:00 2001 From: Brian D Date: Wed, 3 May 2023 10:13:20 -0500 Subject: [PATCH 3/9] docs: pr feedback --- .../docs/docs/core-concepts/queries/index.md | 27 ++++++++++--------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/sites/docs/docs/core-concepts/queries/index.md b/sites/docs/docs/core-concepts/queries/index.md index f7f64b01c9..0583f0ec9a 100644 --- a/sites/docs/docs/core-concepts/queries/index.md +++ b/sites/docs/docs/core-concepts/queries/index.md @@ -108,24 +108,25 @@ These queries can then be used on `my_page.md` with the following [frontmatter]( ```yaml --- sources: - - my_query.sql - - some_category/my_category_query.sql + - q4_data: my_query.sql + - q4_sales_reps: some_category/my_category_query.sql --- ``` -In your evidence file, you can now reference `my_query` and `some_category_my_category_query` (note that `/` became `_`) the same way you would use any other query. - -### Advanced Usage - -SQL file queries can [depend on other query files](/core-concepts/queries/#query-chaining), but they will all need to be referenced in the files you use them in. For example, if `my_query` depends on `some_category_my_category_query`, then you will have to have them both in your [frontmatter](/markdown/#frontmatter), as shown above. +In your evidence file, you can now reference `q4_data` and `q4_sales_reps` the same way you would use any other query. -If the name of your query file is long, or doesn't match how it is being used on the page; it is easy to change how it is refernced on the page. Instead of importing your query as ` - query.sql`, you can import is as ` - renamed: query.sql`. Now instead of the data being `{query}`, it is `{renamed}`. +Optionally, you can omit the query name, and the filename will be used instead; these queries will be available at `my_query` and `some_category_my_category_query` (note that `/` became `_`). -```markdown +```yaml --- sources: - - default_query_name.sql # This will be referenced as default_query_name - - alias: default_query_name.sql # This will be referenced as alias -# ^ This is the key + - my_query.sql + - some_category/my_category_query.sql --- -``` \ No newline at end of file +``` + +### Advanced Usage + +#### File Query Chaining + +SQL file queries can [depend on other query files](/core-concepts/queries/#query-chaining), but they will all need to be referenced in the files you use them in. For example, if `my_query` depends on `some_category_my_category_query`, then you will have to have them both in your [frontmatter](/markdown/#frontmatter), as shown above. From 6e72513f9857061af0a7ffc63a3d17dc434f7f01 Mon Sep 17 00:00:00 2001 From: Brian D Date: Wed, 3 May 2023 10:16:51 -0500 Subject: [PATCH 4/9] docs: pr feedback --- sites/docs/docs/deployment/github-pages.md | 7 ------- sites/docs/docs/deployment/overview.md | 2 +- 2 files changed, 1 insertion(+), 8 deletions(-) delete mode 100644 sites/docs/docs/deployment/github-pages.md diff --git a/sites/docs/docs/deployment/github-pages.md b/sites/docs/docs/deployment/github-pages.md deleted file mode 100644 index 9cd201cef7..0000000000 --- a/sites/docs/docs/deployment/github-pages.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -sidebar_position: 1 -hide_table_of_contents: false -title: Github Pages ---- - -Evidence does not currently support Github Pages. This is because Github Pages sites are hosted with a root path other than `/`, which is not well supported by SvelteKit. For more information you can see some of the discussion on [GitHub](https://github.com/evidence-dev/evidence/issues/603). diff --git a/sites/docs/docs/deployment/overview.md b/sites/docs/docs/deployment/overview.md index 3acd1fd9fd..26bb08f699 100644 --- a/sites/docs/docs/deployment/overview.md +++ b/sites/docs/docs/deployment/overview.md @@ -6,7 +6,7 @@ title: Deployment Overview In production, Evidence is a [static site generator](https://www.netlify.com/blog/2020/04/14/what-is-a-static-site-generator-and-3-ways-to-find-the-best-one/). This means it doesn't run queries when someone visits your site, but pre-builds all possible pages as HTML beforehand. -You can host your Evidence project using Evidence Cloud, cloud services like Netlify or Vercel, or your own infrastructure. +You can host your Evidence project using Evidence Cloud, cloud services like Netlify or Vercel, or your own infrastructure. Evidence does not currently support Github Pages, there is more information on [GitHub](https://github.com/evidence-dev/evidence/issues/603). ## Evidence Cloud From 3a023d5c6d03f5d0f52cca7c8dceed683941a4e3 Mon Sep 17 00:00:00 2001 From: Archie Wood <58074498+archiewood@users.noreply.github.com> Date: Fri, 5 May 2023 15:26:21 -0400 Subject: [PATCH 5/9] Add database specific info to connection page --- .changeset/metal-clouds-confess.md | 5 + .../docs/core-concepts/data-sources/index.md | 119 ++++++++++++++---- sites/docs/docs/guides/bigquery.md | 23 ---- .../ui/Databases/DatabaseSettingsPanel.svelte | 2 +- 4 files changed, 102 insertions(+), 47 deletions(-) create mode 100644 .changeset/metal-clouds-confess.md delete mode 100644 sites/docs/docs/guides/bigquery.md diff --git a/.changeset/metal-clouds-confess.md b/.changeset/metal-clouds-confess.md new file mode 100644 index 0000000000..530cb6b3d4 --- /dev/null +++ b/.changeset/metal-clouds-confess.md @@ -0,0 +1,5 @@ +--- +'@evidence-dev/components': patch +--- + +Update docs link diff --git a/sites/docs/docs/core-concepts/data-sources/index.md b/sites/docs/docs/core-concepts/data-sources/index.md index a5d50397fa..f08641ddea 100644 --- a/sites/docs/docs/core-concepts/data-sources/index.md +++ b/sites/docs/docs/core-concepts/data-sources/index.md @@ -20,27 +20,107 @@ Evidence will save your credentials locally, and run a test query to confirm tha Evidence supports: -- [BigQuery](/guides/bigquery) -- Snowflake -- Redshift -- PostgreSQL -- MySQL -- SQLite -- DuckDB -- [CSV files](#local-data-files) +- [BigQuery](#bigquery) +- [Snowflake](#snowflake) +- [Redshift](#redshift) +- [PostgreSQL](#postgresql) +- [MySQL](#mysql) +- [SQLite](#sqlite) +- [DuckDB](#duckdb) +- [CSV and Parquet files](#local-data-files) - & More We're adding new connectors regularly. [Create a GitHub issue](https://github.com/evidence-dev/evidence/issues) or [send us a message in Slack](https://join.slack.com/t/evidencedev/shared_invite/zt-uda6wp6a-hP6Qyz0LUOddwpXW5qG03Q) if you'd like to use Evidence with a database that isn't currently supported. -## Local Data Files +The source code for Evidence's connectors is available [on GitHub](https://github.com/evidence-dev/evidence/tree/main/packages) -In Evidence, you can query local CSV files directly in SQL. + +## Data Source Specific Info + +All databases can be connected via the UI settings page as described above. Where relevant, additional information is provided below. + +### BigQuery + +Evidence supports connecting to Google BigQuery by using a [service account](https://cloud.google.com/iam/docs/service-accounts) and a JSON key. + +OAuth is not currently supported (though PRs are welcome). + +Follow the instructions below to set up your service account and get a JSON key. + +#### Create a Service Account Key + +1. [Go to the Service Account Page](https://console.cloud.google.com/projectselector/iam-admin/serviceaccounts/create?supportedpurview=project) and click on your project +2. Add a name for your service account, then click Create +3. Assign your service account a role for BigQuery (scroll down the role dropdown to find BigQuery roles). + 1. **BigQuery User** should work for most use cases. + 1. **BigQuery Data Viewer** may be required (depending on your organization's permissions settings in Google Cloud). + 1. Reach out to us if you run into issues or need help with BigQuery permissions. +4. Click Continue, then click Done. You should see a table of users. +5. Click on the email address for the service account you just created, then click the **Keys** tab +6. Click Add Key, then Create New Key, then Create +7. Google will download a JSON Key File to your computer + + + +### Snowflake + +Evidence supports connecting to Snowflake using [Basic Authentication](https://docs.snowflake.com/en/user-guide/api-authentication), i.e. username and password. + +OAuth and Key-Pair authentication are not currently supported, though PRs are welcome. + +### Redshift + +The Redshift connector uses the Postgres connector under the hood, so configuration options are similar. +### PostgreSQL + +#### SSL + +To connect to a Postgres database using SSL, you may need to modify the SSL settings used. Once you have selected a PostgreSQL data connection type, you can set the SSL value as follows: + - `false`: Don't connect using SSL (default) + - `true`: Connect using SSL, validating the SSL certificates. Self-signed certificates will fail using this approach. + - `no-verify`: Connect using SSL, but don't validate the certificates. + +Other SSL options will require the use of a custom connection string. Evidence uses the node-postgres package to manage these connections, and the details of additional SSL options via the connection string can be found at the [package documentation](https://node-postgres.com/features/ssl). + +One scenario might be a Postgres platform that issues a self-signed certificate for the database connection, but provides a CA certificate to validate that self-signed certificate. In this scenario you could use a CONNECTION STRING value as follows: + +``` +postgresql://{user}:{password}@{host}:{port}/{database}?sslmode=require&sslrootcert=/path/to/file/ca-certificate.crt +``` + +Replace the various `{properties}` as needed, and replace `/path/to/file/ca-certificate.crt` with the path and filename of your certificate. + +### MySQL + +#### SSL + +SSL options are: + +- `false` (default) +- `true` +- `Amazon RDS` +- A credentials object + + +### SQLite + +SQLite is a local file-based database. It should be stored in the root of your Evidence project. + +### DuckDB + +DuckDB is a local file-based database. It should be stored in the root of your Evidence project. + +See the [DuckDB docs](https://duckdb.org/docs/guides/index) for more information. + +### CSV and Parquet Files + +In Evidence, you can query local CSV or Parquet files directly in SQL. Get started by selecting the `CSV` connector on the Settings page in your project. -### How to Query a CSV File +#### How to Query a CSV File -#### Inside your Evidence Project +##### Inside your Evidence Project Evidence looks for CSV files stored in a `sources` folder in the root of your Evidence project. You can then query them using this syntax: @@ -48,7 +128,7 @@ Evidence looks for CSV files stored in a `sources` folder in the root of your Ev select * from 'sources/myfile.csv' ``` -#### Absolute Filepaths +##### Absolute Filepaths You can pass in an absolute filepath: @@ -56,7 +136,7 @@ You can pass in an absolute filepath: select * from 'Users/myname/Downloads/myfile.csv' ``` -#### Relative Filepaths +##### Relative Filepaths Paths are **relative to two files deep** in your Evidence project. For example, to query a CSV in the root of an Evidence project, you would use this syntax: @@ -64,11 +144,11 @@ Paths are **relative to two files deep** in your Evidence project. For example, select * from '../../myfile.csv' ``` -### SQL Syntax for Querying CSVs +#### SQL Syntax for Querying CSVs Evidence uses DuckDB to run SQL against a CSVs. For query syntax, see the [DuckDB docs](https://duckdb.org/docs/sql/query_syntax/select). -### Parsing Headers +#### Parsing Headers When parsing headers in CSV files, the `read_csv_auto` helper function provided by DuckDB can be helpful. @@ -80,16 +160,9 @@ In addition to the `HEADER` argument, this function can also accept changes to t Additional information about CSV helper functions can be found in the [DuckDB docs](https://duckdb.org/docs/data/csv). -## SSL with PostgreSQL -To connect to a Postgres database using SSL, you may need to modify the SSL settings used. Once you have selected a PostgreSQL data connection type, you can set the SSL value as follows: - - false: Don't connect using SSL (default) - - true: Connect using SSL, validating the SSL certificates. Self-signed certificates will fail using this approach. - - no-verify: Connect using SSL, but don't validate the certificates. -Other SSL options will require the use of a custom connection string. Evidence uses the node-postgres package to manage these connections, and the details of additional SSL options via the connection string can be found at the [package documentation](https://node-postgres.com/features/ssl). -One scenario might be a Postgres platform that issues a self-signed certificate for the database connection, but provides a CA certificate to validate that self-signed certificate. In this scenario you could use a CONNECTION STRING value as follows: `postgresql://{user}:{password}@{host}:{port}/{database}?sslmode=require&sslrootcert=/path/to/file/ca-certificate.crt`. Replace the various `{properties}` as needed, and replace `/path/to/file/ca-certificate.crt` with the path and filename of your certificate. ## Troubleshooting diff --git a/sites/docs/docs/guides/bigquery.md b/sites/docs/docs/guides/bigquery.md deleted file mode 100644 index 126a6ae7d2..0000000000 --- a/sites/docs/docs/guides/bigquery.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: BigQuery -sidebar_position: 5 ---- - -## BigQuery - -Evidence supports connecting to Google BigQuery by using a [service account](https://cloud.google.com/iam/docs/service-accounts) and a JSON key. - -Follow the instructions below to set up your service account and get a JSON key. - -### Create a Service Account Key - -1. [Go to the Service Account Page](https://console.cloud.google.com/projectselector/iam-admin/serviceaccounts/create?supportedpurview=project) and click on your project -2. Add a name for your service account, then click Create -3. Assign your service account a role for BigQuery (scroll down the role dropdown to find BigQuery roles). - 1. **BigQuery User** should work for most use cases. - 1. **BigQuery Data Viewer** may be required (depending on your organization's permissions settings in Google Cloud). - 1. Reach out to us if you run into issues or need help with BigQuery permissions. -4. Click Continue, then click Done. You should see a table of users. -5. Click on the email address for the service account you just created, then click the **Keys** tab -6. Click Add Key, then Create New Key, then Create -7. Google will download a JSON Key File to your computer diff --git a/sites/example-project/src/components/ui/Databases/DatabaseSettingsPanel.svelte b/sites/example-project/src/components/ui/Databases/DatabaseSettingsPanel.svelte index 19655b2bbe..66d9638eb0 100644 --- a/sites/example-project/src/components/ui/Databases/DatabaseSettingsPanel.svelte +++ b/sites/example-project/src/components/ui/Databases/DatabaseSettingsPanel.svelte @@ -25,7 +25,7 @@ id: 'bigquery', name: 'BigQuery', formComponent: BigqueryForm, - docsHref: 'https://docs.evidence.dev/guides/bigquery' + docsHref: 'https://docs.evidence.dev/core-concepts/data-sources/#bigquery' }, { id: 'postgres', name: 'PostgreSQL', formComponent: PostgresForm }, { id: 'mysql', name: 'MySQL', formComponent: MysqlForm }, From 6985e97d33c74d1c232fe6d9b5e9e67319ad8632 Mon Sep 17 00:00:00 2001 From: Archie Wood <58074498+archiewood@users.noreply.github.com> Date: Fri, 5 May 2023 15:32:14 -0400 Subject: [PATCH 6/9] nits, production connection info --- sites/docs/docs/core-concepts/data-sources/index.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sites/docs/docs/core-concepts/data-sources/index.md b/sites/docs/docs/core-concepts/data-sources/index.md index f08641ddea..164ce1207f 100644 --- a/sites/docs/docs/core-concepts/data-sources/index.md +++ b/sites/docs/docs/core-concepts/data-sources/index.md @@ -16,6 +16,8 @@ To connect your local development environment to a database: Evidence will save your credentials locally, and run a test query to confirm that it can connect. +Connections to databases in production are managed via [environment variables](/cli#environment-variables) + ### Supported data sources Evidence supports: @@ -35,7 +37,7 @@ We're adding new connectors regularly. [Create a GitHub issue](https://github.co The source code for Evidence's connectors is available [on GitHub](https://github.com/evidence-dev/evidence/tree/main/packages) -## Data Source Specific Info +## Data source specific info All databases can be connected via the UI settings page as described above. Where relevant, additional information is provided below. @@ -112,7 +114,7 @@ DuckDB is a local file-based database. It should be stored in the root of your E See the [DuckDB docs](https://duckdb.org/docs/guides/index) for more information. -### CSV and Parquet Files +### CSV and Parquet files In Evidence, you can query local CSV or Parquet files directly in SQL. From fcf6bb0fe1649454d179066658a8e62a9fced60d Mon Sep 17 00:00:00 2001 From: Nidhi Kala Date: Sun, 14 May 2023 20:58:14 -0400 Subject: [PATCH 7/9] value component error icon: white and vertically centered --- .../example-project/src/components/icons/HelpCircleIcon.svelte | 2 +- sites/example-project/src/components/viz/Value.svelte | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/sites/example-project/src/components/icons/HelpCircleIcon.svelte b/sites/example-project/src/components/icons/HelpCircleIcon.svelte index c219593fcf..548c8bf4ac 100644 --- a/sites/example-project/src/components/icons/HelpCircleIcon.svelte +++ b/sites/example-project/src/components/icons/HelpCircleIcon.svelte @@ -20,12 +20,12 @@ diff --git a/sites/example-project/src/components/viz/Value.svelte b/sites/example-project/src/components/viz/Value.svelte index e02cba2b9d..83ceaf2a1e 100644 --- a/sites/example-project/src/components/viz/Value.svelte +++ b/sites/example-project/src/components/viz/Value.svelte @@ -85,7 +85,7 @@ Error - + {error} @@ -117,7 +117,10 @@ } .additional-info-icon { - display: inline; + display: flex; + align-items: center; + height: 100%; + width: 100%; vertical-align: middle; width: 14px; color: white;