Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove docs content #1426

Merged
merged 3 commits into from
Jan 16, 2023
Merged

Remove docs content #1426

merged 3 commits into from
Jan 16, 2023

Conversation

Ekrekr
Copy link
Contributor

@Ekrekr Ekrekr commented Jan 10, 2023

Content is hosted from the main_v1 branch

@Ekrekr Ekrekr requested a review from BenBirt January 10, 2023 11:18
Copy link
Collaborator

@BenBirt BenBirt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before we delete this, do we actually have a plan for open-source documentation?

@Ekrekr
Copy link
Contributor Author

Ekrekr commented Jan 10, 2023

Before we delete this, do we actually have a plan for open-source documentation?

Because this content isn't being served from this branch I think it's confusing to have it around - and little of the current content will be re-used, because it's duplicate with what is available on https://cloud.google.com/dataform/docs. Roadmap:

  1. Keep current docs site running as is until legacy deprecation.
  2. In the meantime, create a new set of content specifically for open source in this branch - in markdown to start with.
  3. Host it via GitHub pages.

Outlined in https://docs.google.com/document/d/1FekFye75nT0I9LxmzRcd8iZbQeo2eiZWmjq13qcDnk4/edit#heading=h.5wtvzv51zb08

Copy link
Collaborator

@BenBirt BenBirt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty skeptical that we won't end up wanting a separate open-source docs site, but we can always bring this stuff back, I guess.

I am concerned that users (who do occasionally send us docs fixes/updates) will have no idea where/how to do that.

(Basically this seems a little over-bullish to me, but I'll leave this up to you.)

@Ekrekr
Copy link
Contributor Author

Ekrekr commented Jan 10, 2023

users (who do occasionally send us docs fixes/updates) will have no idea where/how to do that

This is my point - if they submit changes to the docs site here currently then they don't get published, because of main_v1.

We should be aiming for the use of our legacy docs site to decrease anyway - the links on our main readme now point to the Dataform on GCP docs https://github.com/dataform-co/dataform#data-modeling-with-dataform, and there's a big "send feedback" button for suggesting changes there.

Have added a note to the main readme to clarify this to readers.

@Ekrekr Ekrekr merged commit 878943c into main Jan 16, 2023
@Ekrekr Ekrekr deleted the remove-docs-content branch January 16, 2023 11:43
kolina pushed a commit that referenced this pull request Feb 15, 2023
* Remove docs content

* Add note to main readme about docs transition

* Fix contributing link
lewish added a commit that referenced this pull request Aug 31, 2023
* Progress adding sandbox API

* Fixed versioning

* Working example, if you make runfiles visible

* Working node spawn but sketchy af

* Progress using non fork method, but should probably ue forking method

* Fixed policy enforcement of node process, but still execveat error. Should try using bazel pathed nodejs version

* Added bazel build nodejs binary path instead of from usr root

* Progress, still stuch on execveat issue

* Tidy

* Swap to dynamic startup

* Progress passing compile.js to sandboxed script

* Swapped to running compile_loader.js, added custom absolute path finder. Still have fd > -1 error though

* Simplify input

* Strip fd stuff

* Tidy

* Remove runfiles build dependency

* Remove import, format

* Add fd inferral, binary run seems to be failing silently

* Update to latest SAPI, timplify and tidy

* Entry point experimentation

* Progress updating policies

* More allowed syscalls, with explanations

* Tidy policy order

* Working policies

* Progress setting up test

* Tidy some logging

* Update entry point to new worker bundle locale

* Bump protobufjs from 6.8.8 to 6.11.3 (#1346)

* Bump protobufjs from 6.8.8 to 6.11.3

Bumps [protobufjs](https://github.com/protobufjs/protobuf.js) from 6.8.8 to 6.11.3.
- [Release notes](https://github.com/protobufjs/protobuf.js/releases)
- [Changelog](https://github.com/protobufjs/protobuf.js/blob/v6.11.3/CHANGELOG.md)
- [Commits](protobufjs/protobuf.js@6.8.8...v6.11.3)

---
updated-dependencies:
- dependency-name: protobufjs
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update protobufjs dep versions

* Upgrade to protobufjs v7.0.0, node to 16x

* Bump bazel to version 5.2.0

* Bump test CI to correct bazel version

* 3.5.0

* 3.5.0

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Lewis Hemens <lewishemens@google.com>

* Require defaultLocation to be set in BigQuery (#1353)

* Enforce defaultLocation requirement in core and cli initialization

* Add a test

* Bump version

* Update publish script branch (#1354)

* Explicitly set path in dataform.json location validation error (#1355)

* Audit npm/yarn packages and fix (#1356)

* Audit npm/yarn packages and fix

* Remove docs build/push scripts and triggers

* Cut version 2.0.0 (#1361)

* Update @dataform/core to never return a graph with non-unique action names. (#1366)

* Update @dataform/core to never return a graph with non-unique action names.

* Include a compilation error for each removed action.

* Rename a shadowed variable

* Increment DF_VERSION

* Bump moment-timezone from 0.5.28 to 0.5.37 (#1370)

Bumps [moment-timezone](https://github.com/moment/moment-timezone) from 0.5.28 to 0.5.37.
- [Release notes](https://github.com/moment/moment-timezone/releases)
- [Changelog](https://github.com/moment/moment-timezone/blob/develop/changelog.md)
- [Commits](moment/moment-timezone@0.5.28...0.5.37)

---
updated-dependencies:
- dependency-name: moment-timezone
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Upgrade vm2. (#1372)

* Upgrade vm2.

* bump version

* Publish core.proto within the @dataform/core package (#1378)

* Publish core.proto within the @dataform/core package

* Bump version

* Update readme.md (#1397)

* Update readme.md

* Update readme.md

* Enable `main` compilation by default for all @dataform/core versions >= `2.0.4`. (#1401)

* Use main compilation by default for all @dataform/core versions >= 2.0.4.

* use semver.subset

* Bump decode-uri-component from 0.2.0 to 0.2.2 (#1404)

Bumps [decode-uri-component](https://github.com/SamVerschueren/decode-uri-component) from 0.2.0 to 0.2.2.
- [Release notes](https://github.com/SamVerschueren/decode-uri-component/releases)
- [Commits](SamVerschueren/decode-uri-component@v0.2.0...v0.2.2)

---
updated-dependencies:
- dependency-name: decode-uri-component
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Main <- Master (#1407)

* typo (#1364)

* Buttons label fix (#1365)

"Read the blog post" was linking to the package page and "Visit the package page" was linking to the blog post.

* Update assertions.md (#1377)

Co-authored-by: Dolan <dolanstanley@gmail.com>
Co-authored-by: Stuart Lewis <75398266+stulew93@users.noreply.github.com>

* Make table level assertions have the same tags as the table (#1406)

* Make table level assertions have the same tags as the table

* add tag propagation post assertion adding

* Update version to 2.1.0

* Bump qs from 6.5.2 to 6.5.3 (#1408)

Bumps [qs](https://github.com/ljharb/qs) from 6.5.2 to 6.5.3.
- [Release notes](https://github.com/ljharb/qs/releases)
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](ljharb/qs@v6.5.2...v6.5.3)

---
updated-dependencies:
- dependency-name: qs
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump express from 4.17.1 to 4.17.3 (#1409)

Bumps [express](https://github.com/expressjs/express) from 4.17.1 to 4.17.3.
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/master/History.md)
- [Commits](expressjs/express@4.17.1...4.17.3)

---
updated-dependencies:
- dependency-name: express
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump go.mongodb.org/mongo-driver from 1.2.0 to 1.5.1 (#1357)

Bumps [go.mongodb.org/mongo-driver](https://github.com/mongodb/mongo-go-driver) from 1.2.0 to 1.5.1.
- [Release notes](https://github.com/mongodb/mongo-go-driver/releases)
- [Commits](mongodb/mongo-go-driver@v1.2.0...v1.5.1)

---
updated-dependencies:
- dependency-name: go.mongodb.org/mongo-driver
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Minor typo correction (#1360)

* Remove @dataform/sql package and common_v1 example (#1415)

* Remove @dataform/sql package and common_v1 example

* Remove SQL package from common_v2 example

* Remove build imports from tests of @dataform/sql that weren't doing anything

* Inline timestamp definition in non BQ tests

* Fix timestamp function case

* Bump json5 from 1.0.1 to 1.0.2 (#1416)

Bumps [json5](https://github.com/json5/json5) from 1.0.1 to 1.0.2.
- [Release notes](https://github.com/json5/json5/releases)
- [Changelog](https://github.com/json5/json5/blob/main/CHANGELOG.md)
- [Commits](json5/json5@v1.0.1...v1.0.2)

---
updated-dependencies:
- dependency-name: json5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Remove dead code paths and those not relating to ELT management (#1414)

* Remove code paths not relating to ELT management

* Remove Dataform SQL package

* Remove unused workspace imports

* Remove common_v1 example

* Remove some v1

* Remove @dataform/sql package and common_v1 example

* Remove SQL package from common_v2 example

* Remove build imports from tests of @dataform/sql that weren't doing anything

* Inline timestamp definition in non BQ tests

* Fix timestamp function case

* Restore v1 backwards_compatibility tests

* Environments and schedules (#1418)

* Remove DB tools dependence on `/api` (#1417)

* Remove DB tools dependence on @dataform/api in prep for removal

* Remove unneccessary options

* Remove more

* More simplifying

* Fix import order

* Tweak postgres connection wait

* Remove console log

* Remove promise pool from presto test

* Remove @dataform/assertion-utils in favor of auto-assertions (#1419)

* Remove @dataform/assertion-utils in favor of auto-assertions

* Fix action count

* Fix quotes in tests

* Fix bazel build //... for our distro by bumping versions (#1421)

* Fix bazel build //... for our distro by bumping versions

* Bump bazel to 5.4.0

* Update contributing (#1425)

* Remove docs content (#1426)

* Remove docs content

* Add note to main readme about docs transition

* Fix contributing link

* Update base dataform image to use more recent Node version (#1427)

* Update base dataform image to use more recent Node version - also bump some required deps

* Bump go version too

* Upgrade vm2 version. (#1431)

* Added schema function (#1432)

* Removed redundant toResolvable calls

* Refactored resolve/normalize in session

* Added schema function to common context objects

* Removed unused session import

* Bumped minor version

* Simplified schema assertions in core.spec.ts

* Refactored normalize* => finalize* in session helpers

* Refactored with/without suffix tests to run as parameterized tests

* Removed redundant 'Test' import

* Bump ua-parser-js from 0.7.28 to 0.7.33 (#1433)

Bumps [ua-parser-js](https://github.com/faisalman/ua-parser-js) from 0.7.28 to 0.7.33.
- [Release notes](https://github.com/faisalman/ua-parser-js/releases)
- [Changelog](https://github.com/faisalman/ua-parser-js/blob/master/changelog.md)
- [Commits](faisalman/ua-parser-js@0.7.28...0.7.33)

---
updated-dependencies:
- dependency-name: ua-parser-js
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Refactored name() function to finalize name (#1435)

* Database function (#1438)

* Refactored name() function to finalize name

* Added database() function

* Bumped minor version to 2.3.0

* Refactored database() to add an error when the database is undefined

* Inlined database & ref error messages

* Change sample command to run tests (#1439)

* Update contributing.md (#1446)

* Added includeDependents flag to run command

* Highlight that dev branches should branch off main

* Fix missing credentials error by adding it as the default option (#1447)

* fix missing include dependents (#1448)

* fix missing include dependents

* bump version

* Fix docker cmd for image (#1456)

* Fix docker cmd for image

* Replace with empty cmd instead of help

* Added enum for table's type field (#1453)

* added enum for table's type field

Signed-off-by: Nick Nalivaika <ikolina@google.com>

* update version

Signed-off-by: Nick Nalivaika <ikolina@google.com>

* simplify code by filling enumType field for tables when loading compiled graphs

* small fix for biquery adapter

* make linter happy

* fix integration tests

* do not call setOrValidateTableEnumType when check for table validity, just check enumType value

---------

Signed-off-by: Nick Nalivaika <ikolina@google.com>

* Adds a flag 'track' to allow users to configure analytics with flags (#1450)

* Adds a flag 'no-track' to allow users to configure analytics with flags

* Rework so that no-track bypasses settings.json

* Linting

* Move to --track, fix an issue with --track=true, doc changes

* Update boolean type

* Update version

* Remove code which always executes auto-assertions. (#1460)

* Remove code which always executes auto-assertions.

* fix failing test

* Throw an error for non-string `vars`. (#1461)

* Add job prefix option to the CLI (#1449)

* Add job prefix option to the CLI

* format

* Fix removal of dataform- prefix

* Fix import order

* Import order take 2

* Default empty execution options fix

* fix parameter placement

* Bump vm2 from 3.9.13 to 3.9.15 (#1473)

Bumps [vm2](https://github.com/patriksimek/vm2) from 3.9.13 to 3.9.15.
- [Release notes](https://github.com/patriksimek/vm2/releases)
- [Changelog](https://github.com/patriksimek/vm2/blob/master/CHANGELOG.md)
- [Commits](patriksimek/vm2@3.9.13...3.9.15)

---
updated-dependencies:
- dependency-name: vm2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump vm2 from 3.9.15 to 3.9.16 (#1476)

Bumps [vm2](https://github.com/patriksimek/vm2) from 3.9.15 to 3.9.16.
- [Release notes](https://github.com/patriksimek/vm2/releases)
- [Changelog](https://github.com/patriksimek/vm2/blob/master/CHANGELOG.md)
- [Commits](patriksimek/vm2@3.9.15...3.9.16)

---
updated-dependencies:
- dependency-name: vm2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump vm2 from 3.9.16 to 3.9.17 (#1478)

Bumps [vm2](https://github.com/patriksimek/vm2) from 3.9.16 to 3.9.17.
- [Release notes](https://github.com/patriksimek/vm2/releases)
- [Changelog](https://github.com/patriksimek/vm2/blob/master/CHANGELOG.md)
- [Commits](patriksimek/vm2@3.9.16...3.9.17)

---
updated-dependencies:
- dependency-name: vm2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Remove/update dependencies resulting in updated fsevents dependency version. (#1481)

* Add `schema` & `database` to contextFunctions in compile time (#1485)

* Upgrade vm2 to `3.9.19`. (#1491)

* Remove old vm2 dependency. (#1493)

* Bump loader-utils from 1.2.3 to 1.4.2 (#1492)

Bumps [loader-utils](https://github.com/webpack/loader-utils) from 1.2.3 to 1.4.2.
- [Release notes](https://github.com/webpack/loader-utils/releases)
- [Changelog](https://github.com/webpack/loader-utils/blob/v1.4.2/CHANGELOG.md)
- [Commits](webpack/loader-utils@v1.2.3...v1.4.2)

---
updated-dependencies:
- dependency-name: loader-utils
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump minimatch from 3.0.4 to 3.1.2 (#1495)

Bumps [minimatch](https://github.com/isaacs/minimatch) from 3.0.4 to 3.1.2.
- [Changelog](https://github.com/isaacs/minimatch/blob/main/changelog.md)
- [Commits](isaacs/minimatch@v3.0.4...v3.1.2)

---
updated-dependencies:
- dependency-name: minimatch
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Release `2.5.0`: to allow use of methods added to SQLX in #1485. (#1498)

* Separate sqlx build target into sqlx and format (#1499)

* Separate sqlx build target into sqlx and format

* Fix build dependencies that used //sqlx

* Update sql-