Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deploy @deephaven/jsapi-types during release, provide tools for nightly builds #4907

Merged
merged 20 commits into from
Feb 9, 2024
Merged

Deploy @deephaven/jsapi-types during release, provide tools for nightly builds #4907

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
ce567ff
Add notes on releasing jsapi-types
niloc132 Nov 28, 2023
6e745d7
Revert package-lock.json until we update our tagged node image
niloc132 Nov 28, 2023
8e475d0
Update node image, update builds to use it and enforce npm/node vers
niloc132 Nov 29, 2023
2ab6475
Avoid extra copy of ts/js files
niloc132 Nov 30, 2023
a93d93c
Stubbed in vers bump
niloc132 Nov 30, 2023
eacf5f4
better exclude?
niloc132 Nov 30, 2023
ae30a5e
forgotten out
niloc132 Nov 30, 2023
cfba8dc
unify version format
niloc132 Nov 30, 2023
59263d2
Always update version, set up publish CI for js types
niloc132 Dec 1, 2023
169fbce
Merge branch 'main' into 4888-jsapi-types-followup
niloc132 Dec 15, 2023
b110c84
Remove duplicate engines entry
niloc132 Dec 15, 2023
bb56530
Merge branch 'main' into 4888-jsapi-types-followup
niloc132 Jan 25, 2024
74e95e0
Review feedback from Bender/Matt
niloc132 Jan 26, 2024
ffd4463
LayoutHints.frontColumns must be nullable
niloc132 Jan 29, 2024
732324e
Avoid covariance in generated typescript
niloc132 Jan 29, 2024
1a7739e
Review feedback
niloc132 Jan 29, 2024
c99c4d2
Merge branch 'main' into 4888-jsapi-types-followup
niloc132 Jan 29, 2024
d5d44fe
Bump ts codegen version to fix inheritance issue
niloc132 Jan 30, 2024
22e6dcd
Include .d.ts and npm module tarball as artifacts
niloc132 Feb 1, 2024
5caa58c
Merge branch 'main' into 4888-jsapi-types-followup
niloc132 Feb 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 17 additions & 4 deletions .github/workflows/publish-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,23 +39,29 @@ jobs:
.github/scripts/gradle-properties.sh >> gradle.properties
cat gradle.properties

- name: Setup Node for deployment to npmjs
uses: actions/setup-node@v4
with:
node-version-file: 'web/client-api/types/.nvmrc'
registry-url: 'https://registry.npmjs.org'

# TODO(deephaven-core#2614): Improve gradle/CI assemble and publishing of artifacts

- name: Publish to Maven Local
- name: Build all artifacts, publish to Maven Local
if: ${{ !startsWith(github.ref, 'refs/heads/release/v') }}
uses: burrunan/gradle-cache-action@v1
with:
job-id: publish-local
arguments: server-netty-app:build server-jetty-app:build py-server:build py-embedded-server:build py-client:build publishToMavenLocal
arguments: server-netty-app:build server-jetty-app:build py-server:build py-embedded-server:build py-client:build web-client-api:types:build publishToMavenLocal
gradle-version: wrapper

- name: Publish
- name: Build all artifacts, publish to Sonatype for staging to Maven Central
if: ${{ startsWith(github.ref, 'refs/heads/release/v') }}
uses: burrunan/gradle-cache-action@v1
with:
job-id: publish
# We need to be explicit here about no parallelism to ensure we don't create disjointed staging repositories.
arguments: --no-parallel server-netty-app:build server-jetty-app:build py-server:build py-client:build py-embedded-server:build publish
arguments: --no-parallel server-netty-app:build server-jetty-app:build py-server:build py-client:build py-embedded-server:build web-client-api:types:build publish
gradle-version: wrapper
env:
ORG_GRADLE_PROJECT_ossrhUsername: ${{ secrets.SONATYPE_USERNAME }}
Expand All @@ -74,6 +80,7 @@ jobs:
py/server/build/wheel/
py/embedded-server/build/wheel/
py/client/build/wheel/
web/client-api/types/build/*.tgz

- name: Publish deephaven-core to PyPi
if: ${{ startsWith(github.ref, 'refs/heads/release/v') }}
Expand All @@ -98,3 +105,9 @@ jobs:
user: __token__
password: ${{ secrets.PYDEEPHAVEN_PYPI_TOKEN }}
packages_dir: py/client/build/wheel/

- name: Publish @deephaven/jsapi-types to npmjs
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that I haven't tested this, all lifted from web-client-ui

if: ${{ startsWith(github.ref, 'refs/heads/release/v') }}
env:
NODE_AUTH_TOKEN: ${{ secrets.DEEPHAVENBOT_NPM_TOKEN }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll (I can do this) need to provide deephaven-core repo access to this secret before this action will succeed; I don't know how the npmjs secret management works, but I'm assuming this is a "heavy hammer", but we are okay w/ it @mofojed? The alternative would be a secret that is scoped explicitly for @deephaven/jsapi-types.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Man I wish npmjs had trusted publishers like PyPi does. Way better workflow.

Yes, the DEEPHAVENBOT_NPM_TOKEN is not scoped to a particular package, it is a "heavy hammer". You can create granular tokens that are scoped to specific packages, but I think this token scoped to our organization should be fine (organization wide secret) as it would be a pain to have to create new tokens for each package. Or we can have different tokens for the web-client-ui repo, deephaven-plugins repo, and whatever is published by deephaven-core, but then we need to update those tokens anytime we want to publish a new package in that repo (which happens from time to time in both web-client-ui and deephaven-plugins).
https://docs.github.com/en/actions/publishing-packages/publishing-nodejs-packages#publishing-packages-to-the-npm-registry

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done - secret now accessible from deephaven-core.

run: npm publish --tag latest web/client-api/types/build/deephaven-jsapi-types-*.tgz
devinrsmith marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should be referencing a package.json, not a tar file directly... does this actually work?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - this is the way to say "here's a finished tarball, go ahead and deploy it". The work of making the tarball is done inside of docker (so that we guarantee it builds the way we expect, that it will do on our own machines), so this npm invocation just has access to the finished tarball and the secret.

6 changes: 4 additions & 2 deletions RELEASE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,9 @@ If this step fails, the deephaven-server wheel from the "Upload Artifacts" step
The "Publish pydeephaven to PyPi" uploads the pydeephaven wheel to [PyPi](https://pypi.org/project/pydeephaven/).
If this step fails, the pydeephaven wheel from the "Upload Artifacts" step can be uploaded manually.

The "Publish @deephaven/jsapi-types to npmjs" uploads the TypeScript tarball to [NPM](https://www.npmjs.com/package/@deephaven/jsapi-types).
If this step fails, the deephaven-jsapi-types tarball from the "Upload Artifacts" step can be uploaded manually.

Once the workflow job is done, ensure all publication sources have the new artifacts.

### 5. Download artifacts
Expand Down Expand Up @@ -225,7 +228,7 @@ Create a new [GitHub release](https://github.com/deephaven/deephaven-core/releas

The convention is to have the Release title of the form `vX.Y.Z` and to autogenerate the release notes in comparison to the previous release tag. Question: should we always generate release notes based off of the previous minor release, instead of patch? Our git release workflow suggests we may want to do it always minor to minor.

Upload the Deephaven server application, deephaven-core wheel, pydeephaven wheel, and SBOM artifacts. Also, upload the C++, Java, Python, R and TypeScript docs artifacts.
Upload the Deephaven server application, deephaven-core wheel, pydeephaven wheel, @deephaven/jsapi-types tarball, and SBOM artifacts. Also, upload the C++, Java, Python, R and TypeScript docs artifacts.
(These are the artifacts downloaded in Step #5)

Hit the GitHub "Publish release" button.
Expand Down Expand Up @@ -269,7 +272,6 @@ mention the version explicitly. These files are listed below:
#
gradle.properties
R/rdeephaven/DESCRIPTION
web/client-api/types/package.json
```

This leaves the files "ready" for the next regular release, and also ensures any build done from
Expand Down
4 changes: 2 additions & 2 deletions docker/registry/node/gradle.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
io.deephaven.project.ProjectType=DOCKER_REGISTRY
deephaven.registry.imageName=node:14
deephaven.registry.imageId=node@sha256:a158d3b9b4e3fa813fa6c8c590b8f0a860e015ad4e59bbce5744d2f6fd8461aa
deephaven.registry.imageName=node:20
deephaven.registry.imageId=node@sha256:445acd9b2ef7e9de665424053bf95652e0b8995ef36500557d48faf29300170a
2 changes: 2 additions & 0 deletions proto/raw-js-openapi/.npmrc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
engine-strict=true
provenance=true
1 change: 1 addition & 0 deletions proto/raw-js-openapi/.nvmrc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
v20.10.0
5 changes: 3 additions & 2 deletions proto/raw-js-openapi/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,9 @@ def webpackSources = Docker.registerDockerTask(project, 'webpackSources') {
into 'raw-js-openapi/'
}
from ('Dockerfile')
from ('../package.json')
from ('../package-lock.json')
from ('package.json')
from ('package-lock.json')
from ('.npmrc')
}
parentContainers = [ Docker.registryTask(project, 'node') ]
imageName = 'deephaven/js-out:local-build'
Expand Down
Loading
Loading