Add vsce pre-release step to CI

[fhammerschmidt]

This is a first step to having automatic pre-releases published in the VSCode marketplace.

Somebody with access rights needs to create a secrets environment in this GitHub repository and add (one of) the ReScript team's Personal Access Token (PAT) as an environment variable, named MARKETPLACE_TOKEN .

EDIT:
This PR is basically done, but there is probably still room for some fine-tuning. As mentioned above, someone with write access needs to provide the token(s).

Also the chenglou92.rescript-vscode needs to be updated to whatever will be the VSCode publisher name afterwards.

[zth]

Awesome! How does versioning work here? I'm not fully read up on pre releases in the marketplace.

[fhammerschmidt]

Well, I did not account for that yet, but I guess we also need to have version numbers for pre-releases, separated from main releases. What rust-analyzer does is basically what is also suggested by the docs, i.e. having a version where the minor part is an odd number that is always one higher than the minor part of the main release.

E.g. the pre-release version for rescript-vscode now would be 1.3.1 as the latest release is currently 1.2.1 and so on.

Something is needed to increment the patch number automatically. This can be done with vsce as well.
Otherwise we will always overwrite the current version and users will not be notified for updates.
Ended up fetching the current published version from the VSCode marketplace directly and doing everything in a custom versioning script. It's more transparent that way.

[fhammerschmidt]

Here is a chart with the full workflow for a merge into master (not for a PR build):

flowchart TD
A[pull request gets merged into master branch]
A-->B[build analysis binaries]

subgraph ci [ ]
subgraph textNode1 [ci.yml]
style textNode1 fill:none,stroke:none
end
B-->C[fetch current highest version - 1.4.0]
C-->D[bump-version.js script increments version - 1.4.1]
D-->E[package VSIX with vsce - 1.4.1]
E-->F[prepublish VSIX with vsce - 1.4.1]
E-->G[add packaged VSIX to releases page - Latest master]
end

F & G -->  Done

Loading

[fhammerschmidt]

@zth @cristianoc feel free to review and roast me!
Edit: also pinging @cknitt

[cristianoc]

@fhammerschmidt how does one test this? With a PR on top of this PR that enables for that branch?
Also, some time ago you mentioned you needed some token to continue. Is it still the case, or can this just run as-is?

[fhammerschmidt]

Yes, you can create a branch and add it's name to the push branches in ci.yml. However, as the important bits happen on the master branch, it may be easier in a fork, or all the github.ref == 'refs/heads/master' need to be synced accordingly.

There are now two tokens at play here:

• GITHUB_TOKEN this one should be provided by GitHub automatically.
• MARKETPLACE_TOKEN: This one is still needed for the actual publish to happen. See initial comment.

[zth]

Can we test this in the current namespace we have? I'm thinking it'd be good to do that, get one final release out, and then move to our new real Marketplace ReScript org. Would that essentially be merging this and ensuring the marketplace token is available?

[fhammerschmidt]

I think so, but I cannot guarantee it.

But I guess using the old namespace as a testbed makes a lot of sense. We can then continue on a clean slate when we figured the whole process out.

[cristianoc]

We're a little behind with new releases. Don't know if this has to complete before a new release is done.

[fhammerschmidt]

It is completely independent of main releases, which will still be done manually after this PR gets merged.

[zth]

@fhammerschmidt is this understanding correct:

1. We merge this and add the token you talked about to secrets
2. Provided we're still at 1.3.0 when that happens, the next time we push to master, a 1.4.1 prerelease will automatically be pushed to the Marketplace
3. Nothing is added yet with regards to doing real releases (1.4.0 stable in this case)

...?

[fhammerschmidt]

Ad 1.: Yes, but first add the token and then merge it, the next master build should automatically publish a pre-release then.

Ad 2.:
No, the first case is special. We do not have a pre-release published yet, so VSCE will give us 1.3.0 as the latest published version (pre-releases and releases are lumped together in the versions array) . If you give the bump-version script a 1.3.0 it yields 1.4.0 which is correct because that one is not in use yet.

In a subsequent push to master, it will retrieve the most recent version again which would then be the pre-published 1.4.0, so it will bump to 1.4.1. The above diagram was slightly wrong, but I corrected it now to reflect the publish step after the initial one.

Ad 3.: The only thing with regards to main releases is that you need to keep in mind that we would be forced to use the odd minor numbers for them, while using the even ones for pre-releases, e.g.

Release	Pre-Release
1.3.0	1.4.0
1.3.0	1.4.1
1.3.0	1.4.2
1.3.0	1.4.3
1.3.1	1.4.4
1.5.0	1.6.0
2.0.0	2.1.0

The bump-version script actually missed the case when there is a higher (major or minor) version in package.json, but not in the marketplace, I fixed that now.

But I also realized, that it would be better to start from an even number main release, like 1.4.0
Because when you bump the major, e.g. to 2.0.0, the pre-releases will be the odd numbers then.

I hope that was not too confusing.

[zth]

@fhammerschmidt no that's great! I'm going to get the token sorted and we can merge this.

[fhammerschmidt]

@zth I really think it is inconsistent when we start from 1.3.0. Maybe we can first release a 1.4.0 and merge it afterwards?

If you however don't care that the pre-releases have even minor numbers for 1.x.x but odd numbers for 2.x.x (and every subsequent major bump) then I rest my case.

[zth]

@fhammerschmidt sorry, missed the last part of your message. Yes, I agree. Let's get a 1.4.0 out first, and then merge this after.

[zth]

@fhammerschmidt setting up the token + merging this tomorrow.

[zth]

Turns out I don't have permissions to produce that marketplace token. Pinging @ryyppy who might have more luck in that regard.

[ryyppy]

@fhammerschmidt just created a MARKETPLACE_TOKEN and created the Action secret for this repository. Let me know if this worked!

[fhammerschmidt]

Cool.

@zth If you want I can create another PR just to test if the token works in CI (using https://fig.io/manual/vsce/verify-pat).

[zth]

Cool.

@zth If you want I can create another PR just to test if the token works in CI (using https://fig.io/manual/vsce/verify-pat).

Go for it!

[fhammerschmidt]

Ok, unfortunately you need to do a vsce login first, which would mean we'd need to store another credential in the repository.

I tried this with a dummy token:

➜  rescript-vscode git:(vsce-test) npx vsce verify-pat chenglou92 --pat lol
 ERROR  The Personal Access Token verification has failed. Additional information:

Error: {"$id":"1","innerException":null,"message":"TF400813: Resource not available for anonymous access. Client authentication required.","typeName":"Microsoft.TeamFoundation.Framework.Server.UnauthorizedRequestException, Microsoft.TeamFoundation.Framework.Server","typeKey":"UnauthorizedRequestException","errorCode":0,"eventId":3000} 

@ryyppy you could verify it locally and then I'd say we just byte the bullet, @zth ?

EDIT: It may still work with the correct token. Trying it in #499.

[zth]

@fhammerschmidt what's the status now, are we waiting for @ryyppy to reply in the other PR?

[fhammerschmidt]

As you like. I believe he was or is still sick and was not able to answer.

[fhammerschmidt]

So, to clarify: if you want to be sure the token is valid, you should wait for Patrick. Otherwise feel free to pull the trigger.

[zth]

@fhammerschmidt mind rebasing on master again? Inclined to merge this after that and we'll just try to fix whatever comes up if things don't work like we want to.

[fhammerschmidt]

I am currently on mobile. Could do it in a couple of hours.

[fhammerschmidt]

Rebased!

[zth]

Fire in the hole!