Automate publishing process via CI

[mtrezza]

New Feature / Enhancement Checklist

• I am not disclosing a vulnerability.
• I am not just asking a question.
• I have searched through existing issues.

Current Limitation

Currently, publishing a new version of the Parse Flutter SDK on pub.dev is done manually. This poses several issues:

• Inefficient, as manual steps are required
• Possible divergence of GH versions and pub.dev versions
• Separate permissions management, instead of managing publishing permission as part of GH team permissions

Feature / Enhancement Description

Automate publishing process via GitHub CLI, analogous to, for example publishing parse-server on npm.

TODOs:

• Add auto-publishing to pub.dev (triggered on GitHub release) ci: automate publishing to pub.dev #719
• Validate by creating a manual release (3.1.x or 3.2.0) -- version tag without V prefix!
• Add auto-release workflow (branch renaming, changelog-generation, etc)
• Validate by creating a automatic release via PR-merge

Example Use Case

n/a

Alternatives / Workarounds

n/a

3rd Party References

n/a

[mtrezza]

@RodrigoSMarques It would be great if you could have a look into this, since you seem to be most familiar with the pub.dev publishing process. I am of course available to assist from the GitHub CI side. I think a good starting point would be to look at the parse-server workflows, specifically, the release flow.

cc @phillwiggins

[fischerscode]

@mtrezza @RodrigoSMarques
If you want, I could have a look at this, too.

[fischerscode]

Some more information as I'm now home:

Currently there is no recommended way to publish dart/Flutter packages to pub.dev automatically. The community uses the access and refresh token pub publish created at the first publish. Technically those tokens are not package but account bound and belong to the personal account. This meas, although they work for publishing, in theory they are a security risk, as someone with push access to this repo could extract the (GitHub) Secret containing those tokens and use them in order to authenticate as the user who created them.
As an example: If @RodrigoSMarques would create the GitHub Secrets using his personal account, anybody who has push access to this repo could steal those and publish to his package.
Please note: This is just theory and I don't think anybody would do such a thing (right?) esp. as the fake release could be easily reverted by the pub.dev team. But never the less, it is worth considering the potential risk.

The options I see:

1. Accept the risk and use the this action. (Publish CI as soon as possible)
2. Create a Google Account that has just access to those packages. (Nothing relevant could be stolen.)
3. Publish manually until an official way to publish using CI is added. See this issue (Personally I am using this way for a small package.)

Some thinks the automation should do:

• On pre-release: make sure to only publish prereleases. (Check version)
• On release: makes sure not to publish as a prerelease. (Check version)
• Release the dart package first and the flutter package second (might fail otherwise as flutter might/should depend on dart).

[mtrezza]

@fischerscode Thanks for the analysis.

Yes, for now we'll publish manually until we found a feasible way to automate this.

We should do a careful risk analysis regarding automated publishing and permission exposure. The risk is not only malicious intent by a single person, but maybe more accidental actions or compromised 3rd party accounts, because we cannot control how someone stores their own account credentials.

@RodrigoSMarques Do you have a next release planned, where we can test how we coordinate a manual release?

[Mabenan]

Maybe i can give some new information on stealing secrets from github actions as i already tried this myself to get a lost key back. First of it is not so easy as every secret used in github actions can not be simply printed. I only could get by echo it to a txt file and upload this text file somewhere. Maybe we there are ways of securing the access to the secrets by restricting the environment to branches (or maybe cleanup and restrict the branch creation all together) and then restrict the merging to these branches.

[mtrezza]

We already use GitHub secrets for NPM and docker publishing token in other repos; we can assume the feature is safe as that is exactly what it's designed for. As I understand it, secrets are only accessible on the original repo branches, not on fork branches or PRs. We just have to be careful about what we merge, but that we have to be anyway.

That is one of the reasons we changed the permission model in our repos and are now giving only a select few people write access to the repositories, where before, every org member had write access.

[RodrigoSMarques]

Hi @mtrezza.

How can I contact you so we can talk about releasing new versions of the package?

Since Phil left the project we haven't published any updates, because I don't know how the process was defined.

I believe that as the package is now publisher verified, you could invite me to publish the version.

I await your contact.

[mtrezza]

Maybe we can already set up the CI to automate the publishing in the future? Do you want to look into that and open a PR for a GitHub workflow? Feel free to contact me on Slack.

[RodrigoSMarques]

Hi @mtrezza .
I was away for vacation.

I don't know if there is an option for automated deployment.

My sugestion. Do the update manually and then try to automate.

We have several bugs to be updated and as I have other projects, time is limited to check before the update.

My sugestion. Do the update manually and then try to automate.

We have several bugs to be updated and as I have other projects, time is limited to check before the update.

In this issue we have the analysis about the deployment automation.

[mtrezza]

Sure, @RodrigoSMarques I think we could give you temporary maintainer access if you would be willing to publish a release? Then we have more time to figure out how to automate this.

[mtrezza]

@RodrigoSMarques Friendly ping in case you overlooked my previous comment.

[RodrigoSMarques]

riendly ping in case you overlooked my previous comment.

Good Morning.
I did not forget. I'm in a period of a big holiday here in my region in Brazil.

Add my email (my github username @gmail.com) to pub.dev and I'll be able to publish the new version of the package.

He was involved in several personal projects.

I will try to get back to support for this package.

[RodrigoSMarques]

@mtrezza Friendly ping in case you overlooked my previous comment.

[mtrezza]

I should be able to give you permissions later today, I'll ping you here.

[mtrezza]

@RodrigoSMarques you've been granted publisher permission, please let me know if it works and thanks for your help to publish manually.

To hopefully gather more interest in automating publishing, I've added a bounty label; @fischerscode you wrote a comprehensive analysis, would you want to take look and open a PR?

[fischerscode]

@mtrezza I'll have a look at it. Do you think it would be best to publish to pub.dev once a new github release has been created or should we consider automatic github releases as well?

We could consider creating automatic patches to update the dependencies.
Also, we could create releases automatically once a pr has been merged to eg. master. But I would have to research if there are any best practices.

CC @RodrigoSMarques

[mtrezza]

Amazing @fischerscode, we should add auto-release to this SDK as we are moving all repositories to auto-release. That means on each commit (depending on the commit message), a release will be automatically published.

The auto-release mechanism is essentially the same as in the Parse JS SDK, but it comes with its own complexities, so I would break it into two PRs:

1. Automate pub.dev publishing when a new GH release is published: we still need to manually add a changelog entry with every PR and determine the next version number based on the changelog entries.
2. Move to auto-release: it should be easy to refactor the CI workflow from step (1); if you are interested you can do that too based on how we already do it in the Parse JS SDK and I'll be on standby for any questions, otherwise I can take over this part; for auto-release we use semantic-release across all repos, also for non-JS packages, see for example the Parse Android SDK.

[fischerscode]

@mtrezza I've created PR #719. It addresses the first point.
Pub.dev does not support token authentication at the moment, as a result, the action requires the refresh and access token (OAuth2) of a Google account, that can publish. I would suggest creating a bot account, granting it publish rights (pub.dev), use it to log in to google using pub and then create the secrets PUBLISH_OAUTH_ACCESS_TOKEN, PUBLISH_OAUTH_REFRESH_TOKEN based on credentials.json.

https://github.com/k-paxian/dart-package-publisher#accesstoken

I assume you are one of the administrators of the publisher. So I think it would be best, if you would create the bot account.

[fischerscode]

2. if you are interested you can do that too based on how we already do it in the Parse JS SDK and I'll be on standby for any questions

I could do that, but (as you wrote) we should focus on Nr. 1 first.

[mtrezza]

@fischerscode That's amazing and amazingly quick! I will create a Google account for that and add the secrets so maybe we can even try that out with our next publishing.

@RodrigoSMarques You may want to wait with manual publishing, so we try this out?

[RodrigoSMarques]

@fischerscode That's amazing and amazingly quick! I will create a Google account for that and add the secrets so maybe we can even try that out with our next publishing.

@RodrigoSMarques You may want to wait with manual publishing, so we try this out?

We can wait to test.

Who was having problems, now has a version to continue through the Fork

[parse-github-assistant]

Thanks for opening this issue!

• 🎉 We are excited about your ideas for improvement!

[mtrezza]

Do I understand it correctly, that these 2 packages (dart, flutter) will always have the same version number?

[mtrezza]

pub publish wont upload the lock file. So we can't publish it.

If that is really a limitation of pub.dev, then that seems like a flaw to me. It basically means that the CI we run is almost irrelevant, because the developers using the Parse Flutter SDK may be using other dependency versions which did not pass our CI. It defies the purpose of the lock file, which is there for exactly that reason.

Yes, the caret means "no breaking change" upgrades, but any version upgrade (major, minor or patch) can potentially introduce a bug. That bug could be detected in our CI, but since we don't run the CI with these specific dependencies, we cannot know.

Maybe we get in touch with pub.dev to inquire about this limitation?

I guess in the meantime we can just publish with caret to finally test our the release process.

[fischerscode]

If that is really a limitation of pub.dev, then that seems like a flaw to me.

You can download the code uploaded to pub.dev. It doesn't contain the lock file.

Yes, the caret means "no breaking change" upgrades, but any version upgrade (major, minor or patch) can potentially introduce a bug.

Maybe we get in touch with pub.dev to inquire about this limitation?

I don't think they will change how pub is working, I think the team of google really thought about what is a good behavior.
And to be honest, it is possible to pin the versions by using 1.2.3 instead of ^1.2.3, it's not just how most packages do it and how it's probably recommended.

Of course the approach of dynamically resolving the versions of the dependencies has a downside when it comes to the situation you described, but I think it is the best way they could have done it when they designed pub, since it allows the usage of irregularly updated packages while using patched nested dependencies.
So especially when it comes to security, I think it is a good approach.

If you feel like it would be better for the overall concept of the parse-platform, we could also introduce *packagename*_fixed packages, that have the dependencies fixed. But personally I think it should not be needed, since the dependencies we use (hopefully) have good testing.

I guess in the meantime we can just publish with caret to finally test our the release process.

👍

[mtrezza]

Quick update: I'll do a test release in the coming days so we can verify the manual release mechanism.

[mtrezza]

Test status:

• Leave version unchanged; keep dry-run in workflow; create GH release to see what happens
• Update version, keep dry-run, see what happens
• Remove dry-run and publish

Insights:

• Dry run locally prevents publishing a version that already exists in the registry:

  Local version: [3.1.0]
  Remote version: [3.1.0]
  Remote & Local versions are equal, skip publishing.

• Dry-run for flutter may not work as part of release workflow, because as long as dart dependency is not published, resolving fails; maybe resolving can be skipped for dry-run? Anyway, deactivating dry-run job for now.

  ERR : Because parse_server_sdk_flutter depends on parse_server_sdk ^3.1.1 which doesn't match any versions, version solving failed.

• Not sure how we can publish different versions for flutter / dart if GitHub allows only 1 version to be specified for a release. Since this is a fake monorepo, it may not even be possible without release-automation logic. For manual release it also wouldn't work to publish the dart and flutter independently, because the git tag history would be the same.

• Dart publishing seems to work, skipped for next release, as 3.1.1 is a flutter-only release. The following warning is reported:

  • 3 checked-in files are ignored by a .gitignore.
    Previous versions of Pub would include those in the published package.
    Consider adjusting your .gitignore files to not ignore those files, and if you do not wish to
    publish these files use .pubignore. See also dart.dev/go/pubignore

    Files that are checked in while gitignored:
    example/linux/flutter/ephemeral/.plugin_symlinks/connectivity_plus_linux
    example/linux/flutter/ephemeral/.plugin_symlinks/path_provider_linux
    example/linux/flutter/ephemeral/.plugin_symlinks/shared_preferences_linux

@fischerscode What do you think about the warning above? Do we need these 3 files checked in and published? Full log here.

[fischerscode]

@mtrezza Sorry for the late response...

Dry run locally prevents publishing a version that already exists in the registry:

👍

Dry-run for flutter may not work as part of release workflow, because as long as dart dependency is not published, resolving fails; maybe resolving can be skipped for dry-run? Anyway, deactivating dry-run job for now.

Makes sens, forgot about that... Don't think there is a way around that. Since dry-run runs the test, as far as I know.

Not sure how we can publish different versions for flutter / dart if GitHub allows only 1 version to be specified for a release. Since this is a fake monorepo, it may not even be possible without release-automation logic. For manual release it also wouldn't work to publish the dart and flutter independently, because the git tag history would be the same.

Maybe tag with dart-vx.y.z and flutter-vx.y.z (or something similar)?

Dart publishing seems to work, skipped for next release, as 3.1.1 is a flutter-only release. The following warning is reported:

No priority, since it is only the example, but I think it would be best to remove them from the .gitignore, since it seems to be suggested.

[mtrezza]

They are actually symlinks, so we won't check them in. The symlink that are currently checked in point to:

/Users/rodrigo/flutter/.pub-cache/hosted/pub.dartlang.org/path_provider_linux-2.0.0/

[mtrezza]

I managed to run https://github.com/parse-community/Parse-SDK-Flutter/actions/runs/2404786674 successfully in dry-run. II think we are good now to make a release.

The GitHub Action will only publish the dart or flutter package if the local version is different from the currently published version, independently for each package. For example, if there has been only a change in the flutter package, it would publish only the flutter package, skipping the dart package.

Releasing in this repository is currently a challenge, because this "monorepo" doesn't actually use a monorepo framework and requires creating manual GitHub releases. Probably the biggest confusion is caused when using a common version for both packages, as has been done until now.

For example, for the next release (3.1.1) there has only been a change in the flutter package. The flutter package changelog shows the new version with a description of the fix. But what should the dart package changelog show? It also needs to show the new version, but there has been no change between the previous and the new version. It would be unusual to write "no changes to previous version" in the changelog, and that would happen frequently here. In the past, the solution to that was to write the flutter package changes into the dart package changelog. But that is confusing as well, because these changes are unrelated to the dart package.

So from hereon we'll separate release version and changelog per package.

[mtrezza]

@fischerscode @RodrigoSMarques It seems we have successfully published flutter-3.1.1.

Could you please check whether the published package is OK?

The release process currently works like this:

1. Merge PR that includes version increment and changelog entries for the specific package:
   • only add changelog entries to the package to which the changes relate (should already be added with every PR)
   • make sure changelog entries conform to Parse Platform org-wide format, see Parse Server changelog
   • only increment the version of a package if the package has changelog entries
   • increment the version according to the type of changes:
     • major version increment if there is a breaking change, or
     • minor version increment if there is at least one feat entry, or
     • patch version increment if there are only fix entires
   • make sure the comparison link at the top of the changelog has the correct tags
2. Create a GitHub release
   • Tag: <package>-<version>, e.g. dart-1.2.3 or flutter-1.2.3
   • Description: Copy/paste the changelog entry from the package
3. Trigger release (the following steps are not necessary anymore due to ci: add auto-release #758)
   • Manually trigger the GitHub Action workflow release-manual
     • Tag: same as release tag, e.g. dart-1.2.3
     • Package: choose the package to relase, e.g. dart

Note:

• If a change spans both packages, the dart package has to be published first. As the flutter package depends on the dart package, publishing the flutter package that depends on an unpublished version of the dart package will fail and not publish to the registry.

Potential future improvements:

• Auto-trigger the publish workflow based on the release tag prefix, makes step# 3 an automatic step --> done with ci: add auto-release #758
• Add a monorepo auto-release framework that automates step# 1 and 2

[fischerscode]

Could you please check whether the published package is OK?

Looks good to me 🚀

make sure changelog entries conform to Parse Platform org-wide format, see Parse Server changelog

When is the hash/link of the commit added? After a PR got merged?

Manually trigger the GitHub Action workflow release-manual

Is it allowed for everyone having the permission to create a GitHub Release?

[mtrezza]

When is the hash/link of the commit added? After a PR got merged?

Well, you are right, the changelog entry is usually auto-generated after the merge. As long as we're doing manual releases, we can simplify the changelog entry I guess and omit the commit hash.

Is it allowed for everyone having the permission to create a GitHub Release?

Actually, this was more for self-documentation. Only the maintainer team can trigger releases, of which only org owners are members. GitHub doesn't provide a fine-grained permission model, so we are restricting write permissions for the moment. We are looking into making this more automated in the future. The process in this repo is currently as with other repos in the org:

1. PR submitted
2. PR author asks @parse-community/android-sdk-review for review
3. Members of that team do the reviews (the more the better); reviewers usually engage and give feedback even before a review is formally requested, to guide the PR author down the right path
4. Once all doubts are cleared, someone asks @parse-community/android-sdk-maintenance for merge

[mtrezza]

@parse-community/flutter-sdk Does anyone know of a dart-specific framework that is used to manage mono-repositories?

[fischerscode]

The firebase packages are managed by utilizing melos (docs).

Since I found it (#668 (comment)) I've played a bit with it. And to be honest, it seems perfect for managing monorepos without much work. As a bonus it supports linking the packages using a "path"-dependency for development without changing the pubspec.yaml.

[mtrezza]

Does it generate a changelog as well? That is currently the most work for us when it comes to making a release. In the other framework we use across the org the changelog is created automatically from the commit messages.

Edit: It does, as described in the docs.

Does anyone want to create a PR to set this up?

[fischerscode]

While having this document in mind, I propose the following release procedure.

Procedure:

1. PRs are created and merged without changes to the versions. (They use Conventional Commits as titles and are squash merged)
2. Once it is time to release, a authorized person triggers the workflow release-preparation.
3. The workflow runs melos version --no-git-tag-version (on master), tests, melos publish (it's a dry run), commits the changes (chore(release): prepare for release (Default)) and then creates a PR to master called Release (or something similar). The PR gets a label called release. (Additional tests run on only those PRs.)
4. Once the PR is approved and merged, CI (workflow release) triggers automatically and runs melos publish --no-dry-run --git-tag-version. Afterwards, the generated git tags are pushed.

All names are of course only suggestions and can be changed.

Considerations:

• Do we want to support pre-releases of the packages (pub.dev supports it, Melos too.)
• Do we want to publish after each commit or only when a human triggers it.

Advantages:

• Due to the PR, nothing is published on accident.
• Little work when publishing.
• In case something unexpected happens, an admin could easily publish manually using the melos commands.
• All the boring stuff (versions, changelog) is managed by melos.
• The pubspec.yaml's could use real version dependencies, since melos bootstrap links them locally.

Disadvantages:

• Releases have to be triggered manually.
• Everybody contributing has to install melos (dart pub global activate melos) to bootstrap the repo.
• Due to melos changing the pubspec.lock for locally linking the packages of the repo, ci: add lock file to repository #750 might not be possible.

[mtrezza]

Do we want to support pre-releases of the packages

Yes

Do we want to publish after each commit or only when a human triggers it.

As we currently do it org wide; release an alpha version after each commit, once every month release a beta version with all changes in alpha, and a stable version with all changes in beta. We can probably skip the beta release here. The pre-releases should be separated by branch and follow our branch model: alpha (working branch), beta (optional, not in this repo), release (stable releases).

You can take a look at the Parse Server repo how this is managed; also how we manage the changelog there, as we have a separate alpha / beta / release changelog.

You can also take a look at our current commit message syntax, also for reverting PRs. We are using the same syntax org wide, so melos must be compatible with it.

[mbfakourii]

@mtrezza

At what stage are these issues? Is the problem solved?

[mtrezza]

There is no problem, auto-release is not just added yet because it needs more investigation. We have auto-release processes in other repos, but not for a mono-repository as this one. We'll likely use the same framework that we use in the other repos, which supports mono-repos. Once we have the current manual release process fully working, we can take a look at this one.

[mtrezza]

After some tests with mono-repo release using semantic-release, it seems quite a shaky endeavor. semantic release don't support mono-repos OOTB and doesn't plan on doing so anytime soon for strategic reasons. Extensions (multi-semantic-release, semantic-release-monorepo as at best hacks and show fundamental issues (ESM compatibility, semantic release-hacks, etc.). It may work well for mono-repos with one of these extensions, until it won't for some reason. Given that we are looking for a mono-repo solution not only for the Parse Flutter SDK, but use it as the default across repos, relying on such a fragile release processes is a risk we want to avoid.

Let's take this opportunity to try out auto as an alternative solution. It seems to be well maintained, looking at the commit history, seems mature (10 years), has a large user and collaborator base, and brings mono-repo support OOTB. I'll do some testing. If viable, we may use it instead of semantic-release here to gain more experience with it before extending its use across other repos.

[Moumouls]

sounds promising @mtrezza i can suggest to take a look to Turbo repo/Changeset ( combined with npm or yarn)

https://turbo.build/repo/docs/handbook/publishing-packages/versioning-and-publishing

it seems that auto is recommended, it's a good sign

[Nidal-Bakir]

@mtrezza
I'm really sorry but at this point, I'm lost there are 72 comments. Can you give me a link or something to read in order to understand what is our end goal? I really want to help and work on this but until now I did not get the full picture of what is the issue or what is the current limitation. Because we already have a CI/CD automated pipeline. What are the current issues with them?

Thanks.

[mtrezza]

The current CD is not automated. I have to manually make the release and the PR author has to manually write the changelog. The points in the comment are still correct; for more details see the comments thereafter. Also related: parse-community/parse-server#7293.

[mbfakourii]

@mtrezza
I think it will be useful for this issuse
https://dart.dev/tools/pub/automated-publishing

[mtrezza]

We are already publishing to pub.dev automated. Publishing is just 1 step of many others that we currently do manually. I need to create the release manually on GitHub and we need to write the changelog manually. This all should be automated.