Release 0.27 - June 2019 #7816

dslomov · 2019-03-22T14:39:44Z

Target RC date: June 3rd

Ragora · 2019-05-29T17:21:53Z

I would like 84c5dc9#diff-73cc3e84377e7c63ef4406039e060016 (or an equivalent thereof) to be in this release.

For my employer I started this fairly large migration to bazel seeing from documentation it did everything we needed, however, it seems pkg_deb does not have a templates parameter which we use quite heavily in our existing software meaning the resulting packages in the current state are kindof useless.

I'm at a point of having to make a bazel build from master just for this one feature.

Amazing tool otherwise, though! It's just this one thing is kindof a stickling point at the moment.

laurentlb · 2019-05-29T17:27:25Z

It will definitely be included. Any change submitted in May will be included. Changes submitted in June are unlikely to part of 0.27.

Thanks for the feedback!

Ragora · 2019-05-29T17:29:22Z

Cool, was checking because I got confused about the 0.26 release still not having it, guess I just unlucky about the way times line up then. In the meantime I'll use a custom build, but at least it's not long to wait.

laurentlb · 2019-05-29T17:39:37Z

Status of Bazel 0.27

I plan to keep this message up to date during the release process and provide the important information.

Target baseline: June 3rd
Expected release date: Monday, June 17th
List of release blockers

To report a release-blocking bug: file a bug, use the Release blocker label, and cc @laurentlb. Release blockers are regressions in Bazel relative to Bazel 0.25 or Bazel 0.26.

Task list:

Update GitHub issues for incompatible changes
Pick release baseline: 8c3b3fb
Create release candidate: https://releases.bazel.build/0.27.0/rc5/index.html
Check downstream projects: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/1034 (a few failures, likely due to the projects themselves)
Create draft release announcement
Send for review the release announcement PR: Release announcement for Bazel 0.27 bazel-blog#178
Push the release, notify package maintainers: https://github.com/bazelbuild/bazel/releases/tag/0.27.0
Update the documentation
Publish the announcement (blog & mailing-list)

brandjon · 2019-05-31T17:43:21Z

I'm about to flip --incompatible_use_python_toolchains. This was introduced in 0.25, flipped at head for 0.26, and immediately rolled back when it broke downstream. So it's now had two months of migration window.

All known downstream breakages are summarized in this comment. Most breakages are because targets had been declaring themselves (perhaps by default) to be Python 3, even though they were really only compatible with Python 2. Previously they sometimes got ran under Python 2 anyway due to #4815, but this flag fixes that bug on every platform besides Windows. Migration usually consists of adding python_version = "PY2" to Python targets, and --host_force_python=PY2 if any of them are used in the host configuration.

+cc Green Team sheriffs @vladmos and @laszlocsomor.

laurentlb · 2019-06-03T17:03:22Z

Creating 0.27.0rc1 with:

scripts/release/release.sh create 0.27.0 5935259

jin · 2019-06-03T18:22:53Z

0.26.0 / 0.27.0 is broken in IntelliJ when building java_plugin targets - bazelbuild/intellij#845

Please cherry-pick b0403a7 (if it's after baseline)

brandjon · 2019-06-03T20:37:58Z

Created blockers #8547 and #8549 to track my effort to add better diagnostics for users whose Python targets fail due to toolchains.

philwo · 2019-06-03T20:39:24Z

Can we ensure that 8c3b3fb gets into the release? It resolves a major issue with —keep_going that causes failing builds to hide the error and exit with code 0 (#7674). This in turn causes Bazel’s CI to Mark pipelines as green, even though the projects are broken.

laurentlb · 2019-06-04T18:34:33Z

Created 0.27rc2 with:

scripts/release/release.sh create --force_rc=2 0.27.0 8c3b3fb

(this includes both b0403a7 and 8c3b3fb)

You can download the release candidate here: https://releases.bazel.build/0.27.0/rc2/index.html

brandjon · 2019-06-04T22:20:18Z

Of the two release blockers I filed, one is fixed and the other has a pending CL. Both concern the ease of migrating to toolchains but do not affect CI.

The already submitted one is 123c68d and can be cherrypicked now. Will post when the other is submitted.

brandjon · 2019-06-05T02:18:55Z

And the other one is 052167e.

meisterT · 2019-06-05T07:42:35Z

Laurent, this might require a cherry pick: #8539

davido · 2019-06-06T05:53:28Z

Laurent, this might require a cherry pick: #8539

As pointed out in discussion in the above issue, bad5a2b introduced very serious regression (and is included in 0.27rc2):

The previous javac version had a non-standard modification to default to
the Java 8 language level, which wasn't carried forward to 11.

As the consequence, after the upgrade from 0.26 to 0.27rc2 wrong byte code is generated, when building with default_java_toolchain. Even though, Java 8 is used, byte code with major version 55 is produced that corresponds to Java 11.

  $ bazel version
  Build label: 0.27.0rc2

  $ $JAVA_HOME/bin/java -version
  openjdk version "1.8.0_212"

  $ bazel build java/com/google/gerrit/common:server 
  [...]
  Target //java/com/google/gerrit/common:server up-to-date:
  bazel-bin/java/com/google/gerrit/common/libserver.jar
INFO: Elapsed time: 46.555s, Critical Path: 30.31s
INFO: 36 processes: 27 linux-sandbox, 9 worker.
INFO: Build completed successfully, 38 total actions

  $ javap -verbose -cp bazel-bin/java/com/google/gerrit/common/libserver.jar com.google.gerrit.common.data.SubscribeSection | grep major 
  major version: 55

On Bazel 0.26 the last command would correctly print Java 8 byte code major version:

  $ javap -verbose -cp bazel-bin/java/com/google/gerrit/common/libserver.jar com.google.gerrit.common.data.SubscribeSection | grep major 
  major version: 52

As the consequence, after building Gerrit Code Review project with 0.27.rc2, the produced artifact cannot be run any more on Java 8 (all is fine on 0.26):

  $ $JAVA_HOME/bin/java -version
  openjdk version "1.8.0_212"

  $ bazel build release
  [...]

  $ $JAVA_HOME/bin/java -jar bazel-bin/release.war init -d ../test_site_xxx_yyy_zzz
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.UnsupportedClassVersionError: Main has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:495)

Can this be fixed, or at very least, can this disruptive change be documented in the release documentation?

philwo · 2019-06-06T13:56:47Z

Please note that we finished the migration to Ubuntu 16.04 for release builds and all other pipelines on Bazel CI now. This means that 0.27.0rc3 and following versions will be built on Ubuntu 16.04 and will no longer compatible with Ubuntu 14.04.

We will mention this in the release notes, too.

iirina · 2019-06-06T14:39:48Z

@laurentlb 6ef6d87 fixes #8539 and should be cherry picked for backwards compatibility.

laurentlb · 2019-06-06T17:04:08Z

rc3 created with:

scripts/release/release.sh create --force_rc=3 0.27.0 8c3b3fb 123c68d 052167e 6ef6d87

Available here: https://releases.bazel.build/0.27.0/rc3/index.html

brandjon · 2019-06-06T17:41:16Z

Filed #8576 for bugs I just noticed in 052167e. (Sorry.) Working on it now.

brandjon · 2019-06-07T00:09:31Z

Fixed #8576 with 50fa3ec, please cherrypick.

irengrig · 2019-06-07T08:04:50Z

Please cherry-pick 6efc5b7, a fix for #8564.

jin · 2019-06-07T19:20:52Z

Please cherry pick e2a626c for bazel-contrib/rules_jvm_external#165 and android/testing-samples#279

Thank you!

laurentlb · 2019-06-07T20:09:13Z

rc4 created with:

scripts/release/release.sh create --force_rc=4 0.27.0 8c3b3fb 123c68d 052167e 6ef6d87 50fa3ec e2a626c

#8564 (comment) was not included.

Download rc4: https://releases.bazel.build/0.27.0/rc4/index.html
Status of downstream projects: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/1027

brandjon · 2019-06-10T22:11:52Z

Update on toolchain downstream breakages here.

brandjon · 2019-06-11T17:43:21Z

#8536 is now release-blocking as I decide how to address the gerrit breakage. The error mode only occurs when

using --incompatible_strict_action_env in combination with
the default Python toolchain
on a mac, and
in a build that includes a genrule-within-a-genrule (or genrule underneath another host-configured target)

This combination should be pretty rare.

brandjon · 2019-06-17T15:30:57Z

@vbatts There is no hard dependency, it's just a default that is now correctly being followed. Buried in the log output is an error message about using --host_force_python and/or the non-strict Python toolchain. See #7899.

petemounce · 2019-06-17T21:07:52Z

Published to chocolatey.

vbatts · 2019-06-18T13:15:19Z

i'm still having issues getting this built for fedora and centos.
Centos python 2 or 3 seems to be a clunky now.
And for fedora, it is failing on:

ERROR: /builddir/build/BUILD/bazel-0.27.0/src/BUILD:311:2: Executing genrule //src:embedded_tools_nojdk failed (Exit 127): bash failed: error executing command 
  (cd /tmp/bazel_P4UK5kXs/out/execroot/io_bazel && \
  exec env - \
    PATH=/builddir/.local/bin:/builddir/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; bazel-out/host/bin/src/create_embedded_tools "bazel-out/k8-opt/bin/src/embedded_tools_nojdk.zip" bazel-out/k8-opt/bin/src/embedded_tools_nojdk.params')
Execution platform: @bazel_tools//platforms:host_platform

brandjon · 2019-06-18T14:00:43Z

@vbatts Can you open a new issue and we'll debug there? Did you pass --host_force_python=PY2? Please include the full output (in case there was anything emitted by the failing target aside from that error log line), invocation line, and platform.

vbatts · 2019-06-18T14:42:21Z

python detection issue: #8665

buchgr · 2019-06-19T15:51:07Z

If there will be a patch release, please cherry-pick 484ffae and d75b6cf

vbatts · 2019-06-20T13:26:14Z

fedora, centos7 and rhel8 rpms pushed.
https://copr.fedorainfracloud.org/coprs/vbatts/bazel/build/941771/

iirina · 2019-06-25T08:00:42Z

@laurentlb is there going to be a patch release? d458963 should be cherry picked (fixes #8670 which is a regression in bazel 0.27).

philwo · 2019-06-25T14:08:09Z

I did all required changes to fix #8652 now. If a patch release for 0.27.0 is created, it will automatically be built on CentOS 7, which will fix the linked issue and restore compatibility with Ubuntu 14.04, CentOS 7 and RHEL 7.

vbatts · 2019-06-25T14:16:02Z

Cool cool to see official rpms. We'll see how well they work for Fedora. I'm glad to offload that from my copr. As for transitioning the thousands of subscribers to my builds: * Are you going to make a yum repo for these centos rpms? * Is this a beta build for now? * Is there going to be an announcement? (With transition steps for users)

…

-------- Original Message --------

On Jun 25, 2019, 10:08, Philipp Wollermann wrote: I did all required changes to fix [#8652](#8652) now. If a patch release for 0.27.0 is created, it will automatically be built on CentOS 7, which will fix the linked issue and restore compatibility with Ubuntu 14.04, CentOS 7 and RHEL 7. — You are receiving this because you were mentioned. Reply to this email directly, [view it on GitHub](#7816?email_source=notifications&email_token=AAAQL2LWK6LNPUV2NW46ZJTP4IRHLA5CNFSM4HAO36LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYQLWEQ#issuecomment-505461522), or [mute the thread](https://github.com/notifications/unsubscribe-auth/AAAQL2NEPCJ3YQW2SWBFCP3P4IRHLANCNFSM4HAO36LA).

philwo · 2019-06-25T14:23:09Z

@vbatts For this change, I just did the work on Bazel to pass its test suite on CentOS 7 (required a few small patches due to e.g. an older Git version in CentOS 7) and create a Docker container for our CI that contains what we need to build and test Bazel.

Then I added CentOS 7 as a supported platform to our CI and changed our scripts to build the "default" Linux binaries on it. I verified that when using these binaries built on CentOS 7, our downstream pipeline still passes (even when you use these binaries on e.g. Ubuntu 18.04). I didn't expect any issues here, but wanted to make sure.

That means, all official releases of Bazel will now have their Linux binaries built on CentOS 7 (instead of Ubuntu 16.04). This restores compatibility with Ubuntu 14.04, CentOS 7, while also staying compatible with newer Linux versions. As they should otherwise be functionally identical and pass all of our downstream tests, I'd consider them production ready.

It would be nice to create official RPMs and setup a yum mirror, but that's a next step - as much as I personally like working with CentOS and Fedora, I really don't know anything about how to build good packages for it. :) I'm happy to look at your COPR packages and give it a try though and get your feedback to make sure that it looks good.

vbatts · 2019-06-25T15:44:00Z

@philwo ah okay. so the *.zip of binaries is good to go for those folks. Then I will keep rolling with the copr builds for now.

If you were to make rpms and a yum(dnf) repo, copr is about as handy as anything. It does purge older builds, but if you're tagging releases, then folks can rebuild something about as equivalent. There is some automation to automatically trigger builds, but I haven't taken the time to iron that out. For now I mainly bump the version in the bazel.spec file and run make copr.

Once bazel gets aarch64 build support, I can flip the switch to enable build chroots for all fedora's and centos/rhel (7 and 8).

jayconrod · 2019-06-26T21:34:24Z

@laurentlb Gentle ping on @iirina's comment above: will there be a patch release?

rules_go coverage tests are failing in CI with 0.27.0, and d458963 should fix it. We don't have a workaround for this issue at the moment.

laurentlb · 2019-06-27T14:32:41Z

Bazel 0.27.1rc1 is available here: https://releases.bazel.build/0.27.1/rc1/index.html

It was created with:
scripts/release/release.sh create 0.27.1 c82eb48 d458963

So it's a cherry-pick of Irina's commit.
Please give it a try.

philwo · 2019-06-27T15:05:21Z

Dear users, please also verify that Bazel 0.27.1rc1 restores compatibility with Ubuntu 14.04 and CentOS 7.

jayconrod · 2019-06-27T17:28:17Z

I've verified 0.27.1rc1 fixes #8670 (bazel-contrib/rules_go#2100), both locally and on CI (log). Thanks!

laurentlb · 2019-07-02T18:11:00Z

0.27.1 was released.
https://github.com/bazelbuild/bazel/releases/tag/0.27.1

laszlocsomor · 2019-07-03T09:10:35Z

Regression: #8723
Fix is pending review: #8784

I think we'll need a 0.27.2 patch release.

laszlocsomor · 2019-07-04T08:31:21Z

Please create patch release 0.27.2 with 6d0b14b to fix regression #8723.

* Revert "Work around for bazel 0.27.0 issue" This reverts commit 41bf7ea. Bazel 0.27.1 is out: bazelbuild/bazel#7816 (comment) * Trigger CI Signed-off-by: Taiki Ono <taiki@tetrate.io>

laurentlb · 2019-07-08T11:50:24Z

Created 0.27.2rc1 using
scripts/release/release.sh create 0.27.2 c82eb48 d458963 6d0b14b

The binary is available here: https://releases.bazel.build/0.27.2/rc1/index.html

laurentlb · 2019-07-10T16:30:37Z

Released: https://github.com/bazelbuild/bazel/releases/tag/0.27.2

dslomov added the release label Mar 22, 2019

dslomov changed the title ~~Release 0.27 - June~~ Release 0.27 - June 2019 Mar 22, 2019

bazelbuild deleted a comment Apr 19, 2019

laurentlb self-assigned this Apr 30, 2019

aehlig pinned this issue May 28, 2019

laurentlb mentioned this issue Jun 11, 2019

--incompatible_strict_action_env breaks autodetecting Python toolchain on Mac #8536

Closed

jin mentioned this issue Jun 17, 2019

bazel 0.27 appears to no longer run on Trusty/RHEL/CentOS due to glibc compat #8652

Closed

Globegitter mentioned this issue Jun 26, 2019

Fix depset issues with/upgrade to Bazel 0.27.1 grpc/grpc-web#578

Merged

laurentlb closed this as completed Jul 2, 2019

taiki45 mentioned this issue Jul 3, 2019

Use Bazel 0.27.1 tetratelabs-attic/getenvoy-package#9

Merged

laszlocsomor mentioned this issue Jul 4, 2019

Release 0.28 - July 2019 (stable) #8571

Closed

laurentlb reopened this Jul 8, 2019

laurentlb unpinned this issue Jul 19, 2019

laurentlb closed this as completed Jul 24, 2019

Release 0.27 - June 2019 #7816

Release 0.27 - June 2019 #7816

Comments

dslomov commented Mar 22, 2019

Ragora commented May 29, 2019

laurentlb commented May 29, 2019

Ragora commented May 29, 2019

laurentlb commented May 29, 2019 • edited Loading

Status of Bazel 0.27

brandjon commented May 31, 2019

laurentlb commented Jun 3, 2019

jin commented Jun 3, 2019 • edited Loading

brandjon commented Jun 3, 2019

philwo commented Jun 3, 2019 • edited Loading

laurentlb commented Jun 4, 2019 • edited Loading

brandjon commented Jun 4, 2019

brandjon commented Jun 5, 2019

meisterT commented Jun 5, 2019

davido commented Jun 6, 2019 • edited Loading

philwo commented Jun 6, 2019

iirina commented Jun 6, 2019

laurentlb commented Jun 6, 2019

brandjon commented Jun 6, 2019

brandjon commented Jun 7, 2019

irengrig commented Jun 7, 2019

jin commented Jun 7, 2019

laurentlb commented Jun 7, 2019

brandjon commented Jun 10, 2019

brandjon commented Jun 11, 2019

brandjon commented Jun 17, 2019

petemounce commented Jun 17, 2019

vbatts commented Jun 18, 2019 • edited Loading

brandjon commented Jun 18, 2019

vbatts commented Jun 18, 2019 • edited Loading

buchgr commented Jun 19, 2019

vbatts commented Jun 20, 2019

iirina commented Jun 25, 2019

philwo commented Jun 25, 2019

vbatts commented Jun 25, 2019 via email

philwo commented Jun 25, 2019

vbatts commented Jun 25, 2019

jayconrod commented Jun 26, 2019

laurentlb commented Jun 27, 2019

philwo commented Jun 27, 2019

jayconrod commented Jun 27, 2019

laurentlb commented Jul 2, 2019

laszlocsomor commented Jul 3, 2019

laszlocsomor commented Jul 4, 2019

laurentlb commented Jul 8, 2019

laurentlb commented Jul 10, 2019

laurentlb commented May 29, 2019 •

edited

Loading

jin commented Jun 3, 2019 •

edited

Loading

philwo commented Jun 3, 2019 •

edited

Loading

laurentlb commented Jun 4, 2019 •

edited

Loading

davido commented Jun 6, 2019 •

edited

Loading

vbatts commented Jun 18, 2019 •

edited

Loading

vbatts commented Jun 18, 2019 •

edited

Loading