Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not use File objects in dictionary keys #1545

Merged
merged 1 commit into from
Apr 22, 2021
Merged

Conversation

aherrmann
Copy link
Member

The function cc_library_key generates the dictionary keys by which LibLibraryToLink are mapped to their corresponding HaskellCcLibraryInfo. Before this used a struct over None or File objects. This changes uses None or string instead, where string is the .path attribute of the File object.

Using a File object in a dictionary key fails if the File object used at lookup time is semantically the same, but has a different identity due to implementation details in Bazel.

Closes #1536

@meteorcloudy
Copy link
Contributor

Tested with Bazel@HEAD: https://buildkite.com/bazel/rules-haskell-haskell/builds/725

@aherrmann aherrmann marked this pull request as ready for review April 21, 2021 14:03
@@ -241,13 +241,17 @@ def create_link_config(hs, posix, cc_libraries_info, libraries_to_link, binary,

return (cache_file, static_libs, dynamic_libs)

def _path_or_none(f):
if f != None:
return f.path
Copy link
Member

@facundominguez facundominguez Apr 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this could go as a local definition inside of cc_library_key.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind, this is not allowed in Bazel 4.0.0:

ERROR: /home/runner/.cache/bazel/_bazel_runner/3e6d0af2c61b2bb9852d7167e763fc51/external/rules_haskell/haskell/private/cc_libraries.bzl:247:5: nested functions are not allowed. Move the function to the top level.
ERROR: error loading package '': in /home/runner/.cache/bazel/_bazel_runner/3e6d0af2c61b2bb9852d7167e763fc51/external/rules_haskell/haskell/cabal.bzl: Extension 'haskell/private/cc_libraries.bzl' has errors

It seems to be allowed in Bazel HEAD though, the change worked when tested with USE_BAZEL_VERSION="last_green".

@aherrmann aherrmann added the merge-queue merge on green CI label Apr 22, 2021
@aherrmann
Copy link
Member Author

Encountered an issue with BES on Windows:

FATAL: bazel crashed due to an internal error. Printing stack trace:
java.util.concurrent.RejectedExecutionException: Task com.google.common.util.concurrent.TrustedListenableFutureTask@7eb86512[status=PENDING, info=[task=[running=[NOT STARTED YET], com.google.devtools.build.lib.remote.ByteStreamBuildEventArtifactUploader$$Lambda$737/0x00000001007d2c40@6e4e775f]]] rejected from java.util.concurrent.ThreadPoolExecutor@31968453[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 253]
	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(Unknown Source)
	at com.google.common.util.concurrent.MoreExecutors$ListeningDecorator.execute(MoreExecutors.java:586)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
	at com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:66)
	at com.google.devtools.build.lib.remote.ByteStreamBuildEventArtifactUploader.upload(ByteStreamBuildEventArtifactUploader.java:222)
	at com.google.devtools.build.lib.buildeventstream.BuildEventArtifactUploader.uploadReferencedLocalFiles(BuildEventArtifactUploader.java:104)
	at com.google.devtools.build.lib.buildeventservice.BuildEventServiceUploader.enqueueEvent(BuildEventServiceUploader.java:196)
	at com.google.devtools.build.lib.buildeventservice.BuildEventServiceTransport.sendBuildEvent(BuildEventServiceTransport.java:95)
	at com.google.devtools.build.lib.runtime.BuildEventStreamer.post(BuildEventStreamer.java:268)
	at com.google.devtools.build.lib.runtime.BuildEventStreamer.buildEvent(BuildEventStreamer.java:472)
	at com.google.devtools.build.lib.runtime.BuildEventStreamer.buildEvent(BuildEventStreamer.java:481)
	at com.google.devtools.build.lib.runtime.BuildEventStreamer.clearPendingEvents(BuildEventStreamer.java:307)
	at com.google.devtools.build.lib.runtime.BuildEventStreamer.clearEventsAndPostFinalProgress(BuildEventStreamer.java:634)
	at com.google.devtools.build.lib.runtime.BuildEventStreamer.close(BuildEventStreamer.java:354)
	at com.google.devtools.build.lib.runtime.BuildEventStreamer.closeOnAbort(BuildEventStreamer.java:336)
	at com.google.devtools.build.lib.buildeventservice.BuildEventServiceModule.forceShutdownBuildEventStreamer(BuildEventServiceModule.java:409)
	at com.google.devtools.build.lib.buildeventservice.BuildEventServiceModule.afterCommand(BuildEventServiceModule.java:578)
	at com.google.devtools.build.lib.runtime.BlazeRuntime.afterCommand(BlazeRuntime.java:626)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:603)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:231)
	at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:543)
	at com.google.devtools.build.lib.server.GrpcServerImpl.lambda$run$1(GrpcServerImpl.java:606)
	at io.grpc.Context$1.run(Context.java:579)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Error: Process completed with exit code 37.

Possibly related to bazelbuild/bazel#12575.

Retrying...

The function `cc_library_key` generates the dictionary keys by which
`LibLibraryToLink` are mapped to their corresponding
`HaskellCcLibraryInfo`. Before this used a `struct` over `None` or
`File` objects. This changes uses `None` or `string` instead, where
`string` is the `.path` attribute of the `File` object.

Using a `File` object in a dictionary key fails if the `File` object
used at lookup time is semantically the same, but has a different
identity due to implementation details in Bazel.

Closes #1536
@mergify mergify bot merged commit a210199 into master Apr 22, 2021
@mergify mergify bot deleted the trim-test-config branch April 22, 2021 12:29
@mergify mergify bot removed the merge-queue merge on green CI label Apr 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

rules_haskell is incompatible with Bazel's new "trim_test_configuration" default setting
3 participants