Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specs are broken on Crystal master #88

Closed
straight-shoota opened this issue May 30, 2024 · 2 comments · Fixed by #89
Closed

Specs are broken on Crystal master #88

straight-shoota opened this issue May 30, 2024 · 2 comments · Fixed by #89

Comments

@straight-shoota
Copy link

straight-shoota commented May 30, 2024

Specs for habitat are failing on current Crystal master / nightly build. This was discovered in the ecosystem tests (https://github.com/crystal-lang/test-ecosystem/actions/runs/9306105007/job/25614556241).

The responsible code is this:

habitat/src/habitat.cr

Lines 234 to 238 in 5fd8d7b

{% if decl.type.is_a?(Union) && decl.type.types.map(&.id).includes?(Nil.id) %}
{% nilable = true %}
{% else %}
{% nilable = false %}
{% end %}

It compares type membership based id which is based on the name representation instead of the type itself. Names can be ambiguous, i.e. there are multiple names for the same type. Here the problem is that the id Nil is different from ::Nil.

The failure seems to be caused by a change in the compiler which fixes the id of global paths to include the leading :: which had previsouly been missing (crystal-lang/crystal#14490).

With decl as foo : String?, the expression decl.type.types.map(&.id) evaluates to [String, Nil] on Crystal 1.12.2. In Crystal master it's [String, ::Nil] (which is correct because String? is equivalent to String | ::Nil).

Simply changing the check to includes?(::Nil.id) seems logical, but that won't work because ::Nil implicitly resolves to a TypeNode, not a Path. The path returned from TypeNode#id is always global, but it does not have a :: prefix.

My initial understanding is that this code shouldn't have worked in the first place because it relied on the erroneous id generation that was fixed in the mentioned issue.

We have to analyze the compiler issue a bit more and it might be possible that another compiler change might fix this. So for now I'm just giving a heads up.
And maybe there's a chance to refactor this macro code so it won't break in the future if it was indeed only working because it relied on erroneous behaviour.

Spec output
    ........E.EFE..
   
   Failures:
   
     1) Habitat can check for missing settings
        Failure/Error: expect_raises(Habitat::MissingSettingError, %(settings.constant_setting = RandomClass)) do
   
          Expected Habitat::MissingSettingError with "settings.constant_setting = RandomClass", got #<Habitat::MissingSettingError: The 'this_can_be_nil' setting for FakeServer was nil, but the setting is required.
   
          Try this...
   
            FakeServer.configure do |settings|
              settings.this_can_be_nil = some_value
            end
          > with backtrace:
            # src/habitat.cr:12:3 in 'raise_if_missing_settings!'
            # spec/habitat_spec.cr:287:7 in '->'
            # /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/example.cr:50:13 in 'internal_run'
            # /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/example.cr:38:16 in 'run'
            # /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:20:23 in 'internal_run'
            # /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:344:14 in 'run'
            # /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:20:23 in 'internal_run'
            # /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:155:7 in 'run'
            # /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/dsl.cr:237:7 in 'execute_examples'
            # /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/dsl.cr:220:13 in '->'
            # /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/at_exit_handlers.cr:14:19 in 'run'
            # /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/main.cr:52:14 in 'exit'
            # /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/main.cr:47:5 in 'main'
            # /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/main.cr:129:3 in 'main'
            # /lib/x86_64-linux-gnu/libc.so.6 in '??'
            # /lib/x86_64-linux-gnu/libc.so.6 in '__libc_start_main'
            # /home/runner/.cache/crystal/crystal-run-spec.tmp in '_start'
            # ???
   
        # spec/habitat_spec.cr:286
   
     2) Habitat works with nilable types
   
          Nil assertion failed (NilAssertionError)
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/nil.cr:113:7 in 'not_nil!'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/nil.cr:109:3 in 'not_nil!'
            from spec/habitat_spec.cr:7:3 in 'this_can_be_nil'
            from spec/habitat_spec.cr:245:5 in '->'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/example.cr:50:13 in 'internal_run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/example.cr:38:16 in 'run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:20:23 in 'internal_run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:344:14 in 'run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:20:23 in 'internal_run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:155:7 in 'run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/dsl.cr:237:7 in 'execute_examples'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/dsl.cr:220:13 in '->'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/at_exit_handlers.cr:14:19 in 'run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/main.cr:52:14 in 'exit'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/main.cr:47:5 in 'main'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/main.cr:129:3 in 'main'
            from /lib/x86_64-linux-gnu/libc.so.6 in '??'
            from /lib/x86_64-linux-gnu/libc.so.6 in '__libc_start_main'
            from /home/runner/.cache/crystal/crystal-run-spec.tmp in '_start'
            from ???
   
   
     3) Habitat can set and reset config using a block
   
          Nil assertion failed (NilAssertionError)
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/nil.cr:113:7 in 'not_nil!'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/nil.cr:109:3 in 'not_nil!'
            from spec/habitat_spec.cr:7:3 in 'this_can_be_nil'
            from spec/habitat_spec.cr:262:5 in '->'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/example.cr:50:13 in 'internal_run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/example.cr:38:16 in 'run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:20:23 in 'internal_run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:344:14 in 'run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:20:23 in 'internal_run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:155:7 in 'run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/dsl.cr:237:7 in 'execute_examples'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/dsl.cr:220:13 in '->'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/at_exit_handlers.cr:14:19 in 'run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/main.cr:52:14 in 'exit'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/main.cr:47:5 in 'main'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/main.cr:129:3 in 'main'
            from /lib/x86_64-linux-gnu/libc.so.6 in '??'
            from /lib/x86_64-linux-gnu/libc.so.6 in '__libc_start_main'
            from /home/runner/.cache/crystal/crystal-run-spec.tmp in '_start'
            from ???
   
   
     4) Habitat can be converted to a Hash
   
          Nil assertion failed (NilAssertionError)
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/nil.cr:113:7 in 'not_nil!'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/nil.cr:109:3 in 'not_nil!'
            from spec/habitat_spec.cr:7:3 in 'this_can_be_nil'
            from spec/habitat_spec.cr:7:3 in 'to_h'
            from spec/habitat_spec.cr:308:5 in '->'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/example.cr:50:13 in 'internal_run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/example.cr:38:16 in 'run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:20:23 in 'internal_run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:344:14 in 'run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:20:23 in 'internal_run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/context.cr:155:7 in 'run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/dsl.cr:237:7 in 'execute_examples'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/spec/dsl.cr:220:13 in '->'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/at_exit_handlers.cr:14:19 in 'run'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/main.cr:52:14 in 'exit'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/main.cr:47:5 in 'main'
            from /home/runner/work/_temp/crystal-nightly-nightly-undefined/share/crystal/src/crystal/main.cr:129:3 in 'main'
            from /lib/x86_64-linux-gnu/libc.so.6 in '??'
            from /lib/x86_64-linux-gnu/libc.so.6 in '__libc_start_main'
            from /home/runner/.cache/crystal/crystal-run-spec.tmp in '_start'
            from ???
   
   
   Finished in 103.82 milliseconds
   15 examples, 1 failures, 3 errors, 0 pending
   
   Failed examples:
   
   crystal spec spec/habitat_spec.cr:271 # Habitat can check for missing settings
   crystal spec spec/habitat_spec.cr:243 # Habitat works with nilable types
   crystal spec spec/habitat_spec.cr:259 # Habitat can set and reset config using a block
   crystal spec spec/habitat_spec.cr:296 # Habitat can be converted to a Hash

You can reproduce this with a nightly build of Crystal (for example docker run --rm -it -v $(pwd):/app -w /app crystallang/crystal:nightly-alpine crystal spec).

@straight-shoota
Copy link
Author

straight-shoota commented May 30, 2024

This patch seems to fix most of the error. There's still one spec failing, because a different error is raised then the exected one. But it looks like the actual error might be valid as well?

--- i/src/habitat.cr
+++ w/src/habitat.cr
@@ -231,11 +231,12 @@ class Habitat
         # NOTE: We can't use the macro level `type.resolve.nilable?` here because
         # there's a few declaration types that don't respond to it which would make the logic
         # more complex. Metaclass, and Proc types are the main, but there may be more.
-        {% if decl.type.is_a?(Union) && decl.type.types.map(&.id).includes?(Nil.id) %}
+        {% if Nil <= decl.type.resolve %}
Spec output
Failures:

  1) Habitat can check for missing settings
     Failure/Error: expect_raises(Habitat::MissingSettingError, %(settings.constant_setting = RandomClass)) do

       Expected Habitat::MissingSettingError with "settings.constant_setting = RandomClass", got #<Habitat::MissingSettingError: The 'this_can_be_nil' setting for FakeServer was nil, but the setting is required.

       Try this...

         FakeServer.configure do |settings|
           settings.this_can_be_nil = some_value
         end
       > with backtrace:
         # src/habitat.cr:12:3 in 'raise_if_missing_settings!'
         # spec/habitat_spec.cr:287:7 in '->'
         # /usr/share/crystal/src/spec/example.cr:50:13 in 'internal_run'
         # /usr/share/crystal/src/spec/example.cr:38:16 in 'run'
         # /usr/share/crystal/src/spec/context.cr:20:23 in 'internal_run'
         # /usr/share/crystal/src/spec/context.cr:344:14 in 'run'
         # /usr/share/crystal/src/spec/context.cr:20:23 in 'internal_run'
         # /usr/share/crystal/src/spec/context.cr:155:7 in 'run'
         # /usr/share/crystal/src/spec/dsl.cr:237:7 in 'execute_examples'
         # /usr/share/crystal/src/spec/dsl.cr:220:13 in '->'
         # /usr/share/crystal/src/crystal/at_exit_handlers.cr:14:19 in 'run'
         # /usr/share/crystal/src/crystal/main.cr:52:14 in 'exit'
         # /usr/share/crystal/src/crystal/main.cr:47:5 in 'main'
         # /usr/share/crystal/src/crystal/main.cr:129:3 in 'main'
         # /lib/ld-musl-x86_64.so.1 in '??'

     # spec/habitat_spec.cr:286

Finished in 75.27 milliseconds
15 examples, 1 failures, 0 errors, 0 pending

Failed examples:

crystal spec spec/habitat_spec.cr:271 # Habitat can check for missing settings

EDIT: Ah no, apparently this won't work if the path needs to be resolved in the local scope (e.g. Dot in ASettingForEverything).

@jwoertink
Copy link
Member

Oh wow. Thanks for reporting this! I wasn't aware of the ecosystem test. That's pretty amazing. I don't have time to look in to this at the moment, but I'll dive in next week. I wonder if I can just say includes Nil or includes ::Nil 🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants