Add tool unreachable

[straight-shoota]

This adds a compiler tool to identify methods that are never called.

It's based off earlier work by @bcardiff presented in #3801 (comment) (although a majority of the code has changed since that poc).

I have tested this on the compiler source code. There are no false positives, so I'm quite certain that we're good there.
Asserting recall is much more difficult. I cannot be sure about whether some unused definitions might be missed. But it seems to be pretty complete.

Resolves #3801

[straight-shoota]

I have used some simple tools for working with the output of this command. They are not part of the tool itself to keep it simple and separate concerns.
I figure they could probably be placed in the documentation as well.

$ sort -t $'\t' -k3 -gr < unreachable.tsv | head # print the methods with most loc
/home/johannes/src/crystal-lang/crystal/src/compiler/crystal/interpreter/value.cr:15:3  Crystal::Repl::Value#value      54 lines
/home/johannes/src/crystal-lang/crystal/src/compiler/crystal/codegen/codegen.cr:29:5    Crystal::Program#evaluate       42 lines
/home/johannes/src/crystal-lang/crystal/src/compiler/crystal/syntax/to_s.cr:781:5       Crystal::ToSVisitor#visit       35 lines
/home/johannes/src/crystal-lang/crystal/src/compiler/crystal/tools/playground/agent.cr:11:3     Crystal::Playground::Agent#i    33 lines
/home/johannes/src/crystal-lang/crystal/src/compiler/crystal/semantic/cleanup_transformer.cr:1112:5     Crystal::CleanupTransformer#simple_constant?    31 lines
/home/johannes/src/crystal-lang/crystal/src/compiler/crystal/semantic/type_declaration_processor.cr:332:11      Crystal::TypeDeclarationProcessor#check_non_nilable_for_generic_module  27 lines
/home/johannes/src/crystal-lang/crystal/src/compiler/crystal/tools/playground/agent.cr:61:11    Crystal::Playground::Agent#send 12 lines
/home/johannes/src/crystal-lang/crystal/src/compiler/crystal/semantic/main_visitor.cr:606:5     Crystal::MainVisitor#first_time_accessing_meta_type_var?        12 lines
/home/johannes/src/crystal-lang/crystal/src/compiler/crystal/semantic/ast.cr:502:5      Crystal::MetaVar#inspect        12 lines
/home/johannes/src/crystal-lang/crystal/src/compiler/crystal/semantic/type_declaration_processor.cr:360:11      Crystal::TypeDeclarationProcessor#has_syntax_nil?       11 lines

# delete all reported methods
tac unreachable.tsv | while read -r line; do
  fileinfo=$(echo "$line" | cut -f1)

  path=$(echo "$fileinfo" | cut -d: -f1)
  line=$(echo "$fileinfo" | cut -d: -f2)
  column=$(echo "$fileinfo" | cut -d: -f3)

  sed -i -E "${line},/^$(printf %*s $(expr $column - 1))end/d" "$path"
done

In case you have multiple entry points, you can run the tool multiple times and combine the results to figure out methods that are called in neither:

comm -12 --nocheck-order unreachable1.tsv unreachable2.tsv

[bcardiff]

I believe so. Otherwise we might never find that the method is no longer needed. But I would understand if not reported.

If the entry point is specs those might mark it as used, but without specs then it could be pointed out.

If only the abstract is there when building the source, should we get an abstract check error?

[straight-shoota]

If only the abstract is there when building the source, should we get an abstract check error?

Yes, that's how it works currently. If an abstract method is never called, the tool reports all implementations as unreachable. But not the abstract def. Then after removing all the reported implementations, you'll get an abstract check error.

I think that's good enough for an MVP implementation.

[bcardiff]

Another use case we might not be checking is macro expanded methods. It's more complex. If a property foo : String is declared, should we signal if no method expanded by it is called or if any method is not used?

[bcardiff]

And detecting if a macro definition is never used is also something we could consider adding in the future

[straight-shoota]

@bcardiff Currently the tool does not consider any defs that have been expanded from a macro (because the location is a virtual file). So yeah we're missing out on property et al.
I agree that it's more complex to figure out what to do in the case of a single macro expanding to multiple defs. Maybe a heuristic criterion could be whether the macro itself is defined in the current codebase or in a library. But I figure this would require some more in depth experimentation and evaluation.

Testing other features than defs (macros, constants, types) for being unused would be further feature enhancements. They also may need some more careful planning and are out of scope for this PR.

[bcardiff]

I agree. It sounds to me like we have a new tool 🚀

[beta-ziliani]

The specs are failing in Windows

[straight-shoota]

The failure on Windows was due to #13831. The latest commit should fix that.

Stale

[beta-ziliani]

LGTM. I leave a minor comment and a question.