[Long term goal] Replace rustc-ap-syntax crates with rust-analyzer

[topecongiro]

rustfmt relies on rustc-ap-syntax and its friends, which are a direct port of the rustc parser, to parse files and construct AST. This strategy has both pros and cons:

• Pros
  • We do not need to write our parser, which would be a significant work
  • We always get the latest and correct syntax
• Cons
  • The rustc parser panics occasionally, which complicates error handlings in rustfmt
  • Building rustc-ap-* crates consume a lot of time
  • As libsyntax and its friends are not developed with tool-development in mind, we frequently encounter breaking changes
  • The rustc repository forbids tools to depend on different versions of rustc-ap-* crates, which significantly complicates updating and releasing rustfmt

All things considered, I have been looking for alternatives that would replace rustc-ap-*s.

I have been experimenting with rust-analyzer, and so far, it seems reasonable - it can parse a file, offers the span-ish feature, and panic-free. It does not recursively parse modules though we won't need to do so once #3930 gets resolved. We need more investigation, but overall, I believe rustfmt can replace its frontend with rust-analyzer's parser without any breakages.

Some concerns I am aware of:

• It may not support the syntax that is only available in the latest nightly, though presumably, we could send a patch if necessary
• Its error reporting is not as detailed as that of rustc
• It's still an experimental project and may get abandoned for whatever reasons

[calebcartwright]

Building rustc-ap-* crates consume a lot of time

In your experimentation have you found the build process to be noticeably quicker with rust-analyzer? That'd be a solid win for the rustfmt-dev experience!

Also, I'd be curious how testable the code would be at the unit-test level with rust-analyzer. There's been times when I thought I'd be able to write some unit tests only to hit a wall with libsyntax due to visibility issues, missing constructors, etc. I can understand why libsyntax is that way because, as you mentioned, it's not targetting tool development, but IMO it would be a pro/win if rust-analyzer allowed for more unit-level testing

[topecongiro]

In your experimentation have you found the build process to be noticeably quicker with rust-analyzer? That'd be a solid win for the rustfmt-dev experience!

I have not fully replaced rustc-ap-* crates with rust-analyzer, so I cannot tell for sure, but I am pretty sure the build time will improve.

Concerning testability, yes, I think that switching to rust-analyzer will make it easier for us to write unit tests. Personally, I am not that concerned about this, as the current test infrastructure of rustfmt is already acceptable.

[topecongiro]

It looks like folks are trying to unify the codebase of the rustc's frontend and rust-analyzer: rust-lang/rfcs#2912. They haven't reached a conclusion on to what extent the code should be shared among those two, though we could dream for a stable-parsing library for the rust programming language.