-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Analyzer is very slow for large files #56247
Comments
Hours makes this almost a P1. That's pretty wild. I wonder if @scheglov can do an initial diagnosis / triage. |
My guess is that it used up all the memory on my laptop and started thrashing like crazy. |
I cannot reproduce it. name: issue56247
environment:
sdk: ^3.5.0
dependencies:
ffi: any
objective_c: any Which help a bit, but for
It takes |
Yeah, this generated code is using some unreleased features from package:objective_c. I plan to release it later this week, but if you want to try to repro now, add a git dep: dependencies:
objective_c:
git:
url: git@github.com:dart-lang/native.git
path: pkgs/objective_c There are 9 analysis errors when I run it locally (bugs in ffigen I haven't fixed yet). |
Indeed, only 9 errors, but the performance is still the same, |
@liamappelbe can you still reproduce this? If you can we should figure out if you can capture some profiles for @scheglov. |
Yep, I can still repro. @scheglov just let me know the most useful thing I can capture for you. |
Here is the script that I used to measure timings, and to profile. import 'package:analyzer/dart/analysis/results.dart';
import 'package:analyzer/file_system/overlay_file_system.dart';
import 'package:analyzer/file_system/physical_file_system.dart';
import 'package:analyzer/src/dart/analysis/analysis_context_collection.dart';
void main() async {
var resourceProvider = OverlayResourceProvider(
PhysicalResourceProvider.INSTANCE,
);
var collection = AnalysisContextCollectionImpl(
resourceProvider: resourceProvider,
includedPaths: [
'/Users/scheglov/tmp/2024-07-17/issue56247/lib/large_library.dart',
],
);
var timer = Stopwatch()..start();
for (var analysisContext in collection.contexts) {
print(analysisContext.contextRoot.root.path);
var analysisSession = analysisContext.currentSession;
for (var path in analysisContext.contextRoot.analyzedFiles()) {
if (path.endsWith('.dart')) {
var libResult = await analysisSession.getResolvedLibrary(path);
if (libResult is ResolvedLibraryResult) {
for (var unitResult in libResult.units) {
print(' ${unitResult.path}');
var ep = '\n ';
print(' errors:$ep${unitResult.errors.join(ep)}');
}
}
}
}
}
print('[time: ${timer.elapsedMilliseconds} ms]');
await collection.dispose();
} |
Tried running that script, and I think it is just memory thrashing. If I run it after a fresh reboot it runs in a reasonable amount of time (~30sec). But if I open a bunch of chrome tabs and code editor tabs etc, the analyzer takes forever. If I look at the observer on one of the slow runs, memory use is still about the same as what you're seeing (about 2 to 2.5GB), though the observer page runs very slowly, and the memory graph refreshes much less frequently. There's probably not anything actionable here. When I run this integration test on github CI, I'll just have to make sure the bot has enough memory. |
When generating ObjC bindings using ffigen, it's quite common to have generated files with >10k, or even >100k lines. The analyzer does fine at these sizes, but I'm trying to build a large integration test that generates ~500k lines, and the analyzer starts to choke.
The files are here. (Couldn't use a gist as they're too big)
The text was updated successfully, but these errors were encountered: