compute_stats: raise exception if tree.count_recursive > MAX_TREE_SIZE

[adelcast]

On deep repos, liguist fails silently, which is confusing. Raise an
exception instead.

Signed-off-by: Alejandro del Castillo alejandro.delcastillo@ni.com

Checklist:

• I am associating a language with a new file extension.

  • The new extension is used in hundreds of repositories on github.com
    • Search results for each extension:
      • https://github.com/search?utf8=%E2%9C%93&type=Code&ref=searchresults&q=extension%3AFOOBAR+KEYWORDS+NOT+nothack
  • I have included a real-world usage sample for all extensions added in this PR:
    • Sample source(s):
      • [URL to each sample source, if applicable]
    • Sample license(s):
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

• I am adding a new language.

  • The extension of the new language is used in hundreds of repositories on github.com.
    • Search results for each extension:
      • https://github.com/search?utf8=%E2%9C%93&type=Code&ref=searchresults&q=extension%3AFOOBAR+KEYWORDS+NOT+nothack
  • I have included a real-world usage sample for all extensions added in this PR:
    • Sample source(s):
      • [URL to each sample source, if applicable]
    • Sample license(s):
  • I have included a syntax highlighting grammar.
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

• I am fixing a misclassified language

  • I have included a new sample for the misclassified language:
    • Sample source(s):
      • [URL to each sample source, if applicable]
    • Sample license(s):
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

• I am changing the source of a syntax highlighting grammar

  • Old: https://github-lightshow.herokuapp.com/
  • New: https://github-lightshow.herokuapp.com/

• I am adding new or changing current functionality

  • I have added or updated the tests for the new or changed functionality.

[pchaigno]

LGTM, but this will almost certainly require changes on GitHub's side.

/cc @lildude

[lildude]

LGTM, but this will almost certainly require changes on GitHub's side.

Yup. I'll need to find time to catch the exception as the code currently expects the empty hash.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity, and will be closed if no further activity occurs. If this pull request was overlooked, forgotten, or should remain open for any other reason, please reply here to call attention to it and remove the stale status. Thank you for your contributions.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity, and will be closed if no further activity occurs. If this pull request was overlooked, forgotten, or should remain open for any other reason, please reply here to call attention to it and remove the stale status. Thank you for your contributions.

[lildude]

Still don't have the bandwidth for this work on the GitHub side - will label to stop the pings.

[aallrd]

Hello,
We bumped into this issue today, it took us some time to figure out what was wrong.
The error message proposed by this PR would have saved us quite a lot of time.
Btw, what's the reason for this hardcoded MAX_TREE_SIZE limitation?
Since there is no way to override it we need to patch the gem code in order to be able to use this project on our repo :/

[pchaigno]

@aallrd I'd welcome a pull request to make this threshold a parameter that defaults to the current value.

[aallrd]

@pchaigno would you be open to a PR to simply remove this threshold limitation?

[lildude]

@pchaigno would you be open to a PR to simply remove this threshold limitation?

I'd reject that 😁

The limit is in place because of the impact it has on performance and leads to timeouts on github.com. One repo of more than 100k files on one machine is fine. Millions of them like on github.com may become a problem very quickly.

[aallrd]

@lildude makes sense, thank you for the explanation behind this limitation 👍
I'll see what I can do to add a flag to override this default value.

[ehariyatie2478]

Loom

[Alhadis]

I'd reword this to be:

raise "Repository tree size exceeds MAX_LIMIT (#{MAX_LIMIT})"

Also, did you mean MAX_TREE_SIZE instead of MAX_LIMIT? I don't see the latter being used here.

[lildude]

Closing as this is now moot. If you're using Linguist in your own project, you can now set MAX_TREE_SIZE thanks to the work done in #6806, #6826 and #6830.