Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NSRegularExpression exception while generating objc documentation #370

Closed
matej opened this issue Nov 16, 2015 · 3 comments · Fixed by jpsim/SourceKitten#99
Closed

NSRegularExpression exception while generating objc documentation #370

matej opened this issue Nov 16, 2015 · 3 comments · Fixed by jpsim/SourceKitten#99
Assignees
@jpsim
Labels
bug
Milestone
Before

Comments

@matej
Copy link

matej commented Nov 16, 2015

This might be better suited as a SourceKitten issue, but since we stumbled upon it using jazzy and since using it is the easiest way I know of reproducing it, I'm posting here instead.

We tried using the new jazzy version to generate documentation for PSPDFKit, a fairly large Object-C framework. The command that seems to be getting us the farthest is this one:

jazzy --objc -x --objc,PSPDFKit.h,-x,objective-c,-isysroot,(xcrun --show-sdk-path --sdk iphonesimulator),-I,(pwd) --module PSPDFKit --output docs

Unfortunately this does end up throwing and exception after processing for a bit. sourcekitten[60380:28736521] *** Terminating app due to uncaught exception 'NSRangeException', reason: '*** -[NSRegularExpression enumerateMatchesInString:options:range:usingBlock:]: Range or index out of bounds'.

Here's the full output:

/usr/local/lib/ruby/gems/2.2.0/gems/rouge-1.9.1/lib/rouge/lexers/shell.rb:20: warning: already initialized constant Rouge::Lexers::Shell::KEYWORDS
/usr/local/lib/ruby/gems/2.2.0/gems/rouge-1.9.1/lib/rouge/lexers/shell.rb:20: warning: previous definition of KEYWORDS was here
/usr/local/lib/ruby/gems/2.2.0/gems/rouge-1.9.1/lib/rouge/lexers/shell.rb:25: warning: already initialized constant Rouge::Lexers::Shell::BUILTINS
/usr/local/lib/ruby/gems/2.2.0/gems/rouge-1.9.1/lib/rouge/lexers/shell.rb:25: warning: previous definition of BUILTINS was here
2015-11-16 18:35:55.684 sourcekitten[60380:28736521] *** Terminating app due to uncaught exception 'NSRangeException', reason: '*** -[NSRegularExpression enumerateMatchesInString:options:range:usingBlock:]: Range or index out of bounds'
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff9c72ce32 __exceptionPreprocess + 178
    1   libobjc.A.dylib                     0x00007fff9b7194fa objc_exception_throw + 48
    2   CoreFoundation                      0x00007fff9c79365d +[NSException raise:format:] + 205
    3   Foundation                          0x00007fff9483d085 -[NSRegularExpression(NSMatching) enumerateMatchesInString:options:range:usingBlock:] + 347
    4   Foundation                          0x00007fff9483cf18 -[NSRegularExpression(NSMatching) matchesInString:options:range:] + 159
    5   SourceKittenFramework               0x0000000102ffbe46 _TTSf4g_n___TF21SourceKittenFramework11insertMarksFTGSaVS_17SourceDeclaration_10limitRangeGSqVSC8_NSRange__GSaS0__ + 630
    6   SourceKittenFramework               0x0000000102fff0ff _TFF21SourceKittenFramework11insertMarksFTGSaVS_17SourceDeclaration_10limitRangeGSqVSC8_NSRange__GSaS0__U0_FS0_S0_ + 431
    7   SourceKittenFramework               0x0000000102fff477 _TTRXFo_oV21SourceKittenFramework17SourceDeclaration_oS0_zoPSs9ErrorType__XFo_iS0__iS0_zoPS1___ + 455
    8   SourceKittenFramework               0x0000000102ffc428 _TPA__TTRXFo_oV21SourceKittenFramework17SourceDeclaration_oS0_zoPSs9ErrorType__XFo_iS0__iS0_zoPS1___ + 72
    9   SourceKittenFramework               0x0000000103002d75 _TTSf4g_n___TTSg5GSaV21SourceKittenFramework17SourceDeclaration_GSaS0__Ss14CollectionTypeSs_GVSs17IndexingGeneratorGSaS0___GS2_GSaS0___Ss13GeneratorTypeSs_S0__SiSiSs16ForwardIndexTypeSs_SiSiSs18_SignedIntegerTypeSs_SiSiSs33_BuiltinIntegerLiteralConvertibleSs_Si_GVSs10ArraySliceS0__GS7_S0__Ss9IndexableSsGS7_S0__Ss12SequenceTypeSs_GS2_GS7_S0___GS2_GS7_S0___S3_Ss_S0__SiSiS4_Ss_SiSiS5_Ss_SiSiS6_Ss_Si_GS7_S0___S0__S0__S0____TFeRq_Ss14CollectionType_SsS_3mapu__Rq_S__fq_FzFzqqq_S_9GeneratorSs13GeneratorType7Elementqd__GSaqd___ + 1365
    10  SourceKittenFramework               0x0000000102ffbfea _TTSf4g_n___TF21SourceKittenFramework11insertMarksFTGSaVS_17SourceDeclaration_10limitRangeGSqVSC8_NSRange__GSaS0__ + 1050
    11  SourceKittenFramework               0x0000000102fb1b58 _TFFV21SourceKittenFramework20ClangTranslationUnitcFMS0_FT11headerFilesGSaSS_17compilerArgumentsGSaSS__S0_U3_FGSaVS_17SourceDeclaration_GSaS1__ + 24
    12  SourceKittenFramework               0x0000000102fb0f50 _TPA__TTRXFo_oGSaV21SourceKittenFramework17SourceDeclaration__oGSaS0__zoPSs9ErrorType__XFo_iGSaS0___iGSaS0__zoPS1___ + 64
    13  SourceKittenFramework               0x0000000102fb100f _TPA__TTSg5SSSSSs8HashableSs_GSaV21SourceKittenFramework17SourceDeclaration__GSaS1_____TFFe0_Rq_Ss8Hashable_21SourceKittenFrameworkVSs10Dictionary3mapu0__Rq_S__FGS1_q_q0__FzFzq0_qd__GS1_q_qd___U_FzTQd__Qd_0__TQd__Q__ + 95
    14  SourceKittenFramework               0x0000000102fb10aa _TPA__TTSg5SSSSSs8HashableSs_GSaV21SourceKittenFramework17SourceDeclaration__GSaS1_____TTRG0__Rq_Ss8Hashable_XFo_iq_iq0__iTq_qd___zoPSs9ErrorType__XFo_iTq_q0___iTq_qd___zoPS0___ + 74
    15  SourceKittenFramework               0x0000000102fc4aa8 _TTSf4g_n___TTSg5GVSs10DictionarySSGSaV21SourceKittenFramework17SourceDeclaration__GS_SSGSaS1___Ss14CollectionTypeSs_GVSs19DictionaryGeneratorSSGSaS1___GS3_SSGSaS1___Ss13GeneratorTypeSs_TSSGSaS1____GVSs15DictionaryIndexSSGSaS1___GS5_SSGSaS1___Ss16ForwardIndexTypeSs_SiSiSs18_SignedIntegerTypeSs_SiSiSs33_BuiltinIntegerLiteralConvertibleSs_VSs20_DisabledRangeIndex__GVSs5SliceGS_SSGSaS1____GS10_GS_SSGSaS1____Ss9IndexableSsGS10_GS_SSGSaS1____Ss12SequenceTypeSs_GVSs17IndexingGeneratorGS10_GS_SSGSaS1_____GS13_GS10_GS_SSGSaS1_____S4_Ss_TSSGSaS1____GS5_SSGSaS1___GS5_SSGSaS1___S6_Ss_SiSiS7_Ss_SiSiS8_Ss_S9__GS10_GS_SSGSaS1_____TSSGSaS1____TSSGSaS1____TSSGSaS1______TFeRq_Ss14CollectionType_SsS_3mapu__Rq_S__fq_FzFzqqq_S_9GeneratorSs13GeneratorType7Elementqd__GSaqd___ + 1736
    16  SourceKittenFramework               0x0000000102fb0943 _TTSf4n_n_d___TFV21SourceKittenFramework20ClangTranslationUnitCfMS0_FT11headerFilesGSaSS_17compilerArgumentsGSaSS__S0_ + 2707
    17  sourcekitten                        0x0000000102f3439b sourcekitten + 33691
    18  sourcekitten                        0x0000000102f2ed6c sourcekitten + 11628
    19  sourcekitten                        0x0000000102f2ee13 sourcekitten + 11795
    20  Result                              0x0000000102f8820a _TFO6Result6Result8analysisu0__Rq0_Ss9ErrorType_fGS0_q_q0__FT9ifSuccessFq_qd__9ifFailureFq0_qd___qd__ + 378
    21  Result                              0x0000000102f8945b _TTWu0_Rq0_Ss9ErrorType_GO6Result6Resultq_q0__S0_10ResultTypeS0_FS2_8analysisu__Rq_S2__fq_FT9ifSuccessFqq_S2_5Valueqd__9ifFailureFqq_S2_5Errorqd___qd__ + 235
    22  Result                              0x0000000102f86af1 _TFeRq_6Result10ResultType_S_S0_7flatMapu__Rq_S0__fq_FFqq_S0_5ValueGOS_6Resultqd__qq_S0_5Error_GS1_qd__qq_S0_5Error_ + 145
    23  sourcekitten                        0x0000000102f2e65e sourcekitten + 9822
    24  Commandant                          0x0000000102f5683e _TPA__TFFV10Commandant10AnyCommandcu__Rqd__S_11CommandTypezq_qqd__S1_11ClientError_FMGS0_q__Fqd__GS0_q__U_FOS_11CommandModeGO6Result6ResultT_GOS_15CommandantErrorQd____ + 94
    25  Commandant                          0x0000000102f565a1 _TFC10Commandant15CommandRegistry10runCommandurfGS0_q__FTSS9argumentsGSaSS__GSqGO6Result6ResultT_GOS_15CommandantErrorq____ + 273
    26  Commandant                          0x0000000102f56b87 _TFC10Commandant15CommandRegistry4mainurfGS0_q__FT9argumentsGSaSS_11defaultVerbSS12errorHandlerFq_T__T_ + 503
    27  Commandant                          0x0000000102f56987 _TFC10Commandant15CommandRegistry4mainurfGS0_q__FT11defaultVerbSS12errorHandlerFq_T__T_ + 135
    28  sourcekitten                        0x0000000102f2dff0 sourcekitten + 8176
    29  libdyld.dylib                       0x00007fff99c905ad start + 1
    30  ???                                 0x000000000000000a 0x0 + 10
)
libc++abi.dylib: terminating with uncaught exception of type NSException
/usr/local/lib/ruby/gems/2.2.0/gems/jazzy-0.4.0/lib/jazzy/executable.rb:36:in `execute_command': /usr/local/lib/ruby/gems/2.2.0/gems/jazzy-0.4.0/lib/jazzy/SourceKitten/bin/sourcekitten ["doc", "--objc", "PSPDFKit.h", "-x", "objective-c", "-isysroot", "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator9.1.sdk", "-I", "/Users/matej/Desktop/PSPDFKit"] (RuntimeError)

2015-11-16 18:35:55.684 sourcekitten[60380:28736521] *** Terminating app due to uncaught exception 'NSRangeException', reason: '*** -[NSRegularExpression enumerateMatchesInString:options:range:usingBlock:]: Range or index out of bounds'

*** First throw call stack:

(

    0   CoreFoundation                      0x00007fff9c72ce32 __exceptionPreprocess + 178

    1   libobjc.A.dylib                     0x00007fff9b7194fa objc_exception_throw + 48

    2   CoreFoundation                      0x00007fff9c79365d +[NSException raise:format:] + 205

    3   Foundation                          0x00007fff9483d085 -[NSRegularExpression(NSMatching) enumerateMatchesInString:options:range:usingBlock:] + 347

    4   Foundation                          0x00007fff9483cf18 -[NSRegularExpression(NSMatching) matchesInString:options:range:] + 159

    5   SourceKittenFramework               0x0000000102ffbe46 _TTSf4g_n___TF21SourceKittenFramework11insertMarksFTGSaVS_17SourceDeclaration_10limitRangeGSqVSC8_NSRange__GSaS0__ + 630

    6   SourceKittenFramework               0x0000000102fff0ff _TFF21SourceKittenFramework11insertMarksFTGSaVS_17SourceDeclaration_10limitRangeGSqVSC8_NSRange__GSaS0__U0_FS0_S0_ + 431

    7   SourceKittenFramework               0x0000000102fff477 _TTRXFo_oV21SourceKittenFramework17SourceDeclaration_oS0_zoPSs9ErrorType__XFo_iS0__iS0_zoPS1___ + 455

    8   SourceKittenFramework               0x0000000102ffc428 _TPA__TTRXFo_oV21SourceKittenFramework17SourceDeclaration_oS0_zoPSs9ErrorType__XFo_iS0__iS0_zoPS1___ + 72

    9   SourceKittenFramework               0x0000000103002d75 _TTSf4g_n___TTSg5GSaV21SourceKittenFramework17SourceDeclaration_GSaS0__Ss14CollectionTypeSs_GVSs17IndexingGeneratorGSaS0___GS2_GSaS0___Ss13GeneratorTypeSs_S0__SiSiSs16ForwardIndexTypeSs_SiSiSs18_SignedIntegerTypeSs_SiSiSs33_BuiltinIntegerLiteralConvertibleSs_Si_GVSs10ArraySliceS0__GS7_S0__Ss9IndexableSsGS7_S0__Ss12SequenceTypeSs_GS2_GS7_S0___GS2_GS7_S0___S3_Ss_S0__SiSiS4_Ss_SiSiS5_Ss_SiSiS6_Ss_Si_GS7_S0___S0__S0__S0____TFeRq_Ss14CollectionType_SsS_3mapu__Rq_S__fq_FzFzqqq_S_9GeneratorSs13GeneratorType7Elementqd__GSaqd___ + 1365

    10  SourceKittenFramework               0x0000000102ffbfea _TTSf4g_n___TF21SourceKittenFramework11insertMarksFTGSaVS_17SourceDeclaration_10limitRangeGSqVSC8_NSRange__GSaS0__ + 1050

    11  SourceKittenFramework               0x0000000102fb1b58 _TFFV21SourceKittenFramework20ClangTranslationUnitcFMS0_FT11headerFilesGSaSS_17compilerArgumentsGSaSS__S0_U3_FGSaVS_17SourceDeclaration_GSaS1__ + 24

    12  SourceKittenFramework               0x0000000102fb0f50 _TPA__TTRXFo_oGSaV21SourceKittenFramework17SourceDeclaration__oGSaS0__zoPSs9ErrorType__XFo_iGSaS0___iGSaS0__zoPS1___ + 64

    13  SourceKittenFramework               0x0000000102fb100f _TPA__TTSg5SSSSSs8HashableSs_GSaV21SourceKittenFramework17SourceDeclaration__GSaS1_____TFFe0_Rq_Ss8Hashable_21SourceKittenFrameworkVSs10Dictionary3mapu0__Rq_S__FGS1_q_q0__FzFzq0_qd__GS1_q_qd___U_FzTQd__Qd_0__TQd__Q__ + 95

    14  SourceKittenFramework               0x0000000102fb10aa _TPA__TTSg5SSSSSs8HashableSs_GSaV21SourceKittenFramework17SourceDeclaration__GSaS1_____TTRG0__Rq_Ss8Hashable_XFo_iq_iq0__iTq_qd___zoPSs9ErrorType__XFo_iTq_q0___iTq_qd___zoPS0___ + 74

    15  SourceKittenFramework               0x0000000102fc4aa8 _TTSf4g_n___TTSg5GVSs10DictionarySSGSaV21SourceKittenFramework17SourceDeclaration__GS_SSGSaS1___Ss14CollectionTypeSs_GVSs19DictionaryGeneratorSSGSaS1___GS3_SSGSaS1___Ss13GeneratorTypeSs_TSSGSaS1____GVSs15DictionaryIndexSSGSaS1___GS5_SSGSaS1___Ss16ForwardIndexTypeSs_SiSiSs18_SignedIntegerTypeSs_SiSiSs33_BuiltinIntegerLiteralConvertibleSs_VSs20_DisabledRangeIndex__GVSs5SliceGS_SSGSaS1____GS10_GS_SSGSaS1____Ss9IndexableSsGS10_GS_SSGSaS1____Ss12SequenceTypeSs_GVSs17IndexingGeneratorGS10_GS_SSGSaS1_____GS13_GS10_GS_SSGSaS1_____S4_Ss_TSSGSaS1____GS5_SSGSaS1___GS5_SSGSaS1___S6_Ss_SiSiS7_Ss_SiSiS8_Ss_S9__GS10_GS_SSGSaS1_____TSSGSaS1____TSSGSaS1____TSSGSaS1______TFeRq_Ss14CollectionType_SsS_3mapu__Rq_S__fq_FzFzqqq_S_9GeneratorSs13GeneratorType7Elementqd__GSaqd___ + 1736

    16  SourceKittenFramework               0x0000000102fb0943 _TTSf4n_n_d___TFV21SourceKittenFramework20ClangTranslationUnitCfMS0_FT11headerFilesGSaSS_17compilerArgumentsGSaSS__S0_ + 2707

    17  sourcekitten                        0x0000000102f3439b sourcekitten + 33691

    18  sourcekitten                        0x0000000102f2ed6c sourcekitten + 11628

    19  sourcekitten                        0x0000000102f2ee13 sourcekitten + 11795

    20  Result                              0x0000000102f8820a _TFO6Result6Result8analysisu0__Rq0_Ss9ErrorType_fGS0_q_q0__FT9ifSuccessFq_qd__9ifFailureFq0_qd___qd__ + 378

    21  Result                              0x0000000102f8945b _TTWu0_Rq0_Ss9ErrorType_GO6Result6Resultq_q0__S0_10ResultTypeS0_FS2_8analysisu__Rq_S2__fq_FT9ifSuccessFqq_S2_5Valueqd__9ifFailureFqq_S2_5Errorqd___qd__ + 235

    22  Result                              0x0000000102f86af1 _TFeRq_6Result10ResultType_S_S0_7flatMapu__Rq_S0__fq_FFqq_S0_5ValueGOS_6Resultqd__qq_S0_5Error_GS1_qd__qq_S0_5Error_ + 145

    23  sourcekitten                        0x0000000102f2e65e sourcekitten + 9822

    24  Commandant                          0x0000000102f5683e _TPA__TFFV10Commandant10AnyCommandcu__Rqd__S_11CommandTypezq_qqd__S1_11ClientError_FMGS0_q__Fqd__GS0_q__U_FOS_11CommandModeGO6Result6ResultT_GOS_15CommandantErrorQd____ + 94

    25  Commandant                          0x0000000102f565a1 _TFC10Commandant15CommandRegistry10runCommandurfGS0_q__FTSS9argumentsGSaSS__GSqGO6Result6ResultT_GOS_15CommandantErrorq____ + 273

    26  Commandant                          0x0000000102f56b87 _TFC10Commandant15CommandRegistry4mainurfGS0_q__FT9argumentsGSaSS_11defaultVerbSS12errorHandlerFq_T__T_ + 503

    27  Commandant                          0x0000000102f56987 _TFC10Commandant15CommandRegistry4mainurfGS0_q__FT11defaultVerbSS12errorHandlerFq_T__T_ + 135

    28  sourcekitten                        0x0000000102f2dff0 sourcekitten + 8176

    29  libdyld.dylib                       0x00007fff99c905ad start + 1

    30  ???                                 0x000000000000000a 0x0 + 10

)

libc++abi.dylib: terminating with uncaught exception of type NSException
    from /usr/local/lib/ruby/gems/2.2.0/gems/jazzy-0.4.0/lib/jazzy/sourcekitten.rb:136:in `run_sourcekitten'
    from /usr/local/lib/ruby/gems/2.2.0/gems/jazzy-0.4.0/lib/jazzy/doc_builder.rb:57:in `block in build'
    from /usr/local/lib/ruby/gems/2.2.0/gems/jazzy-0.4.0/lib/jazzy/doc_builder.rb:55:in `chdir'
    from /usr/local/lib/ruby/gems/2.2.0/gems/jazzy-0.4.0/lib/jazzy/doc_builder.rb:55:in `build'
    from /usr/local/lib/ruby/gems/2.2.0/gems/jazzy-0.4.0/bin/jazzy:15:in `<top (required)>'
    from /usr/local/bin/jazzy:23:in `load'
    from /usr/local/bin/jazzy:23:in `<main>'

And here are our headers: http://cl.ly/2u3K3S1P3C0B

Any ideas what goes wrong here?
@jpsim
Copy link
Collaborator

jpsim commented Nov 16, 2015

Thanks for reporting this, I've found the issue and it looks pretty easy to fix. I'll ping you again here once I have a fix in place.

PS: really excited to see PSPDFKit trying this out! 😄

jpsim added a commit to jpsim/SourceKitten that referenced this issue Nov 16, 2015
@jpsim
fix an out-of-bounds issue when generating pragma marks
e61400b 
It looks as though clang_getCursorExtent() occasionally exceeds the file's
length. This fixes realm/jazzy#370
@jpsim jpsim mentioned this issue Nov 16, 2015
Merged
@jpsim jpsim reopened this Nov 16, 2015
@jpsim jpsim mentioned this issue Nov 16, 2015
Merged
@jpsim jpsim mentioned this issue Nov 16, 2015
Closed
@jpsim
Copy link
Collaborator

jpsim commented Nov 16, 2015

This issue is now fixed on jazzy's master branch, which you can install locally by running bundle install && bundle exec rake install.

However, PSPDFKit really highlights that jazzy needs some performance optimizations to handle larger projects gracefully. Generating docs for your project takes ~50 seconds on my Macbook Pro!

The silver lining of this is that we've never done much profiling or optimization before, so there must be lots of low hanging fruit that we could tackle to improve the situation. I've opened #372 to track that effort.

The other major problem is a usability one. By my count, PSPDFKit has 116 categories, 273 classes, 352 constants, 103 enums, 97 protocols, 16 type definitions and 31 functions, which means that the resulting docs look like this: https://static.realm.io/jazzy_demo/PSPDFKit

To make this usable, you'll definitely have to use custom templates (-t) that can visually organize the structure of the project in a more approachable way. I'm certainly amenable to PRs that can improve this for everyone in the general sense too.

@jpsim jpsim mentioned this issue Nov 16, 2015
Closed
@matej
Copy link
Author

matej commented Nov 17, 2015

Oh, wow. Thanks for the quick turnaround time on this. Works on my side now as well.

However, PSPDFKit really highlights that jazzy needs some performance optimizations to handle larger projects gracefully. Generating docs for your project takes ~50 seconds on my Macbook Pro!

Considering our release script already takes some serious time, adding 50 seconds would't necessary be a blocker. Also, it was closer to 30 when I tried it on my MacBook Pro. But sure, if there's low hanging fruit, do get it. You should at least have a pretty good benchmark test case now. :)

To make this usable, you'll definitely have to use custom templates (-t) that can visually organize the structure of the project in a more approachable way.

Yeah, I agree. Here's our current (apple doc based) documentation as a comparison: https://pspdfkit.com/api/ios/index.html.

There are some issues with the current generator, e.g., the class documentation of https://pspdfkit.com/api/ios/Classes/PSPDFAnnotationToolbar.html is missing the first paragraph, compared to https://static.realm.io/jazzy_demo/PSPDFKit/Classes/PSPDFAnnotationToolbar.html which parses that correctly. But it's currently a tradeoff between the two. Some other parts, like the configurations property of the same class currently looks better when using apple doc.

Thanks again for the help and great work in general on jazzy!

@1ec5 1ec5 mentioned this issue Dec 8, 2015
Closed
@pigeondev2 pigeondev2 modified the milestone: The Past Nov 22, 2016
@pigeondev2 pigeondev2 added the bug label Nov 23, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jpsim jpsim

Labels
bug
Projects
None yet
Milestone
Before
Development

Successfully merging a pull request may close this issue.

fix an out-of-bounds issue when generating pragma marks jpsim/SourceKitten
3 participants
@matej @jpsim @pigeondev2