Crash on complex Templates

[LtJulius]

I just tried out to build the documentation for my project and the process crashed while trying to generate files for one of my more complex template classes.

terminate called after throwing an instance of 'std::filesystem::__cxx11::filesystem_error'
  what():  filesystem error: cannot create directories: File name too long [docs/hg/LoaderChain_u003c_u000a_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020RequestType__u002c_u000a_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020ResponseType__u002c_u000a_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020StageRefTypes__u002c_u000a_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020CacheStageType__u002c_u000a_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020FeedbackStageType__u002c_u000a_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020TransformerStageType__u002c_u000a_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020SourceLoaderType__u002c_u000a_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020true_u000a_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u0020_u003e]
Aborted (core dumped)

template <
        IsRequestValueType RequestType_,
        IsResponseValueType ResponseType_ = ResourceBase,
        typename StageRefTypes_ = MetaTypeWrapperList<
            _STD type_identity_t,
            _STD type_identity_t,
            _STD type_identity_t,
            sptr
        >,
        class CacheStageType_ = Cache<RequestType_, ResponseType_>,
        class FeedbackStageType_ = Feedback<RequestType_, ResponseType_>,
        class TransformerStageType_ = Transformer<RequestType_, ResponseType_>,
        class SourceLoaderType_ = SourceLoader,
        bool Streamable_ = assets::IsStreamableAsset<RequestType_>>
class LoaderChain :
    public LoaderChainBase<
        RequestType_, ResponseType_, StageRefTypes_,
        CacheStageType_, FeedbackStageType_, TransformerStageType_, SourceLoaderType_> { ... };

Don't know whether this helps, but if you need further source code or information, just msg me.

[lawmurray]

Thanks @LtJulius. Would you mind posting the code immediately after, like inheritance declaration, or up to the opening curly brace for the class. Looks to me like the parser might be picking up some additional characters, even whitespace, and considering them part of the name.

[LtJulius]

@lawmurray Updated the code snippet, actually there are several more classes like this within the same hierarchy, so LoaderChain<...> -> LoaderChainBase<...> -> Loader<...> -> LoaderBase 😅

[lawmurray]

Thanks for the update @LtJulius. Unfortunately I am not able to reproduce. Keen to get this fixed though, real world complex template code like this is exactly the sort of thing to put Doxide through its paces.

Can you let me know the following so I can keep investigating?

1. The platform you're on (e.g. Linux, Mac).
2. How you installed (e.g. package, or from source).
3. If from source, from the latest GitHub commit?
4. If from source, how did you install the tree-sitter dependency? E.g. also from source or from a package?

Also, do you get the error if you put the code, and only the code (plus a doc comment), in a header file, or does it only occur with additional code around? E.g. I tested with the following header:

/**
 * Test of a complex template.
 */
template <
        IsRequestValueType RequestType_,
        IsResponseValueType ResponseType_ = ResourceBase,
        typename StageRefTypes_ = MetaTypeWrapperList<
            _STD type_identity_t,
            _STD type_identity_t,
            _STD type_identity_t,
            sptr
        >,
        class CacheStageType_ = Cache<RequestType_, ResponseType_>,
        class FeedbackStageType_ = Feedback<RequestType_, ResponseType_>,
        class TransformerStageType_ = Transformer<RequestType_, ResponseType_>,
        class SourceLoaderType_ = SourceLoader,
        bool Streamable_ = assets::IsStreamableAsset<RequestType_>>
class LoaderChain :
    public LoaderChainBase<
        RequestType_, ResponseType_, StageRefTypes_,
        CacheStageType_, FeedbackStageType_, TransformerStageType_, SourceLoaderType_> { };    

Put that in a test.hpp file, then ran:

doxide init
doxide build

And it worked.

I wonder whether tree-sitter encounters a parse error earlier on in the code that knocks it off balance by the time it arrives here.

[LtJulius]

Alright, I reinstalled doxide. This time from source, even though I had to change some include paths within tree-sitter because it failed while resolving the includes paths...

Anyway. build and installed doxide, but got the same error.
Don' know whether it will effect the parsing, but I get a lot of warnings, because previously I used jsdoc:

warning: unrecognized command: @date
warning: unrecognized command: @exception
warning: unrecognized command: @author
warning: unrecognized command: @details

After that, the same error occurs as above. But I digged a bit into it. The actual failing class is a partial template specialization.

template <
        IsRequestValueType RequestType_,
        IsResponseValueType ResponseType_,
        typename StageRefTypes_,
        class CacheStageType_,
        class FeedbackStageType_,
        class TransformerStageType_,
        class SourceLoaderType_>
    class LoaderChain<
            RequestType_,
            ResponseType_,
            StageRefTypes_,
            CacheStageType_,
            FeedbackStageType_,
            TransformerStageType_,
            SourceLoaderType_,
            true
        > :
        public LoaderChainBase<
            RequestType_, ResponseType_, StageRefTypes_,
            CacheStageType_, FeedbackStageType_, TransformerStageType_, SourceLoaderType_> { };

Don't know whether this helps.

Distributor ID: Ubuntu
Description: Ubuntu 22.04.2 LTS
Release: 22.04
Codename: jammy

If you need; I could even send you the whole header file.

[lawmurray]

Thanks @LtJulius , got it this time!

Explanation: for partial template specializations, the entity name is taken as the class name and contents of the angle brackets to ensure that different specializations can be distinguished. The sanitizer for converting entity names to file names was encoding the spaces, which could make for lengthy file names that exceed the file system limit of 255 characters. The fix is to first strip whitespace to shorten the name, then truncate at 255 characters (minus a little space for file extensions).

Thanks again for the report, and please keep them coming if you encounter further issues.

[lawmurray]

Tagged as v0.0.2, packages are rebuilding.

[lawmurray]

Packages are now built and available. Please reopen if this does not seem to be resolved, however.

[lawmurray]

@LtJulius Update on this one, v0.4.0 provides:

1. Better reporting of parse errors, including line and column numbers, for debugging purposes.
2. Error recovery logic to autocorrect some parsing errors related to the use of preprocessor symbols.
3. The ability to define preprocessor symbols in the configuration file.

This should address the remaining side issues sprinkled throughout the thread here.