Markdown reference links cannot be used from DFM include files

[cameronelliott]

Markdown reference links cannot be used from DFM include files

Functional impact

This is a markdown reference link: [example]: http://example.com
This is a DFM include statement: [!include[links](../linksinc.md)]
Currently markdown-reference-links must be used from the same md file in which they are defined.

Minimal repro steps

1. Create a file "a.md":

This is an [example], the last word should be a link.
[!include[links](linksinc.md)]

2. Create a file "linksinc.md"

This is an [example], the last word should be a link.
[example]: http://example.com

3. Include a.md into a docfx project, and build it.

Expected result

Both "[example]" sections should be found an expanded.

Actual result

Only "[example]" sections inside of linksinc.md will be expanded.

Further technical details

It seems the way the markdown engine is built, the included markdown files are being expanded separately, not allowing markdown reference links to be resolved into an include-file.

Maybe the markdown-engine not resolve markdown reference links until !include statements have merged all include files into a single unit.
[This would work more like the Standard C preprocessor]

Thank you for creating Docfx, it is a awesome project!.

[vwxyzh]

It is by designed.
[!include[...](...)] should not take any context from article.

[cameronelliott]

I cannot shy away from offering an honest opinion on this.
I have many years of software experience.

I think this is a mistake.

When you have 10+ complicated links:

[Intel_Quick_Sync_Video]: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video
[Intel_HD_and_Iris_Graphics]: https://en.wikipedia.org/wiki/Intel_HD_and_Iris_Graphics
[List_of_Intel_graphics_processing_units]: https://en.wikipedia.org/wiki/List_of_Intel_graphics_processing_units
[StreamTranscoder]: xref:LimeVideoSDK.QuickSync.StreamTranscoder
[StreamDecoder]: xref:LimeVideoSDK.QuickSync.StreamDecoder  
[LowLevelEncoderCSharp]: LimeVideoSDK.QuickSync.LowLevelEncoderCSharp
[mfxVideoParam]: xref:LimeVideoSDK.QuickSyncTypes.mfxVideoParam
[CPUConvertResize]: xref:LimeVideoSDK.CPUConvertResize
[NV12ToXXXXConverter]: xref:LimeVideoSDK.CPUConvertResize.NV12ToXXXXConverter
[NV12FromXXXXConverter]: xref:LimeVideoSDK.CPUConvertResize.NV12FromXXXXConverter  
[Player1]: xref:samples#Player1
[Decoder1]: xref:samples#Decoder1
[Encoder1]: xref:samples#Encoder1

I REALLY REALLY REALLY want to use an include file to maintain and include the links rather than copying the links into each markdown file!

If I must copy the above blocks into every markdown file, the project becomes very very very hard to maintain and update.

Because of your design decision, an article cannot use markdown-reference-links from included files.

Because of the current design I must copy those links above into dozens [maybe hundreds] of markdown articles!!

I think it is a mistake.

I still love DocFx, and think your team has created something awesome!
:) :)

[cameronelliott]

Because of your design decision, I will probably be forced to use an external pre-processor to maintain markdown-reference-links, and have them copied into .MD files before running DocFx for a build.
I predict others who use DocFx for link heavy documentation will come back and repeat this observation and request in the future.

I still love DocFx, and think your team has created something awesome!
:) :)

[vwxyzh]

@Cameron-elliott the reason is:
Create inc.md like this:

*a*

Then we think wherever you include this file will get a.

If some article use the include file like this:

*[!include[](inc.md)]*

And we pass the context from article, we will get a.
This behavior is strange.
And if some one write an article like this:

*x[!include[](inc.md)]

We will get xa*.

[cameronelliott]

That is not a good argument.

The undesirable behavior you describe is exactly how every other "include file" feature works in every other language!
And with very good reason, that behavior enables useful programming work to be done.

Almost every other language that includes the concept of an include file,
allows the elements of the include files to be referenced from the including-file.
[the elements I am referring to are markdown reference links]

We could compare the two concepts:

1. Allows "including-file" to reference elements from "included-file".
2. Does not allows "including-file" to reference elements from "included-file".

• C and C++ language uses method Create overall layout and add BackEnd projects #1
• PHP uses method Create overall layout and add BackEnd projects #1
• Fortran, Cobol, JSP, HTML-SSI allow follow method Create overall layout and add BackEnd projects #1
• DocFx uses method Back end #2

DocFx is unusual, in that defined elements in included-files [children] are not propagated to the including-file. No other language works this way.
Other languages include ALL elements from the child in the parent!

But, do not listen to my opinion, please go ask senior developers at MSFT-China their opinion,
that is, developers with at least 20 years experience, and ask them:
"should the DocFx include file system allow the 'including-file' [parent] to reference elements from an 'included-file' [child]"

Do not be surprised if they do not understand, and ask "why else would you design an include file system?"

Thank you for creating DocFx, and for reading my comments. I hope we can form a productive effort to move DocFx forward.

[vwxyzh]

By now our design is: include result html, not include source md.
@chenkennt @vicancy

[vwxyzh]

@Cameron-elliott If your document has many reference definition, why not use xref?
e.g.: your article is:

Content is: [Intel Quick Sync Video][Intel_Quick_Sync_Video] and [Intel HD and Iris Graphics][Intel_HD_and_Iris_Graphics]

[Intel_Quick_Sync_Video]: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video
[Intel_HD_and_Iris_Graphics]: https://en.wikipedia.org/wiki/Intel_HD_and_Iris_Graphics

1. Create an xref map file:

   ### YamlMime:XRefMap
   hrefUpdated: true
   references:
   - uid: Intel_Quick_Sync_Video
     name: "Intel Quick Sync Video"
     href: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video
   - uid: Intel_HD_and_Iris_Graphics
     name: "Intel HD and Iris Graphics"
     href: https://en.wikipedia.org/wiki/Intel_HD_and_Iris_Graphics

2. then update your article:

   Content is: @Intel_Quick_Sync_Video and @Intel_HD_and_Iris_Graphics

3. final, update your docfx.json, add the path of xref map in "xref", or use command line option -x

[vicancy]

Unlike C or C++, Markdown syntax is very sensitive to indentation, empty lines, layout, and context. Purely inclusion is error prone and result is unpredictable. Imagine cases when purely include file content inside a table, inside a list, with 4 indent before it, etc.. The result can be totally different from what the include file looks like.

When designing DFM, we hope DFM is easy to use and result predictable. The best case is that the result is quite similar to what markdown editors(like Github) preview shows. So when we introduce include into DFM, we are actually include the final html.

[cameronelliott]

Okay, very good points everybody.
I especially appreciate the advice on creating an xref map file, I will give that a try!
Thank you, and thank you for creating DocFx.
:)

[weitzhandler]

A bit related: #8712.