Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relative links not working in asciidoc render #26203

Open
deparcus opened this issue Jul 28, 2023 · 26 comments
Open

Relative links not working in asciidoc render #26203

deparcus opened this issue Jul 28, 2023 · 26 comments
Labels

Comments

@deparcus
Copy link

Description

I activated asciidoc file render based on https://docs.gitea.com/next/administration/external-renderers:

  • install asciidoctor via apt install asciidoctor
  • add external render in app.ini
[markup.asciidoc]
ENABLED = true
FILE_EXTENSIONS = .adoc,.asciidoc
RENDER_COMMAND = "asciidoctor -s -a showtitle --out-file=- -"
; Input is not a standard input but a file
IS_INPUT_FILE = false

Then I created a branch in one of my repositories and migrated Markdown files to adoc files.
In the main *.adoc file I use relative links to link to other *.adoc files.
Therefore I followed the asciidoctor documentation.

Example file docs/README.adoc:

= Network documentation

== Index

* link:general/[General]
** link:general/netplan.drawio.svg[Netplan]
** link:general/Patchpanels.adoc[Patchpanels]
** link:general/static-IPs.adoc[Static IPs]

I also tried other relative links:

= Network documentation

== Index

* link:./general/[General]
** link:./general/netplan.drawio.svg[Netplan]
** link:./general/Patchpanels.adoc[Patchpanels]
** link:./general/static-IPs.adoc[Static IPs]

In VSCode the links are working and I can navigate to the files in subdirectories by clicking the link in the main file.

The link is interpreted as it starts from the root of the branch. So /docs/ is missing. But as the README.adoc is in the folder docs, the links should be interpreted as starting from there.

This is shown in the screenshot below:
gitea_relative_adoc_links_not_working_correctly

I discovered that issue also in another repository, where I first tried it.

When I use Markdown (currently my master branch), everything works as is should.

Example file doc/README.md:

# Network documentation

## Index

* [General](./general/)
  * [Netplan](./general/netplan.drawio.svg)
  * [Patchpanels](./general/Patchpanels.md)
  * [Static IPs](./general/static-IPs.md)

Working links, see screenshot:
gitea_relative_md_links_working_correctly

Am I doing something wrong?
Or is there maybe an issue with asciidoctor or the asciidoctor command in app.ini?

Would be great if anybody could help me.

Gitea Version

1.20.1

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

gitea_relative_adoc_links_not_working_correctly
gitea_relative_md_links_working_correctly

Git Version

2.30.2

Operating System

Raspbian/Debian 11.7

How are you running Gitea?

Local install with binary and systemd service.

Database

MySQL

@deparcus
Copy link
Author

deparcus commented Jul 28, 2023

Currently I found another strange thing:

It is not working as described above when clicking the /docs/ folder in the file list and starting from there.

But when I go to the README.adoc file in the /docs/ folder via a link in my main README.md file in the root of the repository/branch it is working.

Link to README.adoc in README.md:

gitea_link_README md_to_README adoc

Now (for some reason) relative links in README.adoc are working as they should:

gitea_relative_adoc_links_working_when_coming_from_README

So at the end I'm not sure if this is really a asciidoc render issue or a general behavior of gitea.
But if I just use Markdown (also in my doc folder), everything works as it should no matter how the README.md file in the doc folder is accessed.

@Mo-Gul
Copy link

Mo-Gul commented Aug 19, 2023

I have exactly the same issue on two different Gitea's.

tl;dr of the above is

  • When the docs/README.adoc is active/selected (the address bar shows <repository>/src/branch/main/docs/README.adoc), the link destination works as expected, i.e. are <repository>/src/branch/main/docs/<link_dest>.
  • When "only" the docs folder is selected (the address bar shows <repository>/src/branch/main/docs), the link destination is <repository>/src/branch/main/<link_dest>.
  • When being in the root folder (the address bar shows <repository>), the link destination is <repository>/<link_dest>.

With

  • <repository> for this repository would be https://github.com/go-gitea/gitea,
  • <link_dest> would be e.g. link.adoc, and
  • the link is written as link:<link_dest>.adoc[link destination] in the docs/README.adoc file.

@Mo-Gul
Copy link

Mo-Gul commented Aug 19, 2023

Because GitHub must have had the same issue (I guess), I found github/markup#1039 which might be a good start so find out, how it was solved there.

Hopefully then also xrefs can be handled or -- depending on safe-mode -- include directives are converted automatically to links (as can be seen e.g. at https://github.com/opendevise/asciidoc-samples/blob/main/runtime.adoc)

Maybe @mojavelinux could as least give a hint on how to solve the issue here (as well) ...

@mojavelinux
Copy link

Please don't @ me in projects I'm not involved with. You are welcome to ask in the Asciidoctor project chat at https://chat.asciidoctor.org.

@Mo-Gul
Copy link

Mo-Gul commented Aug 25, 2023

@deparcus, do you want to ask for help in the asciidoctor chat since you started the issue? If not, please let me know and I'll try to do it myself.

@deparcus
Copy link
Author

@Mo-Gul : Of course I could try, but I think you might be able to explain the details and background probably much better than I could.

@Mo-Gul
Copy link

Mo-Gul commented Aug 25, 2023

@deparcus, I'd say you give it a first try and can give a hint to this issue, which I guess is a good start. Then you can add the link to the chat here so I (and others) can follow the conversation. If it is useful that I add my few cents I'll join it ...

@deparcus
Copy link
Author

deparcus commented Aug 25, 2023

@Mo-Gul : I opened the chat at asciidoctor. You can follow and comment there by this link.

@deparcus
Copy link
Author

@Mo-Gul : There seems to be a solution in the asciidoctor chat.

Looks like gitea has to change sth.

How can we proceed here?

@jtiee
Copy link

jtiee commented Sep 1, 2023

The bug is that Gitea does not add a trailing slash in links to repos/folders.

Consider this situation:

The user views https://<gitea-host>/<user>/<repository>. Gitea automatically renders the README file (README.adoc in this case, but any format would suffer the same problem).

Relative links in the auto-rendered README might look like docs/index.adoc. When you click such a link, the browser combines the current URL context with the relative link. The combination uses the existing "folder" (path segment) in the current URL as the context.

What "folder" is that? https://<gitea-host>/<user> because there is no trailing slash after <repository>. So, the browser then attempts to fetch https://<gitea-host>/<user>/docs/index.adoc, which results in a 404.

If Gitea adds the trailing slash to repo/folder links, then when the user clicks the docs/index.adoc link, the browser would fetch https://<gitea-host>/<user>/<repository>/docs/index.adoc, which is correct.

You can manually add a trailing slash (and reload the page) to any repo/folder view and click a relative link in an auto-generated README file to confirm the correct behavior.

I used gitea-1.20.3-darwin-10.12-amd64 for testing.

@Mo-Gul
Copy link

Mo-Gul commented Sep 1, 2023

Seems you are right and the "same" problem exists for markdown files as well. See #18592.

@deparcus
Copy link
Author

deparcus commented Sep 4, 2023

hmm... as I wrote in my first to posts, I cannot reproduce the behaviour when using *.md Markdown files.

@Mo-Gul
Copy link

Mo-Gul commented Sep 7, 2023

I can confirm that with Gitea v1.19.0 a from a README.md in the docs folder work to another MD file in the docs folder works regardless of how the README.md is shown.

Unfortunately, I couldn't find out why so far. But maybe this has something to do with the Slugify rules for Gitea.

@deparcus, could you setup some dummy Gitea servers with previous versions so we could drill down since when the links are working? Maybe then it is easier to find out exactly why/how this is working for markdown files and thus there hopefully is a good start for a asciidoctor solution.

@deparcus
Copy link
Author

deparcus commented Sep 8, 2023

@Mo-Gul : sure. Do you have a feeling from which version I should start? Then I would start updating version by version and try to find out when the discussed Markdown links are working.

@Mo-Gul
Copy link

Mo-Gul commented Sep 9, 2023

Not really, but I guess I would start with v1.11 and then do a "binary search" when it isn't working there up to <v1.19 where it is working. Maybe it also a good idea to start with v1.16.1 which release date is before the above mentioned bug report from me.

Good luck!!

@deparcus
Copy link
Author

ok, thanks. I'll start with v1.11.0 until v1.19.x and do the first probe for the binary search at v1.16.1.

Currently I have limited access to the internet (and cannot download large files) due to a few days off. So I can start checking the issues earliest at end of september / beginning of october.

If this is too late, then probably somebody else could jump in?
Otherwise I'll let you all know when I have the first results.

@deparcus
Copy link
Author

deparcus commented Oct 13, 2023

@Mo-Gul : Sorry for answering that late.

I now set up a gitea 1.11.0 instance based on docker-compose to test the relative md-links.

To me it seems that it's working in that version, but I want to doublecheck if I'm testing the right things, before I continue playing around with the version (what is really easy with docker).

Attached you find a zip-file of my testing repo: relative-md-links-master.zip

Here's my docker-compose.yml:

version: "3"

networks:
  gitea:
    external: false

volumes:
  gitea:
    driver: local
  mysql:
    driver: local

services:
  server:
    image: gitea/gitea:1.11.0
    container_name: gitea
    environment:
      - USER_UID=1000
      - USER_GID=1000
      - GITEA__database__DB_TYPE=mysql
      - GITEA__database__HOST=db:3306
      - GITEA__database__NAME=gitea
      - GITEA__database__USER=gitea
      - GITEA__database__PASSWD=gitea
    restart: always
    networks:
      - gitea
    volumes:
      - gitea:/data
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "3000:3000"
      - "222:22"
    depends_on:
      - db

  db:
    image: mysql:8
    restart: always
    environment:
      - MYSQL_ROOT_PASSWORD=gitea
      - MYSQL_USER=gitea
      - MYSQL_PASSWORD=gitea
      - MYSQL_DATABASE=gitea
    networks:
      - gitea
    volumes:
      - mysql:/var/lib/mysql

Setup instructions:

  1. (on a debian system) Just run apt install docker-compose to install everything you need
  2. run docker-compose up (when you want to see the log output) or docker-compose up -d (if you want to send it to the background) as root in the folder where docker-compose.yml is located to start the containers
  3. After that you have to do the inital gitea setup via http://localhost:3000/ (hint: use db as the hostname and root/gitea as username/password for mysql DB configuration) and then you're ready to go.
  4. Stop the containers with CRTL+C if running in the foreground and with docker-compose down if running in the background.
  5. Clean up your system (after your tests) with docker system prune -a (deletes everything) and docker volume rm <volume_name> (list volumes with docker volume ls). Important: these instructions asumes that you're running no other docker containers and completely cleans up your system, so be careful.

It would be nice if you can confirm:

  1. example repository is correct for testing the relevant things (relative md links)
  2. Version 1.11.0 is working fine

Thanks a lot :)

@Mo-Gul
Copy link

Mo-Gul commented Oct 14, 2023

I don't have much of a clue about docker, so I trust whatever you are doing. But maybe a bit simpler test is
markdown_test-main.zip where there is only one README.md in the docs folder and the link should work when you are in the base repository directory.

Good luck finding the Gitea version where the link isn't working (properly) any more!!

@deparcus
Copy link
Author

I did some more testing, but still got no real result.

First, I found out that binary testing works not that good when descreasing versions. So everytime the database had to be cleared and initial setup and repo creation has to be done.

So I first tested:

  • gitea 1.11.0 with mysql 8 --> relative md links work fine
  • gitea 1.5.0 with mysql 5 (with mysql 8 I got an error during initial db setup) --> relative md links work fine
  • gitea 1.0.0 with mysql 5 --> relative md links work fine

Then I decided just to go versions up and test every .0 version.

What I did:

  • gitea 1.[0-20].0 with mysql 5
  • repository with testfiles as uploaded in my previous comment
  • in every step open gitea in new incognito window (to exclude some caching effects)
  • click on link in README.md from the repo root
  • click on link in the README.md in docs folder
  • verify that linked *.md-file is shown

Result:

  • verything worked as expected

So to be honest I do not know how to proceed here. Probably I'm testing something wrong (that's why I asked for a doublecheck ;-) ).

I think it might be more efficient to search in the code for the hint how to handle asciidoc links similar to md links.

Sorry for not getting a better result here.

But btw: I was really interesting how gitea look, features and console log output changed from the beginning to the latest version :-) Due to docker the test with version switch (when going up) can be done really quickly.

@Mo-Gul
Copy link

Mo-Gul commented Oct 22, 2023

Since I currently can't setup (easily) other Gitea's myself and I trust your results, I agree that this maybe is a dead end.
I also already tried to find the related code parts where links are handled but I most likely I didn't succeed because I don't know anything about the (programming) language.
So hopefully you have more luck than I do ...

@deparcus
Copy link
Author

deparcus commented Oct 27, 2023

Sorry, me neither, because I also don't know the programming language.
Any other way to proceed here? Probably someone with the right skills should have a look.

@Mo-Gul
Copy link

Mo-Gul commented Oct 27, 2023

That is the obvious way, but I have not much hope here, because up to now nobody else than jtiee commented here. I'd be happy to proven wrong.
Unfortunately the same seems to be true for the asciidoctor chat ... 😢

@deparcus
Copy link
Author

deparcus commented Nov 1, 2023

The bug is that Gitea does not add a trailing slash in links to repos/folders.

Consider this situation:

The user views https://<gitea-host>/<user>/<repository>. Gitea automatically renders the README file (README.adoc in this case, but any format would suffer the same problem).

Relative links in the auto-rendered README might look like docs/index.adoc. When you click such a link, the browser combines the current URL context with the relative link. The combination uses the existing "folder" (path segment) in the current URL as the context.

What "folder" is that? https://<gitea-host>/<user> because there is no trailing slash after <repository>. So, the browser then attempts to fetch https://<gitea-host>/<user>/docs/index.adoc, which results in a 404.

If Gitea adds the trailing slash to repo/folder links, then when the user clicks the docs/index.adoc link, the browser would fetch https://<gitea-host>/<user>/<repository>/docs/index.adoc, which is correct.

You can manually add a trailing slash (and reload the page) to any repo/folder view and click a relative link in an auto-generated README file to confirm the correct behavior.

I used gitea-1.20.3-darwin-10.12-amd64 for testing.

Sorry @jtiee , I did not see, that you were the one writing in AsciiDoctor Chat.
Are you sure, that it's really a gitea bug? I'm asking because on my side links in Markdown file are working fine and links in AsciiDoc files not. So why should gitea add the trailing slashes in Markdown but not in AsciiDoc e.g.?
Can you help here?

@jtiee
Copy link

jtiee commented Nov 2, 2023

I've updated my PoC so that I have AsciiDoc and Markdown files with the same content:

$ ls -la
total 16
drwxr-xr-x   6 eedwards  staff  192  2 Nov 09:38 ./
drwxr-xr-x   3 eedwards  staff   96 25 Aug 18:12 ../
drwxr-xr-x  13 eedwards  staff  416  2 Nov 09:42 .git/
-rw-r--r--   1 eedwards  staff   69 25 Aug 18:15 README.adoc
-rw-r--r--   1 eedwards  staff   67  2 Nov 09:38 README.md
drwxr-xr-x  12 eedwards  staff  384  2 Nov 09:41 docs/
$ ls -la docs
total 80
drwxr-xr-x  12 eedwards  staff  384  2 Nov 09:41 ./
drwxr-xr-x   6 eedwards  staff  192  2 Nov 09:38 ../
-rw-r--r--   1 eedwards  staff   79  2 Nov 09:38 READ.md
-rw-r--r--   1 eedwards  staff   81 25 Aug 18:29 README.adoc
-rw-r--r--   1 eedwards  staff   30 25 Aug 18:14 development.adoc
-rw-r--r--   1 eedwards  staff   33  2 Nov 09:39 development.md
-rw-r--r--   1 eedwards  staff  139 25 Aug 18:15 index.adoc
-rw-r--r--   1 eedwards  staff  129  2 Nov 09:39 index.md
-rw-r--r--   1 eedwards  staff   31 25 Aug 18:14 install.adoc
-rw-r--r--   1 eedwards  staff   34  2 Nov 09:39 install.md
-rw-r--r--   1 eedwards  staff   26 25 Aug 18:14 support.adoc
-rw-r--r--   1 eedwards  staff   29  2 Nov 09:39 support.md

I used docs/READ.md instead of docs/README.md to avoid auto-rendering the Markdown file.

Then I compared the links rendered by Gitea in docs/index.adoc and docs/index.md. First, here is the markup for the links:

docs/index.adoc:

* link:install.adoc[Installation]

docs/index.md:

- [Installation](install.md)

As you can see, both implement relative links. When Gitea renders these:

docs/index.adoc:

<a href="install.adoc" rel="nofollow">Installation</a>

docs/index.md:

<a href="/eskwayrd/test/src/branch/main/docs/install.md" rel="nofollow">Installation</a>

As you can see, the rendered Asciidoc files have the relative links as defined in the file. Whereas the Markdown files have an absolute link from the document root. That's the problem.

I don't know if Gitea is making the rendered Markdown links absolute or whether the Markdown renderer used by Gitea is doing that. But nothing is making the AsciiDoc links absolute.

Either Gitea needs to add slashes after folder names, or it needs to make relative links absolute in rendered AsciiDoc files. Adding the slashes after folder names would avoid this issue for any other auto-rendered files. And there's no reason both solutions couldn't be applied.

@lunny
Copy link
Member

lunny commented Nov 3, 2023

There are two envs to help with your asciidoc command. GITEA_PREFIX_SRC and GITEA_PREFIX_RAW could help you to make the links absolute.

About how to use, please see https://docs.gitea.com/next/administration/config-cheat-sheet?_highlight=gitea_prefix_src#markup-markup

@Mo-Gul
Copy link

Mo-Gul commented Dec 5, 2023

@lunny, I have stumbled over these variables at different places so far, e.g. also at #6359 (comment), but unfortunately I couldn't find out how exactly to use these (with a concrete example).

I just gave it some tries with

RENDER_COMMAND="asciidoctor --base-dir=$GITEA_PREFIX_SRC --no-header-footer --attribute=showtitle --out-file=- -"
RENDER_COMMAND="asciidoctor --base-dir=GITEA_PREFIX_SRC --no-header-footer --attribute=showtitle --out-file=- -"
RENDER_COMMAND="asciidoctor --attribute docdir=$GITEA_PREFIX_SRC --no-header-footer --attribute=showtitle --out-file=- -"

but unfortunately none of them worked. What am I doing wrong it these environment variables (really) should be able to solve the problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants