Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#244] Symlink scanner #262

Merged
merged 1 commit into from
Feb 1, 2023
Merged

[#244] Symlink scanner #262

merged 1 commit into from
Feb 1, 2023

Conversation

aeqz
Copy link
Contributor

@aeqz aeqz commented Dec 30, 2022

Description

Problem: as GitHub and GitLab do not render symlinks as the file they point to, and we have disabled the markdown scanner to process them in #261, we are also considering to implement a new scanner for symlinks that verifies them up to some extent.

Solution: in this proposal, a scanner that validates the reference from a symlink has been implemented in the same style as the markdown scanner.

Related issue(s)

Fixes #244

✅ Checklist for your Pull Request

Ideally a PR has all of the checkmarks set.

If something in this list is irrelevant to your PR, you should still set this
checkmark indicating that you are sure it is dealt with (be that by irrelevance).

Related changes (conditional)

  • Tests

    • If I added new functionality, I added tests covering it.
    • If I fixed a bug, I added a regression test to prevent the bug from
      silently reappearing again.
  • Documentation

    • I checked whether I should update the docs and did so if necessary:
  • Public contracts

    • Any modifications of public contracts comply with the Evolution
      of Public Contracts
      policy.
    • I added an entry to the changelog if my changes are visible to the users
      and
    • provided a migration guide for breaking changes if possible

Stylistic guide (mandatory)

@aeqz
Copy link
Contributor Author

aeqz commented Dec 30, 2022

I have created this draft PR to discuss a possible implementation for the symlink scanner and see possible problems. A couple of details observed for the moment:

  • Windows CI is failing not because of the symlinks, but because '/' vs '\' in path outputs. I will review it.
  • The danger checks process is failing because of a broken symlink that I created for testing.

-- | Way to parse a file.
type ScanAction = CanonicalPath -> IO (FileInfo, [ScanError 'Parse])

-- | All supported ways to parse a file.
type FormatsSupport = Extension -> Maybe ScanAction
type FileSupport = CanonicalPath -> IO $ Maybe ScanAction

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of determining which scanner to use just from the file extension, I changed it to a more general check. Another option is to put the symlink scanner in a 'different level' that is above extension check: if it is a symlink, then symlink scanner, else get scanner from extension.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about the type Bool -> Extension -> Maybe ScanAction, where the first bool denotes isLink?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it. It is less general and can be better understood 👍

- reference (relative) :
- text: ""
- link: dir/b.md
- anchor: -
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is how a symlink reference looks like by using the plain system that we already have. It can be improved to show in some way that s.md is a symlink that points to dir/b.md.

src/Xrefcheck/Core.hs Outdated Show resolved Hide resolved
@aeqz aeqz requested a review from Martoon-00 January 10, 2023 15:47
@aeqz aeqz force-pushed the aeqz/#242-no-scan-symlinks-as-md branch 2 times, most recently from 066da26 to 1431587 Compare January 12, 2023 19:21
@aeqz aeqz mentioned this pull request Jan 12, 2023
8 tasks
@aeqz aeqz force-pushed the aeqz/#244-symlink-scanner branch from d9f2a01 to 8cf4d34 Compare January 13, 2023 11:48
@aeqz aeqz force-pushed the aeqz/#242-no-scan-symlinks-as-md branch 2 times, most recently from e45ebd8 to fef5153 Compare January 18, 2023 12:29
@aeqz aeqz force-pushed the aeqz/#244-symlink-scanner branch from 8cf4d34 to e6ae598 Compare January 18, 2023 12:30
Base automatically changed from aeqz/#242-no-scan-symlinks-as-md to master January 19, 2023 17:55
Copy link
Member

@Martoon-00 Martoon-00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I like how the new scanner addition is handled.

Leaving a few comments. And also would eventually like to see how it looks after rebase over #263.

src/Xrefcheck/Scanners/Symlink.hs Show resolved Hide resolved
src/Xrefcheck/Core.hs Outdated Show resolved Hide resolved
src/Xrefcheck/Scan.hs Outdated Show resolved Hide resolved
@aeqz aeqz force-pushed the aeqz/#244-symlink-scanner branch from e6ae598 to 84fcc86 Compare January 25, 2023 14:48
@aeqz aeqz requested a review from Martoon-00 January 25, 2023 15:08
@aeqz aeqz force-pushed the aeqz/#244-symlink-scanner branch from ffbf20f to 63c9e28 Compare January 30, 2023 10:04
]
firstFileSupport :: [FileSupport] -> FileSupport
firstFileSupport fs isSymlink =
safeHead . catMaybes <$> traverse ($ isSymlink) fs
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps instead of taking the first FileSupport that handles the file, we can join all the results of scanning the file with all the FileSupports that can handle it, although currently we only have Markdown and Symlink scanners, which are exclusive.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That farsighted фтв actually makes sense. However I think let's leave it to change to people who will add other scanners.

@aeqz aeqz requested a review from Martoon-00 January 30, 2023 16:31
Copy link
Member

@Martoon-00 Martoon-00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks 👍

Feel free to rebase and merge when you have time.

@aeqz aeqz marked this pull request as ready for review February 1, 2023 12:02
@aeqz aeqz force-pushed the aeqz/#244-symlink-scanner branch from 63c9e28 to 643adf9 Compare February 1, 2023 12:04
@serokell serokell deleted a comment from Martoon-00 Feb 1, 2023
Problem: As GitHub and GitLab do not render symlinks as the file they
point to, we are considering to implement a new scanner for symlinks
that verifies them up to some extent.

Solution: A scanner that validates the reference from a symlink has been
implemented in the same style as the markdown scanner.
@aeqz aeqz force-pushed the aeqz/#244-symlink-scanner branch from 643adf9 to 9421c42 Compare February 1, 2023 12:07
@aeqz aeqz merged commit d50a8e7 into master Feb 1, 2023
@aeqz aeqz deleted the aeqz/#244-symlink-scanner branch February 1, 2023 12:25
@int-index int-index mentioned this pull request Dec 27, 2024
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Review how symlinks are handled
3 participants