-
Notifications
You must be signed in to change notification settings - Fork 121
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature: GitWeb Source Link provider #505
Conversation
Cools. Thanks for the PR, much appreciated! We are currently working on releasing first stable version of SourceLink packages, so I'm gonna wait with merging this until after we finish that process. |
Sounds good. Is there an estimate on how long that will take? |
A couple of days :) |
src/SourceLink.GitWeb/buildMultiTargeting/SourceLink.GitWeb.targets
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, just a couple of issues.
It'd also be useful to have integration tests - they validate that the package works end-to-end. See https://github.com/dotnet/sourcelink/blob/master/src/SourceLink.Git.IntegrationTests/GitLabTests.cs
I'm looking into integration tests now. Will be a separate commit than the above review comments. Edit:
Furthermore, at least in our case, we always use the ssh URL for git operations(not sure if gitweb supports anything but ssh?). So should I also be modifying TranslateRepositoryUrls to keep the ssh url in those properties? I understand source link needs an HTTP address for the content url. |
@Glen-Nicol-Garmin Will you be able to write the integration test? I can help if needed. Re PrivateRepositoryUrl - See https://github.com/dotnet/sourcelink/tree/master/docs#publishrepositoryurl |
Re SSH - the default implementation is provided by TranslateRepositoryUrlsGitTask, from which TranslateRepositoryUrls derives. If the default implementation works then TranslateRepositoryUrls can be empty. AzureDevOpsServer uses a different structure for SSH URL than it uses for HTTP URL, so it needs to override: |
I can do the work for the tests but I am still a bit confused on how those properties should be set. Sounds like in this case they should both be left as SSH urls? I gather that because from my interpretation those properties are supposed to be the URL for the source control software to do a clone/checkout and gitweb only does SSH (as far as I can tell). |
They need to be translated to |
When I read the nuspec docs I interpret that property as the clone URL.
Which in our case is the SSH URL. We can't clone from a https URL. If SourceLink requires it to be a HTTPS url then fine, but we should add documentation for the divergence. |
Can you elaborate on why it is not possible to clone using https URL? |
Our git server only supports the SSH protocol that I am aware of. When you go to a gitweb page it lists the SSH url to clone it. |
That seems odd. Have you tried the https version of the URI? Source Link needs HTTPs support for at least fetching individual raw source files. It would be surprising if you could do that but couldn't clone the repo using HTTPs. |
I thought the point of source link was to translate between repo Urls to content urls where the files can be found and downloaded? We have content over HTTPS. And no simply changing the protocol of the URI does not work. |
That is correct. It's just surprising that you wouldn't have HTTPS URL that would work for clone. I have not seen that before -- GitHub, Azure DevOps, GitLab, BitBucket all allow both HTTPS and SSH (and also git protocol). I guess GitWeb can be configured to only support SSH? If that's the case we can either skip the translation completely by not setting |
BTW, the reason we translate SSH and git protocols to HTTPS is to normalize the build results. You can imagine different build environments choosing different protocols for cloning the repo. The result of the build from all these environments will be the same, regardless of how they cloned the repo. |
How about merging this PR and leaving the integration tests and SSH translation for a follow up? |
I think I need to wrap my head around the SSH translation before I really want to consider it done and ready for consumption. I should be able to get to that next week. |
BTW, I'm going to merge #546 which might cause your PR to fail build due to missing nullable annotations. Should be easy to add though. |
So funny story, I tried removing the translation step altogether (commented out the using Task, the property you mentioned, and the target) still got https URLS. And then I tried overriding all those functions in TranslateRepositoryUrlsGitTask. Neither works in the integration tests. I am thinking I may not have something setup right for those target files. |
Any leads or ideas on why changing those files isn't affecting the integration tests? |
I was able to figure it out. Did not know the integration tests rely on the nuget packages which are not built when you build the solution. |
Oh, sorry about that! Yes, the build doesn't run |
No worries, would you like me to rebase onto master before you review the latest changes? |
@Glen-Nicol-Garmin Yes, please rebase. |
18b13db
to
f8163a0
Compare
GitWeb only currently supports SSH repository URLs. dotnet#505
K, done. You should look at the changes I made to the common translation Task to facilitate checking for supported protocols. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🕐
GitWeb only currently supports SSH repository URLs. dotnet#505
f8163a0
to
1b424fc
Compare
Also, as an aside maybe this belongs in a separate issue, whenever I run the tests an entry like this is added to my user PATH variable:
These are never cleaned up. And it actually ended up breaking a few things on my system in a subtle way. For instance, my user TEMP directory wouldn't get set so all my programs would try to write to the system TEMP directory at C:\windows\TEMP. Which usually results in a Access denied error. It also ended up causing my PATH to not be completely evaluated so things like code wouldn't run from the command line. I guess with enough entries it overflows some max string limit in Windows and there is no error dialog or indicator anywhere when this happens. |
@Glen-Nicol-Garmin I have pushed a commit that removes Designer.cs file. We use a custom target to generate C# source from resx file that does not need a checked-in file. |
{ | ||
// NOTE: i don't know what the spec parameter is for. but for all the other | ||
// tests like this for various providers it is the domain of the Repo URL. | ||
new MockItem("src.intranet.company.com", KVP("ContentUrl", "https://src.intranet.company.com/gitweb" + s2)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
new MockItem("src.intranet.company.com" [](start = 20, length = 39)
It is the host domain that needs to match the domain in the SourceRoot.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Glen-Nicol-Garmin Thanks for the contribution! |
Huzzah! That is great news! Before I forget, should I create an issue for that PATH bug in tests I mentioned? |
@Glen-Nicol-Garmin No need. This should take care of it: #578. |
@Glen-Nicol-Garmin The package is now available in the Azure feed: Let me know if it works in a real-world project. Once you confirm I'll add a mention to Readme.md |
Just tested it out. It works! |
{ | ||
Log.LogError(e.Message); | ||
return; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it ok to just log and return? I'm thinking that the error might be observer only a few week/months later when one debugs the code and sees that source linking doesn't work anymore.
Why not let the exception propagate?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am fairly hazy on the specifics now, but I think logging the error results in an MsBuild error that can't be suppressed. Is the concern that since it is a post build action that error may not be noticed since the files will be on disk? I think the exception being thrown would have the same limitation.
I don't remember the specific use case where the NotSupportedException was thrown to say whether or not the message is helpful enough to tell the user that their URL scheme/format is not supported by Sourcelink. But I thought those were the errors being thrown by the individual translators when they didn't support the repo url.
…11) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [Microsoft.SourceLink.GitHub](https://github.com/dotnet/sourcelink) | `1.1.0` -> `1.1.1` | [![age](https://developer.mend.io/api/mc/badges/age/nuget/Microsoft.SourceLink.GitHub/1.1.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/nuget/Microsoft.SourceLink.GitHub/1.1.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/nuget/Microsoft.SourceLink.GitHub/1.1.0/1.1.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/nuget/Microsoft.SourceLink.GitHub/1.1.0/1.1.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>dotnet/sourcelink (Microsoft.SourceLink.GitHub)</summary> ### [`v1.1.1`](https://github.com/dotnet/sourcelink/releases/tag/1.1.1) [Compare Source](https://github.com/dotnet/sourcelink/compare/1.1.0...1.1.1) #### Notable Changes - Feature: GitWeb Source Link provider by [@​Glen-Nicol-Garmin](https://github.com/Glen-Nicol-Garmin) in [dotnet/sourcelink#505 - Use submodule.name.url to determine the URL of a submodule by [@​tmat](https://github.com/tmat) in [dotnet/sourcelink#587 - Add Microsoft.SourceLink.Tools source package implementing SourceLinkMap by [@​tmat](https://github.com/tmat) in [dotnet/sourcelink#663 - Added support for gitea by [@​Mik4sa](https://github.com/Mik4sa) in [dotnet/sourcelink#674 - Enable source-build with Arcade SDK fixes by [@​dagood](https://github.com/dagood) in [dotnet/sourcelink#692 - Add netstandard2.0 target to tools package by [@​tmat](https://github.com/tmat) in [dotnet/sourcelink#702 - Update license to MIT by [@​tmat](https://github.com/tmat) in [dotnet/sourcelink#730 - Allow using .git directory instead of gitdir redirect in submodules. by [@​crummel](https://github.com/crummel) in [dotnet/sourcelink#653 - Fix discovery of working directory for worktrees by [@​tmat](https://github.com/tmat) in [dotnet/sourcelink#734 - Add support for the new GitLab raw url by [@​rgl](https://github.com/rgl) in [dotnet/sourcelink#713 - Target netcoreapp3.1 by [@​tmat](https://github.com/tmat) in [dotnet/sourcelink#767 #### New Contributors - [@​Glen-Nicol-Garmin](https://github.com/Glen-Nicol-Garmin) made their first contribution in [dotnet/sourcelink#505 - [@​IgorKustov-ChathamFinancial](https://github.com/IgorKustov-ChathamFinancial) made their first contribution in [dotnet/sourcelink#592 - [@​v-chmart](https://github.com/v-chmart) made their first contribution in [dotnet/sourcelink#604 - [@​0xced](https://github.com/0xced) made their first contribution in [dotnet/sourcelink#668 - [@​nycdotnet](https://github.com/nycdotnet) made their first contribution in [dotnet/sourcelink#672 - [@​Mik4sa](https://github.com/Mik4sa) made their first contribution in [dotnet/sourcelink#674 - [@​rgl](https://github.com/rgl) made their first contribution in [dotnet/sourcelink#713 - [@​adiaaida](https://github.com/adiaaida) made their first contribution in [dotnet/sourcelink#737 **Full Changelog**: dotnet/sourcelink@1.0.0...1.1.1 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/apollographql/federation-hotchocolate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy44LjEiLCJ1cGRhdGVkSW5WZXIiOiIzNy44LjEiLCJ0YXJnZXRCcmFuY2giOiJtYWluIn0=--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Hi there, I work for Garmin and we use an internal GitWeb server that doesn't work with any of the existing providers. I created one that should work for many other companies in a similar situation that don't use cloud providers.
I saw in #239 that the suggested solution for a provider that is not currently supported is to copy and edit the GitLab ones. So that is what I did.