Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Git external sources #5

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

mateusrodrigues
Copy link
Member

This PR adds the external sources functionality to Flamenco as described in issue #3.

At first, this is an implementation for Git external sources. However, I tried to make this implementation extensible enough for us to be easily able to add more external source types, such as tarballs.

The external source is described in an INI-style configuration file defined as a "Flamenco file", which does not make it to the final dist result and is called flamenco.external.{source_package}.{release}. It describes the external source where to pull files from in the following format:

type=git
repo=<url>
commit=<commit>
tag=<tag>
branch=<branch>

The properties commit, tag, and branch are mutually exclusive, where commit takes precedence over tag, which takes precedence over branch. If none of these are defined, the behavior is to checkout the repository's default branch.

Closes #3.

This commit adds Git external source support with `GitExternalSource`. The implementation is made extensible by adding an `ExternalSourceBase` as an abstract base class that creates external source objects based on the type parameter of the source descriptors.

The git manipulation code takes a dependency on the `LibGit2Sharp` NuGet package.
@mateusrodrigues mateusrodrigues added the enhancement New feature or request label Aug 20, 2024
Copy link
Collaborator

@dviererbe dviererbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is not urgent, I will do a full review next pulse. I left some comments based on the things I noticed on a first look.

Comment on lines +14 to +28
var fileContentLines = File.ReadAllLines(descriptorFilePath);
var type = fileContentLines.FirstOrDefault(l => l.StartsWith("type"))?.Split("=").Last();

if (type is null)
{
throw new ApplicationException($"{descriptorFilePath} is missing a type.");
}

switch (type)
{
case "git":
var repositoryUrl = fileContentLines.FirstOrDefault(l => l.StartsWith("repo"))?.Split('=').Last();
var branch = fileContentLines.FirstOrDefault(l => l.StartsWith("branch"))?.Split('=').Last();
var tag = fileContentLines.FirstOrDefault(l => l.StartsWith("tag"))?.Split('=').Last();
var commit = fileContentLines.FirstOrDefault(l => l.StartsWith("commit"))?.Split('=').Last();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Why do you use a custom file format instead of established formats like json, yaml or even xml?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's an INI-style file. I aimed at a less verbose, to-the-point, and easily parse-able format. JSON is an option, although INI is even less verbose than that. But I don't think YAML or even XML are suitable, due to their unfriendliness.


// Reference precedence is commit, then tag, then branch. If all of them are null, the default
// behavior is to check out the repo's default branch.
return new GitExternalSource(new Uri(repositoryUrl), commit ?? (tag ?? branch));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Why do you differentiate between commit, tag and branch? For git all three are a commit-ish that git can resolve. Especially, because there is no logic that would block someone from specifying a branch-name as a tag or commit.

Copy link
Member Author

@mateusrodrigues mateusrodrigues Aug 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For readability's sake. At the end of the day, whatever you input will end up in the same git checkout parameter, so there is no intrinsic distinction among the three. However, the file will make it obvious to the reader that you're specifying a branch, tag, or commit.

We can, however, change the property name to something like reference and let the user input a branch, tag, or commit (but this needs to be made obvious in the file's documentation).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would favor using the git terminology commitish. This is well established, but reference also works and is commonly used.

return false;
switch (fileName)
{
case "flamenco.external":
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: This is a constant value that is defined here and in NamingConventions.ValidFlamencoFileNames. This is an anti-pattern, because someone could edit just the value in one place and forget to edit the other.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done ✅

This commit adds the external source functionality to the build debian-tarball command, as well as the file name expected.

It is generally defined that a Flamenco descriptor file, such as the external source descriptor file, is a "flamenco file".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for referenced Git repositories
2 participants