Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CIAPP-5371] validate git tags #15

Merged
merged 33 commits into from
Sep 6, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
0f1490f
move git constants here from dd-trace-rb
anmarchenko Sep 1, 2023
b5c77ba
start environment refactoring with appveyor provider
anmarchenko Sep 1, 2023
895ee04
extract Azure environment extractor to its own class
anmarchenko Sep 1, 2023
1f70294
extract bitbucket extractor to its own class
anmarchenko Sep 1, 2023
d01f763
reshuffle code locations, extract Buddy extractor to a separate class
anmarchenko Sep 1, 2023
ee39589
extract buildkite extractor to a separate class
anmarchenko Sep 1, 2023
ba85509
finish extracting buildkite
anmarchenko Sep 1, 2023
c6d0401
fix linting and better naming and typing
anmarchenko Sep 1, 2023
852bd81
extract Circle CI to a separate extractor class
anmarchenko Sep 4, 2023
d7ec84d
extract GithubActions provider
anmarchenko Sep 4, 2023
91943fb
extract gitlab CI
anmarchenko Sep 4, 2023
7a39ffe
extract Jenkins
anmarchenko Sep 4, 2023
67ca5fc
extract teamcity
anmarchenko Sep 4, 2023
3b898c1
extract travis
anmarchenko Sep 4, 2023
ca3b200
extract bitrise
anmarchenko Sep 4, 2023
c7a2845
extract codefresh
anmarchenko Sep 4, 2023
d2a1035
pull up branch or tag logic for extractors
anmarchenko Sep 4, 2023
f3be086
extract user defined tags extractor
anmarchenko Sep 4, 2023
fd3ed19
Extract local git extractor and move normalization into extractor
anmarchenko Sep 4, 2023
57c5809
validate git tags
anmarchenko Sep 4, 2023
e3e0900
add @tags class var and remove some nil checks
anmarchenko Sep 4, 2023
ad72ac6
refactor commit users for local_git extractor
anmarchenko Sep 4, 2023
13fa56d
providers specs
anmarchenko Sep 5, 2023
fe7573d
fix lint
anmarchenko Sep 5, 2023
0b57a4c
move fixtures to support, add specs for LocalGit and UserDefinedTags
anmarchenko Sep 5, 2023
ba4bbba
ruby 2.7 does not have Hash#except
anmarchenko Sep 5, 2023
99b2e08
fix lint
anmarchenko Sep 5, 2023
d94a26a
show alternative code organization around provider and extractor
GustavoCaso Sep 6, 2023
214f174
Merge branch 'main' into anmarchenko/ci_vis_validate_git_tags
anmarchenko Sep 6, 2023
2487504
Merge pull request #18 from DataDog/move-provider-outside-extractor
anmarchenko Sep 6, 2023
7066de3
fixing tests, extracting git utility to Utils::Git module, fix signat…
anmarchenko Sep 6, 2023
b600726
spec for Utils::Git, frozen string literals
anmarchenko Sep 6, 2023
0837f35
supply provider class to Extractor
anmarchenko Sep 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions lib/datadog/ci/ext/environment.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

require_relative "git"
require_relative "environment/extractor"
require_relative "environment/providers/local_git"
require_relative "environment/providers/user_defined_tags"
anmarchenko marked this conversation as resolved.
Show resolved Hide resolved

module Datadog
module CI
Expand Down Expand Up @@ -31,11 +33,11 @@ def tags(env)

# If user defined metadata is defined, overwrite
tags.merge!(
Environment::Extractor.new(env, provider: Providers::UserDefinedTags).tags
Environment::Extractor.new(env, provider: Providers::UserDefinedTags.new(env)).tags
anmarchenko marked this conversation as resolved.
Show resolved Hide resolved
)

# Fill out tags from local git as fallback
local_git_tags = Environment::Extractor.new(env, provider: Providers::LocalGit).tags
local_git_tags = Environment::Extractor.new(env, provider: Providers::LocalGit.new(env)).tags
local_git_tags.each do |key, value|
tags[key] ||= value
end
Expand Down
20 changes: 4 additions & 16 deletions lib/datadog/ci/ext/environment/extractor.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

require_relative "../environment"
require_relative "../git"
require_relative "../../utils/git"
require_relative "providers"

module Datadog
Expand Down Expand Up @@ -68,13 +69,13 @@ def tags

def normalize_git!
branch_ref = @tags[Git::TAG_BRANCH]
if is_git_tag?(branch_ref)
if Datadog::CI::Utils::Git.is_git_tag?(branch_ref)
@tags[Git::TAG_TAG] = branch_ref
@tags.delete(Git::TAG_BRANCH)
end

@tags[Git::TAG_TAG] = normalize_ref(@tags[Git::TAG_TAG])
@tags[Git::TAG_BRANCH] = normalize_ref(@tags[Git::TAG_BRANCH])
@tags[Git::TAG_TAG] = Datadog::CI::Utils::Git.normalize_ref(@tags[Git::TAG_TAG])
@tags[Git::TAG_BRANCH] = Datadog::CI::Utils::Git.normalize_ref(@tags[Git::TAG_BRANCH])
@tags[Git::TAG_REPOSITORY_URL] = filter_sensitive_info(
@tags[Git::TAG_REPOSITORY_URL]
)
Expand All @@ -88,19 +89,6 @@ def expand_workspace!
end
end

def is_git_tag?(ref)
!ref.nil? && ref.include?("tags/")
end

def normalize_ref(name)
return nil if name.nil?

refs = %r{^refs/(heads/)?}
origin = %r{^origin/}
tags = %r{^tags/}
name.gsub(refs, "").gsub(origin, "").gsub(tags, "")
end

def filter_sensitive_info(url)
return nil if url.nil?

Expand Down
4 changes: 2 additions & 2 deletions lib/datadog/ci/ext/environment/providers.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# frozen_string_literal: true

require_relative "providers/default"
require_relative "providers/base"
require_relative "providers/appveyor"
require_relative "providers/azure"
require_relative "providers/bitbucket"
Expand Down Expand Up @@ -38,7 +38,7 @@ module Providers

def self.for_environment(env)
_, provider_klass = PROVIDERS.find { |provider_env_var, _| env.key?(provider_env_var) }
provider_klass = Providers::Default if provider_klass.nil?
provider_klass = Providers::Base if provider_klass.nil?

provider_klass.new(env)
end
Expand Down
5 changes: 1 addition & 4 deletions lib/datadog/ci/ext/environment/providers/appveyor.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,6 @@ module Providers
# Appveyor: https://www.appveyor.com/
# Environment variables docs: https://www.appveyor.com/docs/environment-variables/
class Appveyor < Base
private

# overridden methods
def provider_name
"appveyor"
end
Expand Down Expand Up @@ -82,7 +79,7 @@ def git_commit_message
commit_message
end

# appveyor-specific methods
private

def github_repo_provider?
return @github_repo_provider if defined?(@github_repo_provider)
Expand Down
5 changes: 1 addition & 4 deletions lib/datadog/ci/ext/environment/providers/azure.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,6 @@ module Providers
# Azure Pipelines: https://azure.microsoft.com/en-us/products/devops/pipelines
# Environment variables docs: https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml
class Azure < Base
private

# overridden methods
def provider_name
"azurepipelines"
end
Expand Down Expand Up @@ -87,7 +84,7 @@ def ci_env_vars
}.to_json
end

# azure-specific methods
private

def build_id
env["BUILD_BUILDID"]
Expand Down
3 changes: 3 additions & 0 deletions lib/datadog/ci/ext/environment/providers/base.rb
Original file line number Diff line number Diff line change
Expand Up @@ -90,10 +90,13 @@ def git_commit_message
def git_commit_sha
end

private

def set_branch_and_tag
branch_or_tag_string = git_branch_or_tag
@branch = @tag = nil

# @type var branch_or_tag_string: untyped
if branch_or_tag_string && branch_or_tag_string.include?("tags/")
@tag = branch_or_tag_string
else
Expand Down
4 changes: 1 addition & 3 deletions lib/datadog/ci/ext/environment/providers/bitbucket.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@ module Providers
# Bitbucket Pipelines: https://bitbucket.org/product/features/pipelines
# Environment variables docs: https://support.atlassian.com/bitbucket-cloud/docs/variables-and-secrets/
class Bitbucket < Base
private

# overridden methods
def provider_name
"bitbucket"
Expand Down Expand Up @@ -57,7 +55,7 @@ def git_tag
env["BITBUCKET_TAG"]
end

# bitbucket-specific methods
private

def url
"https://bitbucket.org/#{env["BITBUCKET_REPO_FULL_NAME"]}/addon/pipelines/home#" \
Expand Down
3 changes: 0 additions & 3 deletions lib/datadog/ci/ext/environment/providers/bitrise.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,6 @@ module Providers
# Bitrise: https://bitrise.io/
# Environment variables docs: https://devcenter.bitrise.io/en/references/available-environment-variables.html
class Bitrise < Base
private

# overridden methods
def provider_name
"bitrise"
end
Expand Down
3 changes: 0 additions & 3 deletions lib/datadog/ci/ext/environment/providers/buddy.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,6 @@ module Providers
# Buddy: https://buddy.works/
# Environment variables docs: https://buddy.works/docs/pipelines/environment-variables
class Buddy < Base
private

# overridden methods
def provider_name
"buddy"
end
Expand Down
3 changes: 0 additions & 3 deletions lib/datadog/ci/ext/environment/providers/buildkite.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,6 @@ module Providers
# Buildkite: https://buildkite.com/
# Environment variables docs: https://buildkite.com/docs/pipelines/environment-variables
class Buildkite < Base
private

# overridden methods
def provider_name
"buildkite"
end
Expand Down
3 changes: 0 additions & 3 deletions lib/datadog/ci/ext/environment/providers/circleci.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,6 @@ module Providers
# Circle CI: https://circleci.com/
# Environment variables docs: https://circleci.com/docs/variables/#built-in-environment-variables
class Circleci < Base
private

# overridden methods
def provider_name
"circleci"
end
Expand Down
3 changes: 0 additions & 3 deletions lib/datadog/ci/ext/environment/providers/codefresh.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,6 @@ module Providers
# Codefresh: https://codefresh.io/
# Environment variables docs: https://codefresh.io/docs/docs/pipelines/variables/#export-variables-to-all-steps-with-cf_export
class Codefresh < Base
private

# overridden methods
def provider_name
"codefresh"
end
Expand Down
20 changes: 0 additions & 20 deletions lib/datadog/ci/ext/environment/providers/default.rb

This file was deleted.

3 changes: 0 additions & 3 deletions lib/datadog/ci/ext/environment/providers/github_actions.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,6 @@ module Providers
# Github Actions: https://github.com/features/actions
# Environment variables docs: https://docs.github.com/en/actions/learn-github-actions/variables#default-environment-variables
class GithubActions < Base
private

# overridden methods
def provider_name
"github"
end
Expand Down
5 changes: 1 addition & 4 deletions lib/datadog/ci/ext/environment/providers/gitlab.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,6 @@ module Providers
# Gitlab CI: https://docs.gitlab.com/ee/ci/
# Environment variables docs: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
class Gitlab < Base
private

# overridden methods
def provider_name
"gitlab"
end
Expand Down Expand Up @@ -99,7 +96,7 @@ def ci_env_vars
}.to_json
end

# gitlab-specific methods
private

def extract_name_email
return @name_email_tuple if defined?(@name_email_tuple)
Expand Down
6 changes: 2 additions & 4 deletions lib/datadog/ci/ext/environment/providers/jenkins.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
require "json"

require_relative "base"
require_relative "../../../utils/git"

module Datadog
module CI
Expand All @@ -12,9 +13,6 @@ module Providers
# Jenkins: https://www.jenkins.io/
# Environment variables docs: https://www.jenkins.io/doc/book/pipeline/jenkinsfile/#using-environment-variables
class Jenkins < Base
private

# overridden methods
def provider_name
"jenkins"
end
Expand All @@ -25,7 +23,7 @@ def pipeline_id

def pipeline_name
if (name = env["JOB_NAME"])
name = name.gsub("/#{normalize_ref(git_branch)}", "") if git_branch
name = name.gsub("/#{Datadog::CI::Utils::Git.normalize_ref(git_branch)}", "") if git_branch
name = name.split("/").reject { |v| v.nil? || v.include?("=") }.join("/")
end
name
Expand Down
Loading