Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Revert "airbyte-ci: improve git diff comparison (#37616)" #37641

Merged
merged 2 commits into from
Apr 26, 2024
Merged

Revert "airbyte-ci: improve git diff comparison (#37616)" #37641

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b0a87ba
Revert "airbyte-ci: improve git diff comparison (#37616)"
erohmensing Apr 26, 2024
bcba5b1
update version
erohmensing Apr 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions airbyte-ci/connectors/pipelines/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ options to the `airbyte-ci` command group.**
| `--is-local/--is-ci` | `--is-local` | | Determines the environment in which the CLI runs: local environment or CI environment. |
| `--git-branch` | The checked out git branch name | `CI_GIT_BRANCH` | The git branch on which the pipelines will run. |
| `--git-revision` | The current branch head | `CI_GIT_REVISION` | The commit hash on which the pipelines will run. |
| `--diffed-branch` | `master` | | Branch to which the git diff will happen to detect new or modified files. |
| `--diffed-branch` | `origin/master` | | Branch to which the git diff will happen to detect new or modified files. |
| `--gha-workflow-run-id` | | | GHA CI only - The run id of the GitHub action workflow |
| `--ci-context` | `manual` | | The current CI context: `manual` for manual run, `pull_request`, `nightly_builds`, `master` |
| `--pipeline-start-timestamp` | Current epoch time | `CI_PIPELINE_START_TIMESTAMP` | Start time of the pipeline as epoch time. Used for pipeline run duration computation. |
Expand Down Expand Up @@ -674,12 +674,13 @@ E.G.: running Poe tasks on the modified internal packages of the current branch:

## Changelog

| Version | PR | Description |
|---------| ---------------------------------------------------------- |----------------------------------------------------------------------------------------------------------------------------|
| 4.10.3 | [#37615](https://github.com/airbytehq/airbyte/pull/37615) | Fix `KeyError` when running `migrate-to-poetry` |
| 4.10.2 | [#37614](https://github.com/airbytehq/airbyte/pull/37614) | Fix `UnboundLocalError: local variable 'add_changelog_entry_result' referenced before assignment` in `migrate_to_base_image` |
| 4.10.1 | [#37622](https://github.com/airbytehq/airbyte/pull/37622) | Temporarily disable regression tests in CI |
| 4.10.0 | [#37616](https://github.com/airbytehq/airbyte/pull/37616) | Improve modified files comparison when the target branch is from a fork. |
| Version | PR | Description |
|---------|------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------|
| 4.10.4 | [#37641](https://github.com/airbytehq/airbyte/pull/37641) | Temporarily revert changes from version 4.10.0 |
| 4.10.3 | [#37615](https://github.com/airbytehq/airbyte/pull/37615) | Fix `KeyError` when running `migrate-to-poetry` |
| 4.10.2 | [#37614](https://github.com/airbytehq/airbyte/pull/37614) | Fix `UnboundLocalError: local variable 'add_changelog_entry_result' referenced before assignment` in `migrate_to_base_image` |
| 4.10.1 | [#37622](https://github.com/airbytehq/airbyte/pull/37622) | Temporarily disable regression tests in CI |
| 4.10.0 | [#37616](https://github.com/airbytehq/airbyte/pull/37616) | Improve modified files comparison when the target branch is from a fork. |
| 4.9.0 | [#37440](https://github.com/airbytehq/airbyte/pull/37440) | Run regression tests with `airbyte-ci connectors test` |
| 4.8.0 | [#37404](https://github.com/airbytehq/airbyte/pull/37404) | Accept a `git-repo-url` option on the `airbyte-ci` root command to checkout forked repo. |
| 4.7.4 | [#37485](https://github.com/airbytehq/airbyte/pull/37485) | Allow java connectors to be written in kotlin. |
Expand Down
2 changes: 1 addition & 1 deletion airbyte-ci/connectors/pipelines/pipelines/cli/airbyte_ci.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ def is_current_process_wrapped_by_dagger_run() -> bool:
@click.option(
"--diffed-branch",
help="Branch to which the git diff will happen to detect new or modified connectors",
default="master",
default="origin/master",
type=str,
)
@click.option("--gha-workflow-run-id", help="[CI Only] The run id of the GitHub action workflow", default=None, type=str)
Expand Down
24 changes: 7 additions & 17 deletions airbyte-ci/connectors/pipelines/pipelines/dagger/containers/git.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,7 @@ async def checked_out_git_container(
diffed_branch: Optional[str] = None,
repo_url: str = AIRBYTE_REPO_URL,
) -> Container:
"""
Create a container with git in it.
We add the airbyte repo as the origin remote and the target repo as the target remote.
We fetch the diffed branch from the origin remote and the current branch from the target remote.
We then checkout the current branch.
"""
"""Builds git-based container with the current branch checked out."""
current_git_branch = current_git_branch.removeprefix("origin/")
diffed_branch = current_git_branch if diffed_branch is None else diffed_branch.removeprefix("origin/")
return await (
Expand All @@ -31,19 +26,14 @@ async def checked_out_git_container(
[
"remote",
"add",
"--fetch",
"--track",
current_git_branch,
"--track",
diffed_branch if diffed_branch is not None else current_git_branch,
"origin",
AIRBYTE_REPO_URL,
]
)
.with_exec(
[
"remote",
"add",
"target",
repo_url,
]
)
.with_exec(["fetch", "origin", diffed_branch])
.with_exec(["fetch", "target", current_git_branch])
.with_exec(["checkout", current_git_branch])
.with_exec(["checkout", "-t", f"origin/{current_git_branch}"])
)
4 changes: 2 additions & 2 deletions airbyte-ci/connectors/pipelines/pipelines/helpers/git.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ def get_current_git_branch() -> str: # noqa D103


async def get_modified_files_in_branch_remote(
current_git_repo_url: str, current_git_branch: str, current_git_revision: str, diffed_branch: str = "master", retries: int = 3
current_git_repo_url: str, current_git_branch: str, current_git_revision: str, diffed_branch: str = "origin/master", retries: int = 3
) -> Set[str]:
"""Use git diff to spot the modified files on the remote branch."""
try:
Expand All @@ -30,7 +30,7 @@ async def get_modified_files_in_branch_remote(
dagger_client, current_git_branch, current_git_revision, diffed_branch, repo_url=current_git_repo_url
)
modified_files = await container.with_exec(
["diff", f"--diff-filter={DIFF_FILTER}", "--name-only", f"origin/{diffed_branch}...target/{current_git_branch}"]
["diff", f"--diff-filter={DIFF_FILTER}", "--name-only", f"{diffed_branch}...{current_git_branch}"]
).stdout()
except SessionError:
if retries > 0:
Expand Down
2 changes: 1 addition & 1 deletion airbyte-ci/connectors/pipelines/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "pipelines"
version = "4.10.3"
version = "4.10.4"
description = "Packaged maintained by the connector operations team to perform CI for connectors' pipelines"
authors = ["Airbyte <contact@airbyte.io>"]

Expand Down
Loading