Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dev: add pre-commit hooks installed by default #12293

Merged
merged 6 commits into from
Jan 8, 2025

Conversation

anshbansal
Copy link
Collaborator

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@github-actions github-actions bot added the devops PR or Issue related to DataHub backend & deployment label Jan 8, 2025
@datahub-cyborg datahub-cyborg bot added the needs-review Label for PRs that need review from a maintainer. label Jan 8, 2025
@datahub-cyborg datahub-cyborg bot added pending-submitter-merge and removed needs-review Label for PRs that need review from a maintainer. labels Jan 8, 2025
Copy link

codecov bot commented Jan 8, 2025

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
977 1 976 0
View the top 1 failed tests by shortest run time
tests.integration.test_great_expectations::test_ge_ingest[test_checkpoint-ge_mcps_golden.json]
Stack Traces | 2.51s run time
docker_compose_runner = <function docker_compose_runner.<locals>.run at 0x7f1c118d4ca0>
pytestconfig = <_pytest.config.Config object at 0x7f1c2b8a77c0>
tmp_path = PosixPath('.../pytest-of-runner/pytest-0/test_ge_ingest_test_checkpoint0')
checkpoint = 'test_checkpoint', golden_json = 'ge_mcps_golden.json', kwargs = {}
test_resources_dir = PosixPath('.../gx-plugin/tests/integration')

    @freeze_time(FROZEN_TIME)
    @pytest.mark.integration
    @pytest.mark.parametrize(
        "checkpoint, golden_json",
        [
            ("test_checkpoint", "ge_mcps_golden.json"),
            ("test_checkpoint_2", "ge_mcps_golden_2.json"),
        ],
    )
    def test_ge_ingest(
        docker_compose_runner,
        pytestconfig,
        tmp_path,
        checkpoint,
        golden_json,
        **kwargs,
    ):
        test_resources_dir = pytestconfig.rootpath / "tests/integration"
    
>       with docker_compose_runner(
            test_resources_dir / "docker-compose.yml", "great-expectations"
        ) as docker_services, mock.patch(
            "datahub.emitter.rest_emitter.DatahubRestEmitter.emit_mcp"
        ) as mock_emit_mcp:

tests/integration/test_great_expectations.py:62: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../hostedtoolcache/Python/3.8.18....../x64/lib/python3.8/contextlib.py:113: in __enter__
    return next(self.gen)
../...../datahub/testing/docker_utils.py:59: in run
    with pytest_docker.plugin.get_docker_services(
.../hostedtoolcache/Python/3.8.18....../x64/lib/python3.8/contextlib.py:113: in __enter__
    return next(self.gen)
venv/lib/python3.8........./site-packages/pytest_docker/plugin.py:212: in get_docker_services
    docker_compose.execute(command)
venv/lib/python3.8........./site-packages/pytest_docker/plugin.py:140: in execute
    return execute(command)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

command = 'docker compose -f ".../gx-plugin/tests/integration/docker-compose.yml" -p "pytest3941-great-expectations" up --build -d'
success_codes = (0,)

    def execute(command: str, success_codes: Iterable[int] = (0,)) -> Union[bytes, Any]:
        """Run a shell command."""
        try:
            output = subprocess.check_output(command, stderr=subprocess.STDOUT, shell=True)
            status = 0
        except subprocess.CalledProcessError as error:
            output = error.output or b""
            status = error.returncode
            command = error.cmd
    
        if status not in success_codes:
>           raise Exception(
                'Command {} returned {}: """{}""".'.format(command, status, output.decode("utf-8"))
            )
E           Exception: Command docker compose -f ".../gx-plugin/tests/integration/docker-compose.yml" -p "pytest3941-great-expectations" up --build -d returned 18: """time="2025-01-08T15:45:31Z" level=warning msg=".../gx-plugin/tests/integration/docker-compose.yml: `version` is obsolete"
E            ge_postgres Pulling 
E            245043d9199c Pulling fs layer 
E            7924d4027ace Pulling fs layer 
E            d13a44a2b630 Pulling fs layer 
E            675515a26492 Pulling fs layer 
E            9c5b56c4f76f Pulling fs layer 
E            023ef977a9f3 Pulling fs layer 
E            a573a1da1d39 Pulling fs layer 
E            c73953d28bae Pulling fs layer 
E            3a3013016cb7 Pulling fs layer 
E            ec54481c8ccb Pulling fs layer 
E            675515a26492 Waiting 
E            c73953d28bae Waiting 
E            023ef977a9f3 Waiting 
E            a573a1da1d39 Waiting 
E            ec54481c8ccb Waiting 
E            7924d4027ace Downloading [====================================>              ]     721B/981B
E            7924d4027ace Download complete 
E            d13a44a2b630 Downloading [>                                                  ]  12.32kB/1.113MB
E            d13a44a2b630 Verifying Checksum 
E            d13a44a2b630 Download complete 
E            245043d9199c Downloading 
E            675515a26492 Downloading [==================================================>]     116B/116B
E            675515a26492 Verifying Checksum 
E            675515a26492 Download complete 
E            9c5b56c4f76f Downloading [>                                                  ]  540.7kB/105.8MB
E            023ef977a9f3 Downloading [===>                                               ]     720B/9.883kB
E            023ef977a9f3 Downloading [==================================================>]  9.883kB/9.883kB
E            023ef977a9f3 Verifying Checksum 
E            023ef977a9f3 Download complete 
E            a573a1da1d39 Downloading [==================================================>]     129B/129B
E            a573a1da1d39 Verifying Checksum 
E            a573a1da1d39 Download complete 
E            9c5b56c4f76f Downloading [======>                                            ]  14.47MB/105.8MB
E            c73953d28bae Downloading [==================================================>]     171B/171B
E            c73953d28bae Verifying Checksum 
E            c73953d28bae Download complete 
E            3a3013016cb7 Downloading [============>                                      ]  1.369kB/5.42kB
E            3a3013016cb7 Downloading [==================================================>]   5.42kB/5.42kB
E            3a3013016cb7 Download complete 
E            9c5b56c4f76f Downloading [============>                                      ]  26.85MB/105.8MB
E            ec54481c8ccb Downloading [==================================================>]     185B/185B
E            ec54481c8ccb Verifying Checksum 
E            ec54481c8ccb Download complete 
E            9c5b56c4f76f Downloading [===============>                                   ]   33.8MB/105.8MB
E            9c5b56c4f76f Downloading [======================>                            ]  46.69MB/105.8MB
E            9c5b56c4f76f Downloading [==========================>                        ]  55.29MB/105.8MB
E            9c5b56c4f76f Downloading [=================================>                 ]   71.9MB/105.8MB
E            9c5b56c4f76f Downloading [====================================>              ]   77.8MB/105.8MB
E            9c5b56c4f76f Downloading [=======================================>           ]  83.67MB/105.8MB
E            9c5b56c4f76f Downloading [===========================================>       ]  91.17MB/105.8MB
E            9c5b56c4f76f Downloading [=================================================> ]  105.6MB/105.8MB
E            9c5b56c4f76f Verifying Checksum 
E            9c5b56c4f76f Download complete 
E           error parsing HTTP 403 response body: no error details found in HTTP response body: "{\"status\":403,\"message\":\"Error: invalid URL signature\"}"
E           """.

venv/lib/python3.8........./site-packages/pytest_docker/plugin.py:35: Exception

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

@anshbansal anshbansal merged commit 58b6a5b into master Jan 8, 2025
48 checks passed
@anshbansal anshbansal deleted the ab-2025-jan-08-add-pre-commit branch January 8, 2025 16:27
llance pushed a commit to llance/datahub that referenced this pull request Jan 13, 2025
llance pushed a commit to llance/datahub that referenced this pull request Jan 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devops PR or Issue related to DataHub backend & deployment pending-submitter-merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants