Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIVE-28629: Upgrade dind in precommit #5546

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

abstractdog
Copy link
Contributor

@abstractdog abstractdog commented Nov 15, 2024

What changes were proposed in this pull request?

Upgrade dind version.

Why are the changes needed?

Due to incompatibility issues.

Does this PR introduce any user-facing change?

No.

Is the change a dependency upgrade?

No.

How was this patch tested?

Precommit picks up the new Jenkinsfile, so this is tested.

@abstractdog
Copy link
Contributor Author

https://ci.hive.apache.org/job/hive-precommit/job/PR-5546/1/console

Obtained Jenkinsfile from 1bbfb42d49ed86b023e6706be534a2c6e21b5b9f

@abstractdog abstractdog changed the title HIVE-28629: hive-precommit job runs shows that precommit runs on external repository's Jenkinsfile - trying to upgrade dind HIVE-28629: Upgrade dind in precommit Nov 15, 2024
@abstractdog
Copy link
Contributor Author

abstractdog commented Nov 15, 2024

there is a port mismatch in the first revision, taking care of that:

2376 vs 2375

from dind:

│ time="2024-11-15T09:40:00.366878279Z" level=info msg="API listen on /var/run/docker.sock"                                                                                                                                                                  │
│ time="2024-11-15T09:40:00.372866219Z" level=info msg="API listen on :2376"

from hive-dev-box:

jenkins@hive-precommit-pr-5546-1-klq14-z6tnc-xr3h7:~$ docker run --rm --name qtestExternalDB-MySQLExternalDB -p 3306:3306 -e MYSQL_ROOT_PASSWORD=qtestpassword -e MYSQL_DATABASE=qtestDB -d mysql:8.4.3
docker: Cannot connect to the Docker daemon at tcp://localhost:2375. Is the docker daemon running?.
See 'docker run --help'.

Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.

@abstractdog
Copy link
Contributor Author

abstractdog commented Jan 15, 2025

to move forward, we need to figure out a way to test this locally, so somehow, in a k8s environment, instantiate the hive testing pod with containers defined here:

hive/Jenkinsfile

Lines 131 to 144 in 88e2175

containerTemplate(name: 'hdb', image: 'wecharyu/hive-dev-box:executor', ttyEnabled: true, command: 'tini -- cat',
alwaysPullImage: true,
resourceRequestCpu: '1800m',
resourceLimitCpu: '8000m',
resourceRequestMemory: '6400Mi',
resourceLimitMemory: '12000Mi',
envVars: [
envVar(key: 'DOCKER_HOST', value: 'tcp://localhost:2375')
]
),
containerTemplate(name: 'dind', image: 'docker:18.05-dind',
alwaysPullImage: true,
privileged: true,
),

cc: @KiranVelumuri

@github-actions github-actions bot removed the stale label Jan 16, 2025
@KiranVelumuri
Copy link
Contributor

to move forward, we need to figure out a way to test this locally, so somehow, in a k8s environment, instantiate the hive testing pod with containers defined here:

hive/Jenkinsfile

Lines 131 to 144 in 88e2175

containerTemplate(name: 'hdb', image: 'wecharyu/hive-dev-box:executor', ttyEnabled: true, command: 'tini -- cat',
alwaysPullImage: true,
resourceRequestCpu: '1800m',
resourceLimitCpu: '8000m',
resourceRequestMemory: '6400Mi',
resourceLimitMemory: '12000Mi',
envVars: [
envVar(key: 'DOCKER_HOST', value: 'tcp://localhost:2375')
]
),
containerTemplate(name: 'dind', image: 'docker:18.05-dind',
alwaysPullImage: true,
privileged: true,
),

cc: @KiranVelumuri

I ran this locally on my laptop, but facing issue due to cgroup(control groups) version mismatch. The docker image is expecting cgroup v1, while the docker VM is using cgroup v2. I tried to alter the cgroup version locally but to no avail. I would update here in case of any progress.

@KiranVelumuri
Copy link
Contributor

@abstractdog @zabetak Could you please help here? This would help me to unblock dependent MySQL upgrade needed for another issue. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants