Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make the airbyte server build fully incremental #7539

Closed
benmoriceau opened this issue Nov 1, 2021 · 0 comments
Closed

Make the airbyte server build fully incremental #7539

benmoriceau opened this issue Nov 1, 2021 · 0 comments
Labels
area/platform issues related to the platform team/compose team/platform-move type/bug Something isn't working

Comments

@benmoriceau
Copy link
Contributor

When running gradlew build for the airbyte-server twice, the jar task is not considered as updated even if it should be.

This makes the build to be non incremental. Example of the error seen:

> Task :airbyte-server:jar
Caching disabled for task ':airbyte-server:jar' because:
  Not worth caching
Task ':airbyte-server:jar' is not up-to-date because:
  Output property 'archiveFile' file /Users/benoitmoriceau/workspace/airbyte/airbyte-server/build/libs/io.airbyte-airbyte-server-0.30.23-alpha.jar has changed.
:airbyte-server:jar (Thread[Execution worker for ':' Thread 8,5,main]) completed. Took 0.522 secs.
:airbyte-server:shadowJar (Thread[Execution worker for ':' Thread 8,5,main]) started.
@benmoriceau benmoriceau added type/bug Something isn't working 2021-q4-platform area/platform issues related to the platform labels Nov 1, 2021
@benmoriceau benmoriceau mentioned this issue Nov 1, 2021
Merged
@jrhizor jrhizor changed the title Make the airbyte server fully incremental Make the airbyte server build fully incremental Nov 2, 2021
@benmoriceau benmoriceau added this to the Platform 2021-11-18 milestone Nov 4, 2021
benmoriceau added a commit that referenced this issue Nov 5, 2021
@benmoriceau
Bmoric/remove docker compose for build (#7500)
4e17fa2 
This making the build using a gradle plugin instead of using docker-compose build.
It aims to make the build to be more incremental as described in #7306

Building the docker image don't rely on docker-compose anymore.
The docker build step is isolated into a dedicated folder (in order to make sure that gradle plugin don't recompute the build of the docker container)
Gradle is responsible for copying the files that docker needs to build its image.
That removes the need of having a dockerignore file.
This might not be effective until #7539 is solved.
schlattk pushed a commit to schlattk/airbyte that referenced this issue Jan 4, 2022
@benmoriceau @schlattk
Bmoric/remove docker compose for build (airbytehq#7500)
0d60f27 
This making the build using a gradle plugin instead of using docker-compose build.
It aims to make the build to be more incremental as described in airbytehq#7306

Building the docker image don't rely on docker-compose anymore.
The docker build step is isolated into a dedicated folder (in order to make sure that gradle plugin don't recompute the build of the docker container)
Gradle is responsible for copying the files that docker needs to build its image.
That removes the need of having a dockerignore file.
This might not be effective until airbytehq#7539 is solved.
@cgardens cgardens removed this from the Platform backlog milestone Apr 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/platform issues related to the platform team/compose team/platform-move type/bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@bleonard @evantahler @benmoriceau @cgardens