Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] added job for swig compilation on Linux with artifacts #2125

Merged
merged 7 commits into from
Apr 30, 2019
Merged

Conversation

StrikerRUS
Copy link
Collaborator

Closed #1945, closed #2124.

Compilation of SWIG wrapper with Clang was tested too: 3ee90d2
image

@@ -41,6 +41,26 @@ RUN curl -sL https://cmake.org/files/v3.14/cmake-3.14.1-Linux-x86_64.sh -o cmake
&& ./cmake.sh --prefix=/usr/local --exclude-subdir \
&& rm cmake.sh

# Install Java
RUN add-apt-repository ppa:openjdk-r/ppa -y \
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OpenJDK-8 is available from default ppa only since Ubuntu 14.10, but we use 14.04.
https://askubuntu.com/a/944260

ENV JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/

# Install SWIG
RUN curl -sL https://downloads.sourceforge.net/project/swig/swig/swig-3.0.12/swig-3.0.12.tar.gz -o swig.tar.gz \
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apt-get installs SWIG 2.0 on Ubuntu 14.04.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that too, thanks for fixing

RUN curl -sL https://downloads.sourceforge.net/project/swig/swig/swig-3.0.12/swig-3.0.12.tar.gz -o swig.tar.gz \
&& tar -xzf swig.tar.gz \
&& cd swig-3.0.12 \
&& ./configure --prefix=/usr/local --without-pcre \
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@imatiach-msft Is it OK to build SWIG without regex? Do you need this functionality in SWIG?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually not sure. Let's start with this for now and validate the binaries work.

make -j4 || exit -1
if [[ $COMPILER == "gcc" ]]; then
objdump -T $BUILD_DIRECTORY/lib_lightgbm.so > $BUILD_DIRECTORY/objdump.log || exit -1
objdump -T $BUILD_DIRECTORY/lib_lightgbm_swig.so >> $BUILD_DIRECTORY/objdump.log || exit -1
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@imatiach-msft Are there any other dependencies which we may want to check, except those that are already mentioned here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope, those are the only two. you have a script to validate glibc and gomp version? That's very cool.

.vsts-ci.yml Outdated Show resolved Hide resolved
@StrikerRUS
Copy link
Collaborator Author

@imatiach-msft Please review

@StrikerRUS StrikerRUS requested a review from guolinke April 22, 2019 02:29
@@ -25,6 +25,8 @@ jobs:
bdist:
TASK: bdist
PYTHON_VERSION: 3.6
swig:
TASK: swig
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, so this is how you turn that 'task' flag on, I see

@@ -15,7 +15,7 @@ jobs:
vmImage: 'ubuntu-16.04'
container: ubuntu1404
strategy:
maxParallel: 6
maxParallel: 7
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does maxParallel do?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@imatiach-msft imatiach-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@StrikerRUS
Copy link
Collaborator Author

@imatiach-msft Now jar file will be generated for each commit, but not PR, unfortunately, due to this restriction: https://github.com/Microsoft/LightGBM/blob/d115769c2a2ddffadc76c7b84739a47937114c77/.vsts-ci.yml#L57

You can download it from Azure Pipelines page. Navigate here: https://lightgbm-ci.visualstudio.com/lightgbm-ci/_build?definitionId=1, select commit, and click to Artifacts in the upper right corner. You need PackageAssets artifact.

image

For the latest artifacts from master branch it's easier to use download badge here:
https://lightgbm.readthedocs.io/en/latest/Installation-Guide.html

image

@StrikerRUS
Copy link
Collaborator Author

@guolinke Please merge it. Since I cannot trigger main docker update from any branch other than master, CI checks will always fail.

@StrikerRUS
Copy link
Collaborator Author

@guolinke Also please re-run CI for merged commit when docker will be ready. I suppose CI is faster than docker and will use outdated docker image for the first time, which will result in failing CI and artifacts' unavailability.

@guolinke guolinke merged commit 739c263 into master Apr 30, 2019
@StrikerRUS StrikerRUS deleted the docker branch April 30, 2019 10:39
@lock lock bot locked as resolved and limited conversation to collaborators Mar 11, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

lightGBM 2.2.200 still has glibc issue /lib64/libstdc++.so.6: version : GLIBCXX_3.4.20' not found
3 participants