Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ci] use Python 3.8 for bdist CI job in old Ubuntu container #5022

Merged
merged 1 commit into from
Feb 22, 2022
Merged

[ci] use Python 3.8 for bdist CI job in old Ubuntu container #5022

merged 1 commit into from
Feb 22, 2022

Conversation

StrikerRUS
Copy link
Collaborator

Workaround for current master branch fails.
Almost every Dask test fails with

2022-02-20T15:57:32.0565604Z ../tests/python_package_test/test_dask.py:323: 
2022-02-20T15:57:32.0566450Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2022-02-20T15:57:32.0567909Z /home/AzDevOps_azpcontainer/.local/lib/python3.10/site-packages/lightgbm/dask.py:1170: in fit
2022-02-20T15:57:32.0568871Z     return self._lgb_dask_fit(
2022-02-20T15:57:32.0570168Z /home/AzDevOps_azpcontainer/.local/lib/python3.10/site-packages/lightgbm/dask.py:1050: in _lgb_dask_fit
2022-02-20T15:57:32.0571146Z     model = _train(
2022-02-20T15:57:32.0572438Z /home/AzDevOps_azpcontainer/.local/lib/python3.10/site-packages/lightgbm/dask.py:752: in _train
2022-02-20T15:57:32.0573499Z     worker_address_to_port = _assign_open_ports_to_workers(client, host_to_workers)
2022-02-20T15:57:32.0575003Z /home/AzDevOps_azpcontainer/.local/lib/python3.10/site-packages/lightgbm/dask.py:129: in _assign_open_ports_to_workers
2022-02-20T15:57:32.0576039Z     found_ports = client.gather(host_ports_futures)
2022-02-20T15:57:32.0577348Z /opt/miniforge/envs/test-env/lib/python3.10/site-packages/distributed/client.py:2146: in gather
2022-02-20T15:57:32.0578148Z     return self.sync(
2022-02-20T15:57:32.0578810Z /opt/miniforge/envs/test-env/lib/python3.10/site-packages/distributed/utils.py:309: in sync
2022-02-20T15:57:32.0579252Z     return sync(
2022-02-20T15:57:32.0579869Z /opt/miniforge/envs/test-env/lib/python3.10/site-packages/distributed/utils.py:376: in sync
2022-02-20T15:57:32.0580323Z     raise exc.with_traceback(tb)
2022-02-20T15:57:32.0580961Z /opt/miniforge/envs/test-env/lib/python3.10/site-packages/distributed/utils.py:349: in f
2022-02-20T15:57:32.0581412Z     result = yield future
2022-02-20T15:57:32.0582017Z /opt/miniforge/envs/test-env/lib/python3.10/site-packages/tornado/gen.py:762: in run
2022-02-20T15:57:32.0582444Z     value = future.result()
2022-02-20T15:57:32.0583188Z /opt/miniforge/envs/test-env/lib/python3.10/site-packages/distributed/client.py:2009: in _gather
2022-02-20T15:57:32.0584071Z     raise exception.with_traceback(traceback)
2022-02-20T15:57:32.0585380Z /opt/miniforge/envs/test-env/lib/python3.10/site-packages/distributed/protocol/pickle.py:75: in loads
2022-02-20T15:57:32.0586281Z     return pickle.loads(x)
2022-02-20T15:57:32.0587452Z /home/AzDevOps_azpcontainer/.local/lib/python3.10/site-packages/lightgbm/__init__.py:8: in <module>
2022-02-20T15:57:32.0588010Z     from .basic import Booster, Dataset, Sequence, register_logger
2022-02-20T15:57:32.0588732Z /home/AzDevOps_azpcontainer/.local/lib/python3.10/site-packages/lightgbm/basic.py:111: in <module>
2022-02-20T15:57:32.0589180Z     _LIB = _load_lib()
2022-02-20T15:57:32.0589824Z /home/AzDevOps_azpcontainer/.local/lib/python3.10/site-packages/lightgbm/basic.py:102: in _load_lib
2022-02-20T15:57:32.0590322Z     lib = ctypes.cdll.LoadLibrary(lib_path[0])
2022-02-20T15:57:32.0590965Z /opt/miniforge/envs/test-env/lib/python3.10/ctypes/__init__.py:452: in LoadLibrary
2022-02-20T15:57:32.0591399Z     return self._dlltype(name)
2022-02-20T15:57:32.0591886Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2022-02-20T15:57:32.0592150Z 
2022-02-20T15:57:32.0592783Z     """create and manipulate C data types in Python"""
2022-02-20T15:57:32.0593427Z     
2022-02-20T15:57:32.0594004Z     import os as _os, sys as _sys
2022-02-20T15:57:32.0594594Z     import types as _types
2022-02-20T15:57:32.0595153Z     
2022-02-20T15:57:32.0595649Z     __version__ = "1.1.0"
2022-02-20T15:57:32.0596461Z     
2022-02-20T15:57:32.0597036Z     from _ctypes import Union, Structure, Array
2022-02-20T15:57:32.0597968Z     from _ctypes import _Pointer
2022-02-20T15:57:32.0598651Z     from _ctypes import CFuncPtr as _CFuncPtr
2022-02-20T15:57:32.0599485Z     from _ctypes import __version__ as _ctypes_version
2022-02-20T15:57:32.0600266Z     from _ctypes import RTLD_LOCAL, RTLD_GLOBAL
2022-02-20T15:57:32.0600828Z     from _ctypes import ArgumentError
2022-02-20T15:57:32.0601595Z     
2022-02-20T15:57:32.0602210Z     from struct import calcsize as _calcsize
2022-02-20T15:57:32.0602892Z     
2022-02-20T15:57:32.0603415Z     if __version__ != _ctypes_version:
2022-02-20T15:57:32.0604151Z         raise Exception("Version number mismatch", __version__, _ctypes_version)
2022-02-20T15:57:32.0604731Z     
2022-02-20T15:57:32.0605037Z     if _os.name == "nt":
2022-02-20T15:57:32.0605378Z         from _ctypes import FormatError
2022-02-20T15:57:32.0605676Z     
2022-02-20T15:57:32.0605970Z     DEFAULT_MODE = RTLD_LOCAL
2022-02-20T15:57:32.0606350Z     if _os.name == "posix" and _sys.platform == "darwin":
2022-02-20T15:57:32.0606758Z         # On OS X 10.3, we use RTLD_GLOBAL as default mode
2022-02-20T15:57:32.0607169Z         # because RTLD_LOCAL does not work at least on some
2022-02-20T15:57:32.0607583Z         # libraries.  OS X 10.3 is Darwin 7, so we check for
2022-02-20T15:57:32.0607927Z         # that.
2022-02-20T15:57:32.0608185Z     
2022-02-20T15:57:32.0608841Z         if int(_os.uname().release.split('.')[0]) < 8:
2022-02-20T15:57:32.0609240Z             DEFAULT_MODE = RTLD_GLOBAL
2022-02-20T15:57:32.0609533Z     
2022-02-20T15:57:32.0609876Z     from _ctypes import FUNCFLAG_CDECL as _FUNCFLAG_CDECL, \
2022-02-20T15:57:32.0610293Z          FUNCFLAG_PYTHONAPI as _FUNCFLAG_PYTHONAPI, \
2022-02-20T15:57:32.0610692Z          FUNCFLAG_USE_ERRNO as _FUNCFLAG_USE_ERRNO, \
2022-02-20T15:57:32.0611090Z          FUNCFLAG_USE_LASTERROR as _FUNCFLAG_USE_LASTERROR
2022-02-20T15:57:32.0611429Z     
2022-02-20T15:57:32.0611865Z     # WINOLEAPI -> HRESULT
2022-02-20T15:57:32.0612198Z     # WINOLEAPI_(type)
2022-02-20T15:57:32.0612536Z     #
2022-02-20T15:57:32.0612915Z     # STDMETHODCALLTYPE
2022-02-20T15:57:32.0613273Z     #
2022-02-20T15:57:32.0613700Z     # STDMETHOD(name)
2022-02-20T15:57:32.0614149Z     # STDMETHOD_(type, name)
2022-02-20T15:57:32.0614590Z     #
2022-02-20T15:57:32.0615017Z     # STDAPICALLTYPE
2022-02-20T15:57:32.0615524Z     
2022-02-20T15:57:32.0615991Z     def create_string_buffer(init, size=None):
2022-02-20T15:57:32.0616767Z         """create_string_buffer(aBytes) -> character array
2022-02-20T15:57:32.0617462Z         create_string_buffer(anInteger) -> character array
2022-02-20T15:57:32.0618164Z         create_string_buffer(aBytes, anInteger) -> character array
2022-02-20T15:57:32.0618603Z         """
2022-02-20T15:57:32.0618968Z         if isinstance(init, bytes):
2022-02-20T15:57:32.0619365Z             if size is None:
2022-02-20T15:57:32.0619743Z                 size = len(init)+1
2022-02-20T15:57:32.0620188Z             _sys.audit("ctypes.create_string_buffer", init, size)
2022-02-20T15:57:32.0620725Z             buftype = c_char * size
2022-02-20T15:57:32.0621228Z             buf = buftype()
2022-02-20T15:57:32.0621713Z             buf.value = init
2022-02-20T15:57:32.0622208Z             return buf
2022-02-20T15:57:32.0622693Z         elif isinstance(init, int):
2022-02-20T15:57:32.0623222Z             _sys.audit("ctypes.create_string_buffer", None, init)
2022-02-20T15:57:32.0623809Z             buftype = c_char * init
2022-02-20T15:57:32.0624228Z             buf = buftype()
2022-02-20T15:57:32.0624624Z             return buf
2022-02-20T15:57:32.0625010Z         raise TypeError(init)
2022-02-20T15:57:32.0625407Z     
2022-02-20T15:57:32.0625828Z     def c_buffer(init, size=None):
2022-02-20T15:57:32.0626416Z     ##    "deprecated, use create_string_buffer instead"
2022-02-20T15:57:32.0627025Z     ##    import warnings
2022-02-20T15:57:32.0627650Z     ##    warnings.warn("c_buffer is deprecated, use create_string_buffer instead",
2022-02-20T15:57:32.0628540Z     ##                  DeprecationWarning, stacklevel=2)
2022-02-20T15:57:32.0629239Z         return create_string_buffer(init, size)
2022-02-20T15:57:32.0629678Z     
2022-02-20T15:57:32.0630077Z     _c_functype_cache = {}
2022-02-20T15:57:32.0630676Z     def CFUNCTYPE(restype, *argtypes, **kw):
2022-02-20T15:57:32.0631276Z         """CFUNCTYPE(restype, *argtypes,
2022-02-20T15:57:32.0632565Z                      use_errno=False, use_last_error=False) -> function prototype.
2022-02-20T15:57:32.0633119Z     
2022-02-20T15:57:32.0633565Z         restype: the result type
2022-02-20T15:57:32.0634155Z         argtypes: a sequence specifying the argument types
2022-02-20T15:57:32.0634706Z     
2022-02-20T15:57:32.0635308Z         The function prototype can be called in different ways to create a
2022-02-20T15:57:32.0636023Z         callable object:
2022-02-20T15:57:32.0636436Z     
2022-02-20T15:57:32.0637011Z         prototype(integer address) -> foreign function
2022-02-20T15:57:32.0637624Z         prototype(callable) -> create and return a C callable function from callable
2022-02-20T15:57:32.0638365Z         prototype(integer index, method name[, paramflags]) -> foreign function calling a COM method
2022-02-20T15:57:32.0639108Z         prototype((ordinal number, dll object)[, paramflags]) -> foreign function exported by ordinal
2022-02-20T15:57:32.0639822Z         prototype((function name, dll object)[, paramflags]) -> foreign function exported by name
2022-02-20T15:57:32.0640235Z         """
2022-02-20T15:57:32.0640533Z         flags = _FUNCFLAG_CDECL
2022-02-20T15:57:32.0640875Z         if kw.pop("use_errno", False):
2022-02-20T15:57:32.0641478Z             flags |= _FUNCFLAG_USE_ERRNO
2022-02-20T15:57:32.0642187Z         if kw.pop("use_last_error", False):
2022-02-20T15:57:32.0642726Z             flags |= _FUNCFLAG_USE_LASTERROR
2022-02-20T15:57:32.0643444Z         if kw:
2022-02-20T15:57:32.0644155Z             raise ValueError("unexpected keyword argument(s) %s" % kw.keys())
2022-02-20T15:57:32.0644851Z         try:
2022-02-20T15:57:32.0645483Z             return _c_functype_cache[(restype, argtypes, flags)]
2022-02-20T15:57:32.0646131Z         except KeyError:
2022-02-20T15:57:32.0646781Z             class CFunctionType(_CFuncPtr):
2022-02-20T15:57:32.0647305Z                 _argtypes_ = argtypes
2022-02-20T15:57:32.0647806Z                 _restype_ = restype
2022-02-20T15:57:32.0648259Z                 _flags_ = flags
2022-02-20T15:57:32.0648718Z             _c_functype_cache[(restype, argtypes, flags)] = CFunctionType
2022-02-20T15:57:32.0649201Z             return CFunctionType
2022-02-20T15:57:32.0649647Z     
2022-02-20T15:57:32.0650082Z     if _os.name == "nt":
2022-02-20T15:57:32.0650626Z         from _ctypes import LoadLibrary as _dlopen
2022-02-20T15:57:32.0651183Z         from _ctypes import FUNCFLAG_STDCALL as _FUNCFLAG_STDCALL
2022-02-20T15:57:32.0651579Z     
2022-02-20T15:57:32.0651917Z         _win_functype_cache = {}
2022-02-20T15:57:32.0652330Z         def WINFUNCTYPE(restype, *argtypes, **kw):
2022-02-20T15:57:32.0652786Z             # docstring set later (very similar to CFUNCTYPE.__doc__)
2022-02-20T15:57:32.0653278Z             flags = _FUNCFLAG_STDCALL
2022-02-20T15:57:32.0655307Z             if kw.pop("use_errno", False):
2022-02-20T15:57:32.0655821Z                 flags |= _FUNCFLAG_USE_ERRNO
2022-02-20T15:57:32.0656232Z             if kw.pop("use_last_error", False):
2022-02-20T15:57:32.0656655Z                 flags |= _FUNCFLAG_USE_LASTERROR
2022-02-20T15:57:32.0657039Z             if kw:
2022-02-20T15:57:32.0657615Z                 raise ValueError("unexpected keyword argument(s) %s" % kw.keys())
2022-02-20T15:57:32.0658200Z             try:
2022-02-20T15:57:32.0658676Z                 return _win_functype_cache[(restype, argtypes, flags)]
2022-02-20T15:57:32.0659134Z             except KeyError:
2022-02-20T15:57:32.0659540Z                 class WinFunctionType(_CFuncPtr):
2022-02-20T15:57:32.0660157Z                     _argtypes_ = argtypes
2022-02-20T15:57:32.0660548Z                     _restype_ = restype
2022-02-20T15:57:32.0661024Z                     _flags_ = flags
2022-02-20T15:57:32.0661447Z                 _win_functype_cache[(restype, argtypes, flags)] = WinFunctionType
2022-02-20T15:57:32.0661895Z                 return WinFunctionType
2022-02-20T15:57:32.0662276Z         if WINFUNCTYPE.__doc__:
2022-02-20T15:57:32.0662740Z             WINFUNCTYPE.__doc__ = CFUNCTYPE.__doc__.replace("CFUNCTYPE", "WINFUNCTYPE")
2022-02-20T15:57:32.0663155Z     
2022-02-20T15:57:32.0663492Z     elif _os.name == "posix":
2022-02-20T15:57:32.0663884Z         from _ctypes import dlopen as _dlopen
2022-02-20T15:57:32.0664255Z     
2022-02-20T15:57:32.0664640Z     from _ctypes import sizeof, byref, addressof, alignment, resize
2022-02-20T15:57:32.0665100Z     from _ctypes import get_errno, set_errno
2022-02-20T15:57:32.0665520Z     from _ctypes import _SimpleCData
2022-02-20T15:57:32.0665859Z     
2022-02-20T15:57:32.0666219Z     def _check_size(typ, typecode=None):
2022-02-20T15:57:32.0666670Z         # Check if sizeof(ctypes_type) against struct.calcsize.  This
2022-02-20T15:57:32.0667190Z         # should protect somewhat against a misconfigured libffi.
2022-02-20T15:57:32.0667609Z         from struct import calcsize
2022-02-20T15:57:32.0667981Z         if typecode is None:
2022-02-20T15:57:32.0668418Z             # Most _type_ codes are the same as used in struct
2022-02-20T15:57:32.0668833Z             typecode = typ._type_
2022-02-20T15:57:32.0669249Z         actual, required = sizeof(typ), calcsize(typecode)
2022-02-20T15:57:32.0669669Z         if actual != required:
2022-02-20T15:57:32.0670105Z             raise SystemError("sizeof(%s) wrong: %d instead of %d" % \
2022-02-20T15:57:32.0670703Z                               (typ, actual, required))
2022-02-20T15:57:32.0671162Z     
2022-02-20T15:57:32.0671599Z     class py_object(_SimpleCData):
2022-02-20T15:57:32.0672076Z         _type_ = "O"
2022-02-20T15:57:32.0672498Z         def __repr__(self):
2022-02-20T15:57:32.0672853Z             try:
2022-02-20T15:57:32.0673215Z                 return super().__repr__()
2022-02-20T15:57:32.0673598Z             except ValueError:
2022-02-20T15:57:32.0674065Z                 return "%s(<NULL>)" % type(self).__name__
2022-02-20T15:57:32.0674625Z     _check_size(py_object, "P")
2022-02-20T15:57:32.0675066Z     
2022-02-20T15:57:32.0675455Z     class c_short(_SimpleCData):
2022-02-20T15:57:32.0675852Z         _type_ = "h"
2022-02-20T15:57:32.0676218Z     _check_size(c_short)
2022-02-20T15:57:32.0676540Z     
2022-02-20T15:57:32.0676884Z     class c_ushort(_SimpleCData):
2022-02-20T15:57:32.0677246Z         _type_ = "H"
2022-02-20T15:57:32.0677609Z     _check_size(c_ushort)
2022-02-20T15:57:32.0677971Z     
2022-02-20T15:57:32.0678391Z     class c_long(_SimpleCData):
2022-02-20T15:57:32.0678897Z         _type_ = "l"
2022-02-20T15:57:32.0679359Z     _check_size(c_long)
2022-02-20T15:57:32.0679733Z     
2022-02-20T15:57:32.0680145Z     class c_ulong(_SimpleCData):
2022-02-20T15:57:32.0680516Z         _type_ = "L"
2022-02-20T15:57:32.0680857Z     _check_size(c_ulong)
2022-02-20T15:57:32.0681389Z     
2022-02-20T15:57:32.0681871Z     if _calcsize("i") == _calcsize("l"):
2022-02-20T15:57:32.0682521Z         # if int and long have the same size, make c_int an alias for c_long
2022-02-20T15:57:32.0683093Z         c_int = c_long
2022-02-20T15:57:32.0683527Z         c_uint = c_ulong
2022-02-20T15:57:32.0684013Z     else:
2022-02-20T15:57:32.0684460Z         class c_int(_SimpleCData):
2022-02-20T15:57:32.0684842Z             _type_ = "i"
2022-02-20T15:57:32.0685199Z         _check_size(c_int)
2022-02-20T15:57:32.0685522Z     
2022-02-20T15:57:32.0685866Z         class c_uint(_SimpleCData):
2022-02-20T15:57:32.0686236Z             _type_ = "I"
2022-02-20T15:57:32.0686589Z         _check_size(c_uint)
2022-02-20T15:57:32.0686929Z     
2022-02-20T15:57:32.0687272Z     class c_float(_SimpleCData):
2022-02-20T15:57:32.0687640Z         _type_ = "f"
2022-02-20T15:57:32.0688133Z     _check_size(c_float)
2022-02-20T15:57:32.0688471Z     
2022-02-20T15:57:32.0688966Z     class c_double(_SimpleCData):
2022-02-20T15:57:32.0689316Z         _type_ = "d"
2022-02-20T15:57:32.0689660Z     _check_size(c_double)
2022-02-20T15:57:32.0690000Z     
2022-02-20T15:57:32.0690349Z     class c_longdouble(_SimpleCData):
2022-02-20T15:57:32.0690755Z         _type_ = "g"
2022-02-20T15:57:32.0691256Z     if sizeof(c_longdouble) == sizeof(c_double):
2022-02-20T15:57:32.0691770Z         c_longdouble = c_double
2022-02-20T15:57:32.0692178Z     
2022-02-20T15:57:32.0692632Z     if _calcsize("l") == _calcsize("q"):
2022-02-20T15:57:32.0693320Z         # if long and long long have the same size, make c_longlong an alias for c_long
2022-02-20T15:57:32.0694286Z         c_longlong = c_long
2022-02-20T15:57:32.0694819Z         c_ulonglong = c_ulong
2022-02-20T15:57:32.0695275Z     else:
2022-02-20T15:57:32.0695714Z         class c_longlong(_SimpleCData):
2022-02-20T15:57:32.0696172Z             _type_ = "q"
2022-02-20T15:57:32.0696658Z         _check_size(c_longlong)
2022-02-20T15:57:32.0697104Z     
2022-02-20T15:57:32.0697557Z         class c_ulonglong(_SimpleCData):
2022-02-20T15:57:32.0698045Z             _type_ = "Q"
2022-02-20T15:57:32.0698539Z         ##    def from_param(cls, val):
2022-02-20T15:57:32.0699577Z         ##        return ('d', float(val), val)
2022-02-20T15:57:32.0700160Z         ##    from_param = classmethod(from_param)
2022-02-20T15:57:32.0700672Z         _check_size(c_ulonglong)
2022-02-20T15:57:32.0701120Z     
2022-02-20T15:57:32.0701551Z     class c_ubyte(_SimpleCData):
2022-02-20T15:57:32.0701977Z         _type_ = "B"
2022-02-20T15:57:32.0702490Z     c_ubyte.__ctype_le__ = c_ubyte.__ctype_be__ = c_ubyte
2022-02-20T15:57:32.0703055Z     # backward compatibility:
2022-02-20T15:57:32.0703693Z     ##c_uchar = c_ubyte
2022-02-20T15:57:32.0704301Z     _check_size(c_ubyte)
2022-02-20T15:57:32.0704828Z     
2022-02-20T15:57:32.0705378Z     class c_byte(_SimpleCData):
2022-02-20T15:57:32.0706003Z         _type_ = "b"
2022-02-20T15:57:32.0706762Z     c_byte.__ctype_le__ = c_byte.__ctype_be__ = c_byte
2022-02-20T15:57:32.0707477Z     _check_size(c_byte)
2022-02-20T15:57:32.0707982Z     
2022-02-20T15:57:32.0708545Z     class c_char(_SimpleCData):
2022-02-20T15:57:32.0709131Z         _type_ = "c"
2022-02-20T15:57:32.0709731Z     c_char.__ctype_le__ = c_char.__ctype_be__ = c_char
2022-02-20T15:57:32.0710401Z     _check_size(c_char)
2022-02-20T15:57:32.0710956Z     
2022-02-20T15:57:32.0711506Z     class c_char_p(_SimpleCData):
2022-02-20T15:57:32.0712087Z         _type_ = "z"
2022-02-20T15:57:32.0712653Z         def __repr__(self):
2022-02-20T15:57:32.0713414Z             return "%s(%s)" % (self.__class__.__name__, c_void_p.from_buffer(self).value)
2022-02-20T15:57:32.0714180Z     _check_size(c_char_p, "P")
2022-02-20T15:57:32.0714762Z     
2022-02-20T15:57:32.0715280Z     class c_void_p(_SimpleCData):
2022-02-20T15:57:32.0715840Z         _type_ = "P"
2022-02-20T15:57:32.0716482Z     c_voidp = c_void_p # backwards compatibility (to a bug)
2022-02-20T15:57:32.0717181Z     _check_size(c_void_p)
2022-02-20T15:57:32.0717814Z     
2022-02-20T15:57:32.0718236Z     class c_bool(_SimpleCData):
2022-02-20T15:57:32.0718605Z         _type_ = "?"
2022-02-20T15:57:32.0718930Z     
2022-02-20T15:57:32.0719318Z     from _ctypes import POINTER, pointer, _pointer_type_cache
2022-02-20T15:57:32.0719705Z     
2022-02-20T15:57:32.0720055Z     class c_wchar_p(_SimpleCData):
2022-02-20T15:57:32.0720426Z         _type_ = "Z"
2022-02-20T15:57:32.0720801Z         def __repr__(self):
2022-02-20T15:57:32.0721470Z             return "%s(%s)" % (self.__class__.__name__, c_void_p.from_buffer(self).value)
2022-02-20T15:57:32.0721947Z     
2022-02-20T15:57:32.0722348Z     class c_wchar(_SimpleCData):
2022-02-20T15:57:32.0722714Z         _type_ = "u"
2022-02-20T15:57:32.0723061Z     
2022-02-20T15:57:32.0723439Z     def _reset_cache():
2022-02-20T15:57:32.0723838Z         _pointer_type_cache.clear()
2022-02-20T15:57:32.0724424Z         _c_functype_cache.clear()
2022-02-20T15:57:32.0724850Z         if _os.name == "nt":
2022-02-20T15:57:32.0725347Z             _win_functype_cache.clear()
2022-02-20T15:57:32.0725779Z         # _SimpleCData.c_wchar_p_from_param
2022-02-20T15:57:32.0726240Z         POINTER(c_wchar).from_param = c_wchar_p.from_param
2022-02-20T15:57:32.0726762Z         # _SimpleCData.c_char_p_from_param
2022-02-20T15:57:32.0727255Z         POINTER(c_char).from_param = c_char_p.from_param
2022-02-20T15:57:32.0727706Z         _pointer_type_cache[None] = c_void_p
2022-02-20T15:57:32.0728098Z     
2022-02-20T15:57:32.0728509Z     def create_unicode_buffer(init, size=None):
2022-02-20T15:57:32.0729352Z         """create_unicode_buffer(aString) -> character array
2022-02-20T15:57:32.0730083Z         create_unicode_buffer(anInteger) -> character array
2022-02-20T15:57:32.0730776Z         create_unicode_buffer(aString, anInteger) -> character array
2022-02-20T15:57:32.0731201Z         """
2022-02-20T15:57:32.0731554Z         if isinstance(init, str):
2022-02-20T15:57:32.0732088Z             if size is None:
2022-02-20T15:57:32.0732819Z                 if sizeof(c_wchar) == 2:
2022-02-20T15:57:32.0733919Z                     # UTF-16 requires a surrogate pair (2 wchar_t) for non-BMP
2022-02-20T15:57:32.0734583Z                     # characters (outside [U+0000; U+FFFF] range). +1 for trailing
2022-02-20T15:57:32.0735079Z                     # NUL character.
2022-02-20T15:57:32.0735661Z                     size = sum(2 if ord(c) > 0xFFFF else 1 for c in init) + 1
2022-02-20T15:57:32.0736099Z                 else:
2022-02-20T15:57:32.0736946Z                     # 32-bit wchar_t (1 wchar_t per Unicode character). +1 for
2022-02-20T15:57:32.0737593Z                     # trailing NUL character.
2022-02-20T15:57:32.0738061Z                     size = len(init) + 1
2022-02-20T15:57:32.0738554Z             _sys.audit("ctypes.create_unicode_buffer", init, size)
2022-02-20T15:57:32.0739007Z             buftype = c_wchar * size
2022-02-20T15:57:32.0739429Z             buf = buftype()
2022-02-20T15:57:32.0739824Z             buf.value = init
2022-02-20T15:57:32.0740291Z             return buf
2022-02-20T15:57:32.0740787Z         elif isinstance(init, int):
2022-02-20T15:57:32.0741354Z             _sys.audit("ctypes.create_unicode_buffer", None, init)
2022-02-20T15:57:32.0741847Z             buftype = c_wchar * init
2022-02-20T15:57:32.0742213Z             buf = buftype()
2022-02-20T15:57:32.0742604Z             return buf
2022-02-20T15:57:32.0742992Z         raise TypeError(init)
2022-02-20T15:57:32.0743341Z     
2022-02-20T15:57:32.0743696Z     
2022-02-20T15:57:32.0744018Z     # XXX Deprecated
2022-02-20T15:57:32.0744415Z     def SetPointerType(pointer, cls):
2022-02-20T15:57:32.0744816Z         if _pointer_type_cache.get(cls, None) is not None:
2022-02-20T15:57:32.0745327Z             raise RuntimeError("This type already exists in the cache")
2022-02-20T15:57:32.0745809Z         if id(pointer) not in _pointer_type_cache:
2022-02-20T15:57:32.0746458Z             raise RuntimeError("What's this???")
2022-02-20T15:57:32.0746906Z         pointer.set_type(cls)
2022-02-20T15:57:32.0747268Z         _pointer_type_cache[cls] = pointer
2022-02-20T15:57:32.0747764Z         del _pointer_type_cache[id(pointer)]
2022-02-20T15:57:32.0748141Z     
2022-02-20T15:57:32.0748548Z     # XXX Deprecated
2022-02-20T15:57:32.0749023Z     def ARRAY(typ, len):
2022-02-20T15:57:32.0749478Z         return typ * len
2022-02-20T15:57:32.0749923Z     
2022-02-20T15:57:32.0750271Z     ################################################################
2022-02-20T15:57:32.0750638Z     
2022-02-20T15:57:32.0750943Z     
2022-02-20T15:57:32.0751343Z     class CDLL(object):
2022-02-20T15:57:32.0751788Z         """An instance of this class represents a loaded dll/shared
2022-02-20T15:57:32.0752307Z         library, exporting functions using the standard C calling
2022-02-20T15:57:32.0753027Z         convention (named 'cdecl' on Windows).
2022-02-20T15:57:32.0753381Z     
2022-02-20T15:57:32.0753868Z         The exported functions can be accessed as attributes, or by
2022-02-20T15:57:32.0754370Z         indexing with the function name.  Examples:
2022-02-20T15:57:32.0754695Z     
2022-02-20T15:57:32.0755244Z         <obj>.qsort -> callable object
2022-02-20T15:57:32.0755774Z         <obj>['qsort'] -> callable object
2022-02-20T15:57:32.0756100Z     
2022-02-20T15:57:32.0756586Z         Calling the functions releases the Python GIL during the call and
2022-02-20T15:57:32.0757166Z         reacquires it afterwards.
2022-02-20T15:57:32.0757567Z         """
2022-02-20T15:57:32.0758054Z         _func_flags_ = _FUNCFLAG_CDECL
2022-02-20T15:57:32.0758600Z         _func_restype_ = c_int
2022-02-20T15:57:32.0759054Z         # default values for repr
2022-02-20T15:57:32.0759992Z         _name = '<uninitialized>'
2022-02-20T15:57:32.0760343Z         _handle = 0
2022-02-20T15:57:32.0760645Z         _FuncPtr = None
2022-02-20T15:57:32.0760916Z     
2022-02-20T15:57:32.0761741Z         def __init__(self, name, mode=DEFAULT_MODE, handle=None,
2022-02-20T15:57:32.0762492Z                      use_errno=False,
2022-02-20T15:57:32.0763126Z                      use_last_error=False,
2022-02-20T15:57:32.0763760Z                      winmode=None):
2022-02-20T15:57:32.0764377Z             self._name = name
2022-02-20T15:57:32.0764995Z             flags = self._func_flags_
2022-02-20T15:57:32.0765437Z             if use_errno:
2022-02-20T15:57:32.0765843Z                 flags |= _FUNCFLAG_USE_ERRNO
2022-02-20T15:57:32.0766345Z             if use_last_error:
2022-02-20T15:57:32.0766773Z                 flags |= _FUNCFLAG_USE_LASTERROR
2022-02-20T15:57:32.0767297Z             if _sys.platform.startswith("aix"):
2022-02-20T15:57:32.0767863Z                 """When the name contains ".a(" and ends with ")",
2022-02-20T15:57:32.0768632Z                    e.g., "libFOO.a(libFOO.so)" - this is taken to be an
2022-02-20T15:57:32.0769148Z                    archive(member) syntax for dlopen(), and the mode is adjusted.
2022-02-20T15:57:32.0769643Z                    Otherwise, name is presented to dlopen() as a file argument.
2022-02-20T15:57:32.0770105Z                 """
2022-02-20T15:57:32.0770608Z                 if name and name.endswith(")") and ".a(" in name:
2022-02-20T15:57:32.0771194Z                     mode |= ( _os.RTLD_MEMBER | _os.RTLD_NOW )
2022-02-20T15:57:32.0771607Z             if _os.name == "nt":
2022-02-20T15:57:32.0772018Z                 if winmode is not None:
2022-02-20T15:57:32.0772437Z                     mode = winmode
2022-02-20T15:57:32.0772770Z                 else:
2022-02-20T15:57:32.0773108Z                     import nt
2022-02-20T15:57:32.0773503Z                     mode = nt._LOAD_LIBRARY_SEARCH_DEFAULT_DIRS
2022-02-20T15:57:32.0774176Z                     if '/' in name or '\\' in name:
2022-02-20T15:57:32.0774608Z                         self._name = nt._getfullpathname(self._name)
2022-02-20T15:57:32.0775067Z                         mode |= nt._LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR
2022-02-20T15:57:32.0775444Z     
2022-02-20T15:57:32.0775788Z             class _FuncPtr(_CFuncPtr):
2022-02-20T15:57:32.0776168Z                 _flags_ = flags
2022-02-20T15:57:32.0776551Z                 _restype_ = self._func_restype_
2022-02-20T15:57:32.0776948Z             self._FuncPtr = _FuncPtr
2022-02-20T15:57:32.0777280Z     
2022-02-20T15:57:32.0777606Z             if handle is None:
2022-02-20T15:57:32.0777995Z >               self._handle = _dlopen(self._name, mode)
2022-02-20T15:57:32.0778458Z E               OSError: dlopen: cannot load any more object with static TLS
2022-02-20T15:57:32.0778711Z

and then

E               lightgbm.basic.LightGBMError: scikit-learn is required for lightgbm.sklearn. You must install scikit-learn and restart your session to use this module.

https://dev.azure.com/lightgbm-ci/lightgbm-ci/_build/results?buildId=12253&view=logs&j=fb919173-48b9-522d-5342-9a59a13eb10b&t=7fcbeee5-d4c1-571c-8072-1d5874591254

Refer to #4893 (comment) for the root cause.

@StrikerRUS StrikerRUS marked this pull request as ready for review February 21, 2022 22:59
jameslamb
jameslamb approved these changes Feb 21, 2022
View reviewed changes
Copy link
Collaborator

@jameslamb jameslamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for fixing this!

@shiyu1994
Copy link
Collaborator

Seems that the test on Windows has failed.

[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.002466 seconds.
You can set `force_col_wise=true` to remove the overhead.
[10]	training's l1: 0.457448	training's l2: 0.217995	valid_1's l1: 0.456464	valid_1's l2: 0.21641
[20]	training's l1: 0.436869	training's l2: 0.205099	valid_1's l1: 0.434057	valid_1's l2: 0.201616
[30]	training's l1: 0.421302	training's l2: 0.197421	valid_1's l1: 0.417019	valid_1's l2: 0.192514
[40]	training's l1: 0.411107	training's l2: 0.192856	valid_1's l1: 0.406303	valid_1's l2: 0.187258
[50]	training's l1: 0.403695	training's l2: 0.189593	valid_1's l1: 0.398997	valid_1's l2: 0.183688
Traceback (most recent call last):
  File "D:\a\1\s\examples\python-guide\plot_example.py", line 69, in <module>
    ax = lgb.plot_tree(gbm, tree_index=53, figsize=(15, 15), show_info=['split_gain'])
  File "C:\Miniconda\envs\test-env\lib\site-packages\lightgbm\plotting.py", line 679, in plot_tree
    graph = create_tree_digraph(booster=booster, tree_index=tree_index,
  File "C:\Miniconda\envs\test-env\lib\site-packages\lightgbm\plotting.py", line 597, in create_tree_digraph
    graph = _to_graphviz(tree_info, show_info, feature_names, precision,
  File "C:\Miniconda\envs\test-env\lib\site-packages\lightgbm\plotting.py", line 434, in _to_graphviz
    raise ImportError('You must install graphviz and restart your session to plot tree.')
ImportError: You must install graphviz and restart your session to plot tree.
[60]	training's l1: 0.398704	training's l2: 0.187043	valid_1's l1: 0.393977	valid_1's l2: 0.181009
[70]	training's l1: 0.394876	training's l2: 0.184982	valid_1's l1: 0.389805	valid_1's l2: 0.178803
[80]	training's l1: 0.391147	training's l2: 0.1828	valid_1's l1: 0.386476	valid_1's l2: 0.176799
[90]	training's l1: 0.388101	training's l2: 0.180817	valid_1's l1: 0.384404	valid_1's l2: 0.175775
[100]	training's l1: 0.385174	training's l2: 0.179171	valid_1's l1: 0.382929	valid_1's l2: 0.175321
Plotting metrics recorded during training...
D:\a\1\s\examples\python-guide\plot_example.py:58: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show()
Plotting feature importances...
D:\a\1\s\examples\python-guide\plot_example.py:62: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show()
Plotting split value histogram...
D:\a\1\s\examples\python-guide\plot_example.py:66: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show()
Plotting 54th tree...
##[error]Cmd.exe exited with code '-1'.
Finishing: Test

@jameslamb
Copy link
Collaborator

I think that failed Windows test was the result of a network interruption during the build which resulted in some package installations failing.

Collecting package metadata (repodata.json): ...working... failed

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "C:\Miniconda\lib\site-packages\urllib3\response.py", line 697, in _update_chunk_length
        self.chunk_left = int(line, 16)
    ValueError: invalid literal for int() with base 16: b''
full error logs (click me) 
Collecting package metadata (repodata.json): ...working... failed

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "C:\Miniconda\lib\site-packages\urllib3\response.py", line 697, in _update_chunk_length
        self.chunk_left = int(line, 16)
    ValueError: invalid literal for int() with base 16: b''
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "C:\Miniconda\lib\site-packages\urllib3\response.py", line 438, in _error_catcher
        yield
      File "C:\Miniconda\lib\site-packages\urllib3\response.py", line 764, in read_chunked
        self._update_chunk_length()
      File "C:\Miniconda\lib\site-packages\urllib3\response.py", line 701, in _update_chunk_length
        raise InvalidChunkLength(self, line)
    urllib3.exceptions.InvalidChunkLength: InvalidChunkLength(got length b'', 0 bytes read)
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "C:\Miniconda\lib\site-packages\requests\models.py", line 760, in generate
        for chunk in self.raw.stream(chunk_size, decode_content=True):

Not sure why the build kept running after that point instead of breaking right there.

I'll close and immediately re-open this PR to re-trigger CI.

@jameslamb jameslamb closed this Feb 22, 2022
@jameslamb jameslamb reopened this Feb 22, 2022
@shiyu1994
Copy link
Collaborator

It is weird that the mai pip test now should fail.
https://dev.azure.com/lightgbm-ci/lightgbm-ci/_build/results?buildId=12264&view=logs&j=11c46b0e-4322-5d4e-fafd-abe3157be7c4&t=813cdb27-ab11-56e8-5950-18a90a0620ef
But it succeeded in previous run.
https://dev.azure.com/lightgbm-ci/lightgbm-ci/_build/results?buildId=12259&view=logs&j=11c46b0e-4322-5d4e-fafd-abe3157be7c4

@shiyu1994
Copy link
Collaborator

Close and re-open to re-trigger CI.

@shiyu1994 shiyu1994 closed this Feb 22, 2022
@shiyu1994 shiyu1994 reopened this Feb 22, 2022
@StrikerRUS StrikerRUS merged commit 406bc7d into master Feb 22, 2022
@StrikerRUS StrikerRUS deleted the fix_ci branch February 22, 2022 23:33
@jameslamb jameslamb mentioned this pull request Feb 23, 2022
Merged
@StrikerRUS StrikerRUS mentioned this pull request Jun 12, 2022
Merged
@jameslamb jameslamb mentioned this pull request Aug 3, 2022
Open
@jameslamb jameslamb mentioned this pull request Oct 7, 2022
Closed
40 tasks
@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@jameslamb jameslamb jameslamb approved these changes

Assignees
No one assigned
Labels
maintenance
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@StrikerRUS @shiyu1994 @jameslamb