"No module named 'fastllama.api' " after pip installation

[cosimoiaia]

I installed it following the README with:
$ pip3 install git+https://github.com/PotatoSpudowski/fastLLaMa.git@main

Installation went fine but when I try to use it:

$ iPython3
Python 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.13.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from fastllama import Model

ModuleNotFoundError Traceback (most recent call last)
Cell In[1], line 1
----> 1 from fastllama import Model

File /usr/local/lib/python3.10/dist-packages/fastllama/init.py:2
1 import os;
----> 2 from .api import *
4 set_library_path(os.path.dirname(os.path.abspath(file)))

ModuleNotFoundError: No module named 'fastllama.api'

[PotatoSpudowski]

Can you please let me know your OS and Hardware?

[einarsApinis]

Same issue: windows 10, NVIDIA GeForce RTX 3060 12gb, 16gb ram, intel processor

[amitsingh19975]

I'm sorry for not getting back to you sooner. I was busy with Webui.

We do not support Windows right now, but we hope to do so in the future. So, for right now, I recommend using WSL.
If you're not using WSL, pip does not build the library because of which this error might occur.

If none of this is true, let us know.

[Celppu]

I get same error in wsl 2

[amitsingh19975]

Could you send me the logs that are generated when installed with pip? Also, could you go inside that Python site-packages and share the file list?

[jhud]

I get the following errors from my make install

  In file included from /tmp/pip-req-build-es8sj3p_/include/llama.hpp:12,
                   from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/logger.hpp: In constructor ‘fastllama::NullLogger::NullLogger()’:
  /tmp/pip-req-build-es8sj3p_/include/logger.hpp:93:31: error: use of deleted function ‘constexpr fastllama::DefaultLogger::DefaultLogger()’
     93 |         NullLogger() noexcept {
        |                               ^
  In file included from /tmp/pip-req-build-es8sj3p_/include/llama.hpp:12,
                   from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/logger.hpp:40:9: note: ‘constexpr fastllama::DefaultLogger::DefaultLogger() noexcept’ is implicitly deleted because its exception-specification does not match the implicit exception-specification ‘noexcept (false)’
     40 |         DefaultLogger() noexcept = default;
        |         ^~~~~~~~~~~~~
  In file included from /tmp/pip-req-build-es8sj3p_/include/llama.hpp:19,
                   from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/tensor/utils.hpp: At global scope:
  /tmp/pip-req-build-es8sj3p_/include/tensor/utils.hpp:212:108: error: use of deleted function ‘constexpr fastllama::Logger::Logger()’
    212 |         bool initializeTensors(ggml_context* ctx, HyperParams const& params, Logger const& logger = Logger{});
        |                                                                                                            ^
  In file included from /tmp/pip-req-build-es8sj3p_/include/llama.hpp:12,
                   from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/logger.hpp:105:9: note: ‘constexpr fastllama::Logger::Logger() noexcept’ is implicitly deleted because its exception-specification does not match the implicit exception-specification ‘noexcept (false)’
    105 |         Logger() noexcept = default;
        |         ^~~~~~
  In file included from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/llama.hpp:106:76: error: use of deleted function ‘constexpr fastllama::Logger::Logger()’
    106 |         bool init(HyperParams const& params, Logger const& logger = Logger{});
        |                                                                            ^
  /tmp/pip-req-build-es8sj3p_/include/llama.hpp:107:51: error: use of deleted function ‘constexpr fastllama::Logger::Logger()’
    107 |         void deinit(Logger const& logger = Logger{});
        |                                                   ^
  /tmp/pip-req-build-es8sj3p_/include/llama.hpp:181:23: error: use of deleted function ‘constexpr fastllama::Logger::Logger()’
    181 |         Logger logger{};
        |                       ^

[amitsingh19975]

I get the following errors from my make install

  In file included from /tmp/pip-req-build-es8sj3p_/include/llama.hpp:12,
                   from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/logger.hpp: In constructor ‘fastllama::NullLogger::NullLogger()’:
  /tmp/pip-req-build-es8sj3p_/include/logger.hpp:93:31: error: use of deleted function ‘constexpr fastllama::DefaultLogger::DefaultLogger()’
     93 |         NullLogger() noexcept {
        |                               ^
  In file included from /tmp/pip-req-build-es8sj3p_/include/llama.hpp:12,
                   from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/logger.hpp:40:9: note: ‘constexpr fastllama::DefaultLogger::DefaultLogger() noexcept’ is implicitly deleted because its exception-specification does not match the implicit exception-specification ‘noexcept (false)’
     40 |         DefaultLogger() noexcept = default;
        |         ^~~~~~~~~~~~~
  In file included from /tmp/pip-req-build-es8sj3p_/include/llama.hpp:19,
                   from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/tensor/utils.hpp: At global scope:
  /tmp/pip-req-build-es8sj3p_/include/tensor/utils.hpp:212:108: error: use of deleted function ‘constexpr fastllama::Logger::Logger()’
    212 |         bool initializeTensors(ggml_context* ctx, HyperParams const& params, Logger const& logger = Logger{});
        |                                                                                                            ^
  In file included from /tmp/pip-req-build-es8sj3p_/include/llama.hpp:12,
                   from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/logger.hpp:105:9: note: ‘constexpr fastllama::Logger::Logger() noexcept’ is implicitly deleted because its exception-specification does not match the implicit exception-specification ‘noexcept (false)’
    105 |         Logger() noexcept = default;
        |         ^~~~~~
  In file included from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/llama.hpp:106:76: error: use of deleted function ‘constexpr fastllama::Logger::Logger()’
    106 |         bool init(HyperParams const& params, Logger const& logger = Logger{});
        |                                                                            ^
  /tmp/pip-req-build-es8sj3p_/include/llama.hpp:107:51: error: use of deleted function ‘constexpr fastllama::Logger::Logger()’
    107 |         void deinit(Logger const& logger = Logger{});
        |                                                   ^
  /tmp/pip-req-build-es8sj3p_/include/llama.hpp:181:23: error: use of deleted function ‘constexpr fastllama::Logger::Logger()’
    181 |         Logger logger{};
        |                       ^

Update the compiler.

[jhud]

I get the following errors from my make install

  In file included from /tmp/pip-req-build-es8sj3p_/include/llama.hpp:12,
                   from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/logger.hpp: In constructor ‘fastllama::NullLogger::NullLogger()’:
  /tmp/pip-req-build-es8sj3p_/include/logger.hpp:93:31: error: use of deleted function ‘constexpr fastllama::DefaultLogger::DefaultLogger()’
     93 |         NullLogger() noexcept {
        |                               ^
  In file included from /tmp/pip-req-build-es8sj3p_/include/llama.hpp:12,
                   from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/logger.hpp:40:9: note: ‘constexpr fastllama::DefaultLogger::DefaultLogger() noexcept’ is implicitly deleted because its exception-specification does not match the implicit exception-specification ‘noexcept (false)’
     40 |         DefaultLogger() noexcept = default;
        |         ^~~~~~~~~~~~~
  In file included from /tmp/pip-req-build-es8sj3p_/include/llama.hpp:19,
                   from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/tensor/utils.hpp: At global scope:
  /tmp/pip-req-build-es8sj3p_/include/tensor/utils.hpp:212:108: error: use of deleted function ‘constexpr fastllama::Logger::Logger()’
    212 |         bool initializeTensors(ggml_context* ctx, HyperParams const& params, Logger const& logger = Logger{});
        |                                                                                                            ^
  In file included from /tmp/pip-req-build-es8sj3p_/include/llama.hpp:12,
                   from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/logger.hpp:105:9: note: ‘constexpr fastllama::Logger::Logger() noexcept’ is implicitly deleted because its exception-specification does not match the implicit exception-specification ‘noexcept (false)’
    105 |         Logger() noexcept = default;
        |         ^~~~~~
  In file included from /tmp/pip-req-build-es8sj3p_/lib/llama.cpp:1:
  /tmp/pip-req-build-es8sj3p_/include/llama.hpp:106:76: error: use of deleted function ‘constexpr fastllama::Logger::Logger()’
    106 |         bool init(HyperParams const& params, Logger const& logger = Logger{});
        |                                                                            ^
  /tmp/pip-req-build-es8sj3p_/include/llama.hpp:107:51: error: use of deleted function ‘constexpr fastllama::Logger::Logger()’
    107 |         void deinit(Logger const& logger = Logger{});
        |                                                   ^
  /tmp/pip-req-build-es8sj3p_/include/llama.hpp:181:23: error: use of deleted function ‘constexpr fastllama::Logger::Logger()’
    181 |         Logger logger{};
        |                       ^

Update the compiler.

Thanks! I was using standard GCC-9 Ubuntu 22.04. I used sudo apt install g++-12 with update-alternatives and now it builds and runs fine.

[zhiyixu]

is fastllama support windows now?

[Tsunami014]

no, or at least not for me yet, I'm getting the same issue still