This chapter presents a set of best practices for setting up your environment. To ensure a smooth experience with the notebooks in the subsequent chapters, it is strongly recommended that you follow the corresponding steps below to configure your environment properly.
First of all, choose a proper system. Here's a list of recommended hardware and OS.
⚠️ Hardware
- Intel PCs, at least 16GB RAM.
- Servers equipped with Intel® Xeon® processors, at least 32G RAM.
⚠️ Operating System
- Ubuntu 20.04 or later
- CentOS 7 or later
- Windows 10/11, with or without WSL
Next, use a python environment management tool (we recommend using Conda) to create a python enviroment and install necessary libs.
Follow the instructions corresponding to your OS below.
For Linux users, open a terminal and run below commands.
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash ./Miniconda3-latest-Linux-x86_64.sh
conda initNote Follow the instructions popped up on the console until conda initialization finished successfully.
For Windows users, download conda installer here and execute it.
After the installation finished, open "Anaconda Powershell Prompt (Miniconda3)" for following steps.
For WSL users, ensure you have already installed WSL2. If not, refer to here for how to install.
Open a WSL2 shell and run the same commands as in 2.2.1.1 Linux section.
Note Python 3.9 is recommended for running IPEX-LLM.
Create a Python 3.9 environment with the name you choose, for example llm-tutorial:
conda create -n llm-tutorial python=3.9
Then activate the environment llm-tutorial:
conda activate llm-tutorial
The one-line command below will install the latest ipex-llm with all the dependencies for common LLM application development.
pip install --pre --upgrade ipex-llm[all]
The jupyter library is required for running the tutorial notebooks (i.e. the .ipynb files). Under your activated Python 3.9 environment, run:
pip install jupyter
The recommended command to start jupyter service is slightly different on PC and server.
On PC, just run the command in shell:
jupyter notebook
On server, it is recommended to use all physical cores of a single socket for better performance. So run below command instead:
# e.g. for a server with 48 cores per socket
export OMP_NUM_THREADS=48
numactl -C 0-47 -m 0 jupyter notebookCongratulations! Now you can use a web browser to access the jupyter service url and execute the notebooks provided in this tutorial.
If you're new to LLMs and LLM applicaiton development, there's something you might want to know.
To start, you'll need to obtain a model. There are numerous open-source LLMs available in the community. If you don't have a specific target in mind, consider selecting one that ranks higher on LLM leaderboards. These leaderboards evaluate and compare the capabilities of various LLMs. For instance,
- Open LLM LeaderBoard hosted by Huggingface.
- Chatbot Arena Leaderboard hosted by llmsys.
Most of these leaderboards include reference links to the models listed. If a model is open source, you can easily download it directly from the provided link and give it a try.
As of writing, many popular LLMs are hosted on Huggingface. An example model homepage hosted on huggingface looks like this.
To download models from huggingface, you can either use git or huggingface provided APIs. Refer to Download Model from Huggingface for details about how to download models.
Usually, the models downloaded from Huggingface can be loaded using Huggingface Transformers library API. IPEX-LLM provides APIs to easily work with such models. Read the following chapters to to find out more.