See this page for details on installing Python and the minimum libraries required for your development environment: Install Python

Create and activate a virtual environment for your project directory::
$ mkdir -p /home/mtitek/dev/llm $ cd /home/mtitek/dev/llm $ python -m venv .env $ source .env/bin/activate
If at any time you want to deactivate the virtual environment in your project directory and clean it up, run the following commands:
$ cd /home/mtitek/dev/llm $ deactivate $ rm -rf .env

See this page for more details: https://huggingface.co/docs/huggingface_hub/main/en/guides/cli

To install the Hugging Face CLI package:
$ pip install "huggingface_hub[cli]"
To test the installation of the Hugging Face CLI package:
$ huggingface-cli version huggingface_hub version: 0.30.2
You can use the CLI to download models:
$ huggingface-cli download microsoft/DialoGPT-small

See this page for more details: https://huggingface.co/docs/transformers/en/installation

Install Transformers with pip in your newly created virtual environment:
$ pip install transformers
To install a CPU-only version of Transformers and a machine learning framework PyTorch, run the following command: $ pip install 'transformers[torch]'
To test if the installation was successful, run the following command. It should return a label and a score for the provided text: $ python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('hugging face is the best'))" [{'label': 'POSITIVE', 'score': 0.999839186668396}]

Example:
from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("microsoft/DialoGPT-small") tokenizer = AutoTokenizer.from_pretrained("microsoft/DialoGPT-small") print(model) print(tokenizer)
Output:
GPT2TokenizerFast( name_or_path='microsoft/DialoGPT-small', vocab_size=50257, model_max_length=1024, is_fast=True, padding_side='right', truncation_side='right', special_tokens={'bos_token': '<|endoftext|>', 'eos_token': '<|endoftext|>', 'unk_token': '<|endoftext|>'}, clean_up_tokenization_spaces=True, added_tokens_decoder={50256: AddedToken("<|endoftext|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=True),} ) GPT2LMHeadModel( (transformer): GPT2Model( (wte): Embedding(50257, 1024) (wpe): Embedding(1024, 1024) (drop): Dropout(p=0.1, inplace=False) (h): ModuleList( (0-23): 24 x GPT2Block( (ln_1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True) (attn): GPT2Attention( (c_attn): Conv1D(nf=3072, nx=1024) (c_proj): Conv1D(nf=1024, nx=1024) (attn_dropout): Dropout(p=0.1, inplace=False) (resid_dropout): Dropout(p=0.1, inplace=False) ) (ln_2): LayerNorm((1024,), eps=1e-05, elementwise_affine=True) (mlp): GPT2MLP( (c_fc): Conv1D(nf=4096, nx=1024) (c_proj): Conv1D(nf=1024, nx=4096) (act): NewGELUActivation() (dropout): Dropout(p=0.1, inplace=False) ) ) ) (ln_f): LayerNorm((1024,), eps=1e-05, elementwise_affine=True) ) (lm_head): Linear(in_features=1024, out_features=50257, bias=False) )

See these pages for more details:
https://pypi.org/project/llama-cpp-python/
https://python.langchain.com/docs/integrations/llms/llamacpp/

To install the llama-cpp-python package: $ pip install llama-cpp-python

Download this model:
$ wget https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf
Python code:
from llama_cpp import Llama model = Llama(model_path="Phi-3-mini-4k-instruct-q4.gguf")
Output:
llama_model_loader: loaded meta data with 24 key-value pairs and 195 tensors from Phi-3-mini-4k-instruct-q4.gguf (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = phi3 ...

See this page for more details: https://python.langchain.com/docs/how_to/installation/

To install the main LangChain package:
$ pip install langchain
To install the LangChain core package:
$ pip install langchain-core
To install the LangChain community package:
$ pip install langchain-community
To install the LangChain command Line Interface (CLI) package:
$ pip install langchain-cli
To test the installation of the LangChain CLI package:
$ langchain-cli --version langchain-cli 0.0.36