快速开始与安装

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

⚠️ Note that this file is in Markdown but contains specific syntax for our doc-builder (similar to MDX) that may not be rendered properly in your Markdown viewer.

-->

Installation

Transformers works with PyTorch. It has been tested on Python 3.10+ and PyTorch 2.5+.

Virtual environment

uv is an extremely fast Rust-based Python package and project manager and requires a virtual environment by default to manage different projects and avoids compatibility issues between dependencies.

It can be used as a drop-in replacement for pip, but if you prefer to use pip, remove uv from the commands below.

[!TIP] Refer to the uv installation docs to install uv.

Create a virtual environment to install Transformers in.

uv venv .env
source .env/bin/activate

Python

Install Transformers with the following command.

uv is a fast Rust-based Python package and project manager.

To install Transformers with PyTorch for NVIDIA GPU (CUDA), install the appropriate CUDA drivers for PyTorch.

Run the command below to check if your system detects an NVIDIA GPU.

nvidia-smi

uv pip install "transformers[torch]"

</hfoption> <hfoption id="NVIDIA Spark (ARM64)">

To install Transformers with PyTorch on NVIDIA Spark devices (such as an RTX Spark laptop) running ARM64, install PyTorch from the NVIDIA PyPI index. These devices require NVIDIA's ARM64 builds of PyTorch, which are not available on the default PyPI index or the standard PyTorch wheel index.

Run the command below to check if your system detects an NVIDIA GPU.

nvidia-smi

Install PyTorch from the NVIDIA PyPI index, then install Transformers.

uv pip install torch --index-url https://pypi.nvidia.com
uv pip install transformers

</hfoption> <hfoption id="CPU">

To install a CPU-only version of Transformers, run the following command.

uv pip install torch --index-url https://download.pytorch.org/whl/cpu
uv pip install transformers

</hfoption> <hfoption id="Intel GPU (XPU)">

To install Transformers with PyTorch for Intel GPU (XPU), install the appropriate Intel GPU (XPU) drivers for PyTorch and add the Intel GPU (XPU) PyTorch index URL.

After installing the drivers, run the command below to check if your system detects an Intel GPU.

xpu-smi

uv pip install "transformers[torch]" --extra-index-url https://download.pytorch.org/whl/xpu

</hfoption> </hfoptions>

Test whether the install was successful with the following command. It should return a label and score for the provided text.

python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('hugging face is the best'))"
[{'label': 'POSITIVE', 'score': 0.9998704791069031}]

Source install

Installing from source installs the latest version rather than the stable version of the library. It ensures you have the most up-to-date changes in Transformers and it's useful for experimenting with the latest features or fixing a bug that hasn't been officially released in the stable version yet.

The downside is that the latest version may not always be stable. If you encounter any problems, please open a GitHub Issue so we can fix it as soon as possible.

Install from source with the following command.

uv pip install git+https://github.com/huggingface/transformers

Check if the install was successful with the command below. It should return a label and score for the provided text.

python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('hugging face is the best'))"
[{'label': 'POSITIVE', 'score': 0.9998704791069031}]

Editable install

An editable install is useful if you're developing locally with Transformers. It links your local copy of Transformers to the Transformers repository instead of copying the files. The files are added to Python's import path.

git clone https://github.com/huggingface/transformers.git
cd transformers
uv pip install -e .

[!WARNING] You must keep the local Transformers folder to keep using it.

Update your local version of Transformers with the latest changes in the main repository with the following command.

cd ~/transformers/
git pull

conda

conda is a language-agnostic package manager. Install Transformers from the conda-forge channel in your newly created virtual environment.

conda install conda-forge::transformers

Set up

After installation, you can configure the Transformers cache location or set up the library for offline usage.

Cache directory

When you load a pretrained model with [~PreTrainedModel.from_pretrained], the model is downloaded from the Hub and locally cached.

If you pass a commit hash, Transformers uses the local cache for that commit's files and does not re-check the Hub for each one (including files that are known to be missing). This is the default behavior when no revision is specified.

If you pass a branch or tag, Transformers contacts the Hub once at the start of the load to pick the commit, then uses the cache the same way for that commit's files.

The default directory given by the shell environment variable HF_HUB_CACHE is ~/.cache/huggingface/hub. On Windows, the default directory is C:\Users\username\.cache\huggingface\hub.

Cache a model in a different directory by changing the path in the following shell environment variables (listed by priority).

HF_HUB_CACHE (default)
HF_HOME
XDG_CACHE_HOME + /huggingface (only if HF_HOME is not set)

Offline mode

To use Transformers in an offline or firewalled environment requires the downloaded and cached files ahead of time. Download a model repository from the Hub with the [~huggingface_hub.snapshot_download] method.

[!TIP] Refer to the Download files from the Hub guide for more options for downloading files from the Hub. You can download files from specific revisions, download from the CLI, and even filter which files to download from a repository.

from huggingface_hub import snapshot_download

snapshot_download(repo_id="meta-llama/Llama-2-7b-hf", repo_type="model")

Set the environment variable HF_HUB_OFFLINE=1 to prevent HTTP calls to the Hub when loading a model.

HF_HUB_OFFLINE=1 \
python examples/pytorch/language-modeling/run_clm.py --model_name_or_path meta-llama/Llama-2-7b-hf --dataset_name Salesforce/wikitext ...

Another option for only loading cached files is to set local_files_only=True in [~PreTrainedModel.from_pretrained].

from transformers import LlamaForCausalLM

model = LlamaForCausalLM.from_pretrained("./path/to/local/directory", local_files_only=True)

Offline mode (or local_files_only=True) can still turn a branch or tag into a commit if an earlier online load saved that mapping in the cache. If the mapping was never saved, Transformers keeps the branch or tag you asked for and continues with the regular offline load. You get the same cache hits or missing-file errors as a normal offline load.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

⚠️ Note that this file is in Markdown but contains specific syntax for our doc-builder (similar to MDX) that may not be rendered properly in your Markdown viewer. -->

Transformers

Transformers acts as the model-definition framework for state-of-the-art machine learning models in text, computer vision, audio, video, and multimodal models, for both inference and training.

It centralizes the model definition so that this definition is agreed upon across the ecosystem. transformers is the pivot across frameworks: if a model definition is supported, it will be compatible with the majority of training frameworks (Axolotl, Unsloth, DeepSpeed, FSDP, PyTorch-Lightning, ...), inference engines (vLLM, SGLang, TGI, ...), and adjacent modeling libraries (llama.cpp, mlx, ...) which leverage the model definition from transformers.

We pledge to help support new state-of-the-art models and democratize their usage by having their model definition be simple, customizable, and efficient.

There are over 1M+ Transformers model checkpoints on the Hugging Face Hub you can use.

Explore the Hub today to find a model and use Transformers to help you get started right away.

Explore the Models Timeline to discover the latest text, vision, audio and multimodal model architectures in Transformers.

Features

Transformers provides everything you need for inference or training with state-of-the-art pretrained models. Some of the main features include:

Pipeline: Simple and optimized inference class for many machine learning tasks like text generation, image segmentation, automatic speech recognition, document question answering, and more.
Trainer: A comprehensive trainer that supports features such as mixed precision, torch.compile, and FlashAttention for training and distributed training for PyTorch models.
generate: Fast text generation with large language models (LLMs) and vision language models (VLMs), including support for streaming and multiple decoding strategies.

Design

[!TIP] Read our Philosophy to learn more about Transformers' design principles.

Transformers is designed for developers and machine learning engineers and researchers. Its main design principles are:

Fast and easy to use: Every model is implemented from only three main classes (configuration, model, and preprocessor) and can be quickly used for inference or training with [Pipeline] or [Trainer].
Pretrained models: Reduce your carbon footprint, compute cost and time by using a pretrained model instead of training an entirely new one. Each pretrained model is reproduced as closely as possible to the original model and offers state-of-the-art performance.

Learn

If you're new to Transformers or want to learn more about transformer models, we recommend starting with the LLM course. This comprehensive course covers everything from the fundamentals of how transformer models work to practical applications across various tasks. You'll learn the complete workflow, from curating high-quality datasets to fine-tuning large language models and implementing reasoning capabilities. The course contains both theoretical and hands-on exercises to build a solid foundational knowledge of transformer models as you learn.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

⚠️ Note that this file is in Markdown but contains specific syntax for our doc-builder (similar to MDX) that may not be rendered properly in your Markdown viewer.

-->

Accelerator selection

You can control which accelerators (CUDA, XPU, MPS, HPU, etc.) PyTorch sees and in what order during distributed training. Prioritize faster devices or limit training to a subset of available hardware. It works with both DistributedDataParallel and DataParallel, and doesn't require Accelerate or the DeepSpeed integration.

Order of accelerators

Use the hardware-specific environment variable to select accelerators and set their order. Set it on the command line per run, or add it to ~/.bashrc or another startup config file.

[!WARNING] Avoid exporting environment variables because if you forget how an environment variable was set up, you may silently train on the wrong accelerators. Set the environment variable on the same command line as the training run.

For example, to select accelerators 0 and 2 out of four:

CUDA_VISIBLE_DEVICES=0,2 torchrun trainer-program.py ...

PyTorch sees only GPUs 0 and 2, which are mapped to cuda:0 and cuda:1. To reverse the order (use GPU 2 as cuda:0 and GPU 0 as cuda:1):

CUDA_VISIBLE_DEVICES=2,0 torchrun trainer-program.py ...

To run without any GPUs:

CUDA_VISIBLE_DEVICES= python trainer-program.py ...

Control the order of CUDA devices with CUDA_DEVICE_ORDER.

Order by PCIe bus ID (matches nvidia-smi):

export CUDA_DEVICE_ORDER=PCI_BUS_ID

Order by compute capability (fastest first):

export CUDA_DEVICE_ORDER=FASTEST_FIRST

</hfoption> <hfoption id="Intel XPU">

ZE_AFFINITY_MASK=0,2 torchrun trainer-program.py ...

PyTorch sees only XPUs 0 and 2, which are mapped to xpu:0 and xpu:1. To reverse the order (use XPU 2 as xpu:0 and XPU 0 as xpu:1):

ZE_AFFINITY_MASK=2,0 torchrun trainer-program.py ...

Control the order of Intel XPUs with:

export ZE_ENABLE_PCI_ID_DEVICE_ORDER=1

For more on device enumeration and sorting on Intel XPU, see the Level Zero documentation.

</hfoption> </hfoptions>