Hugging Face AutoModel and AutoTokenizer

AutoModel and AutoTokenizer are the Hugging Face classes that load the correct architecture for any model ID. You do not need to know whether a checkpoint is BERT, RoBERTa or Llama: the Auto classes read config.json and pick the right Python class for you.

Using them directly, instead of a pipeline, gives you full control over inputs, outputs and performance.

The from_pretrained Method

from transformers import AutoTokenizer, AutoModel

name = "distilbert/distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModel.from_pretrained(name)

inputs = tokenizer("Hello there", return_tensors="pt")
outputs = model(**inputs)
print(outputs.last_hidden_state.shape)

torch.Size([1, 4, 768])

The base AutoModel returns hidden states: one 768-number vector for each of the 4 tokens ([CLS], hello, there, [SEP]). These vectors are useful features but not yet predictions.

Task-Specific Heads

To get predictions, load a class with a "head" for your task on top of the base model:

AutoModelForSequenceClassification: one label per text
AutoModelForTokenClassification: one label per token (NER)
AutoModelForQuestionAnswering: start and end positions of an answer
AutoModelForCausalLM: next-token prediction for GPT-style text generation
AutoModelForSeq2SeqLM: encoder-decoder models like T5 and BART

Classification by Hand

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

name = "distilbert/distilbert-base-uncased-finetuned-sst-2-english"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(name)

inputs = tok(["Great value!", "Waste of money."], padding=True, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits

probs = logits.softmax(dim=-1)
for p in probs:
    label_id = int(p.argmax())
    print(model.config.id2label[label_id], round(float(p.max()), 3))

POSITIVE 0.999
NEGATIVE 0.998

logits has shape [2, 2]: two sentences, two classes. softmax turns logits into probabilities, and config.id2label maps the class index back to a readable name. torch.no_grad() skips gradient tracking, which saves memory when you are only predicting.

Precision and Device Options

Large models are usually loaded in half precision and placed on the GPU automatically. In transformers v5 the argument is dtype (older code uses torch_dtype, which still works with a deprecation warning).

from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-0.5B-Instruct",
    dtype="auto",        # use the precision stored in the checkpoint, e.g. bfloat16
    device_map="auto",   # put layers on GPU(s), spill to CPU if needed
)
print(model.dtype, model.device)

torch.bfloat16 cuda:0   (sample — depends on your hardware)

Saving a Model Locally

model.save_pretrained("./my-model")
tok.save_pretrained("./my-model")
# later
model = AutoModelForSequenceClassification.from_pretrained("./my-model")

./my-model/config.json, model.safetensors, tokenizer.json, ... are written

Frequently asked questions

What is the difference between AutoModel and AutoModelForCausalLM?

AutoModel loads only the base network and returns hidden states. AutoModelForCausalLM adds a language-modeling head that predicts the next token, which is what you need for text generation.

What does from_pretrained do?

It downloads (or reads from cache) a model's config and weights, builds the correct architecture and loads the weights into it, ready to use.

Should I use torch_dtype or dtype?

Use dtype in current transformers releases. torch_dtype is the older name and is kept for backward compatibility.

When should I use a pipeline instead of AutoModel?

Use a pipeline for quick prototypes and standard tasks. Use the Auto classes when you need custom batching, access to logits or hidden states, or plan to fine-tune.