Introduction

For ordinary sentence multi-class classification, BertForSequenceClassification is the right model class. BertForMultipleChoice is for a different problem shape: each training example contains several candidate answers, and the model chooses one option from that set.

What BertForSequenceClassification Expects

BertForSequenceClassification takes one encoded sequence per example and predicts one label from num_labels classes.

That matches tasks like:

• sentiment classification
• topic classification
• intent detection
• toxic versus non-toxic labeling
• any single-sentence or sentence-pair classification problem with a fixed label set

A minimal example looks like this:

1from transformers import AutoTokenizer, BertForSequenceClassification
2import torch
3
4tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
5model = BertForSequenceClassification.from_pretrained(
6    "bert-base-uncased",
7    num_labels=3,
8)
9
10texts = [
11    "The movie was fantastic.",
12    "The package arrived late.",
13]
14
15batch = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
16logits = model(**batch).logits
17predicted = torch.argmax(logits, dim=-1)
18print(predicted)

The tensor shape is straightforward: one input example leads to one logits vector of length num_labels.

What BertForMultipleChoice Expects

BertForMultipleChoice is shaped for problems where each example comes with several candidate options and exactly one should win.

Examples include:

• reading comprehension with answer choices
• question answering with candidate endings
• sentence completion with several alternatives

Its input shape is different. Instead of one sequence per example, you provide num_choices sequences per example.

1from transformers import AutoTokenizer, BertForMultipleChoice
2import torch
3
4tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
5model = BertForMultipleChoice.from_pretrained("bert-base-uncased")
6
7prompt = "The capital of France is"
8choices = ["Paris", "Berlin", "Madrid"]
9texts = [[f"{prompt} {choice}" for choice in choices]]
10
11encoded = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
12logits = model(**encoded).logits
13predicted_choice = torch.argmax(logits, dim=-1)
14print(predicted_choice)

The model outputs one score per choice, not one score per abstract class label.

Why This Matters For Multi-Class Sentence Labeling

Suppose your task is to classify a sentence into one of these categories:

• sports
• politics
• business

That is a normal multi-class classification problem. The input is one sentence. The output is one label from a fixed taxonomy.

That is exactly what BertForSequenceClassification is designed for.

Using BertForMultipleChoice here would force you to reformulate each class as a choice string and create one input sequence per class for every training example. That is unnecessary overhead and changes the problem shape in a way the original task does not require.

The Input Shape Difference Is The Real Answer

The easiest way to remember the distinction is by shape.

BertForSequenceClassification:

• input shape is effectively [batch_size, seq_len]
• output shape is [batch_size, num_labels]

BertForMultipleChoice:

• input shape is effectively [batch_size, num_choices, seq_len]
• output shape is [batch_size, num_choices]

That difference alone usually resolves the model selection question.

If your training example naturally has one text and one label, use sequence classification. If each example naturally has one prompt and several candidate sequences to compare, use multiple choice.

A Correct Training Sketch For Sequence Classification

Here is a minimal training-oriented example for a sentence classification dataset.

1from transformers import AutoTokenizer, BertForSequenceClassification
2import torch
3
4texts = [
5    "The team won the championship.",
6    "The company reported lower revenue.",
7    "Parliament passed the bill.",
8]
9labels = torch.tensor([0, 1, 2])
10
11tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
12model = BertForSequenceClassification.from_pretrained(
13    "bert-base-uncased",
14    num_labels=3,
15)
16
17batch = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
18outputs = model(**batch, labels=labels)
19print(outputs.loss)
20print(outputs.logits.shape)

This matches the task directly and keeps the loss function aligned with ordinary multi-class classification.

When BertForMultipleChoice Is Actually Right

There are cases where multiple choice is the correct formulation even if the choices look like classes. For example, if each label is represented by a different candidate sentence and the decision depends on comparing the full semantics of those candidate sentences against a prompt, the multiple-choice head can make sense.

But that is a task about choosing among per-example options, not about predicting one global class ID from a fixed label vocabulary.

Common Pitfalls

• Using BertForMultipleChoice just because the task has more than two classes.
• Ignoring the input shape difference between one sequence per example and multiple candidate sequences per example.
• Reformulating a simple label-classification problem into a choice-ranking problem for no benefit.
• Forgetting to set num_labels on BertForSequenceClassification.
• Mixing task semantics with model class names instead of checking what tensors each model expects.

Summary

• For ordinary sentence multi-class classification, use BertForSequenceClassification.
• 'BertForMultipleChoice is for examples that contain several candidate options per input.'
• The decisive difference is input and output shape, not the number of labels alone.
• Use sequence classification when each example is one text mapped to one label from a fixed set.
• Use multiple choice only when each example naturally includes competing candidate sequences.