Introduction

Infer.NET's Bayesian Point Machine is naturally a binary classifier, so it predicts one of two outcomes. If your problem has three or more classes, you do not usually change BPM into a different native multiclass algorithm. Instead, you build a multiclass system around multiple binary BPM models.

Why BPM Is Binary by Default

A standard BPM learns weights that separate positive and negative labels. Conceptually, it models whether a score falls on one side of a decision boundary. That is perfect for yes-or-no tasks but not enough for problems like classifying a message as support, billing, or sales.

The most common solution is one-vs-rest training. For each class, create a separate binary dataset where that class is positive and every other class is negative. At prediction time, run every model and choose the class with the highest score or probability.

One-vs-Rest Architecture

Suppose your labels are "A", "B", and "C". You train three BPM classifiers:

• 'A versus not A'
• 'B versus not B'
• 'C versus not C'

The orchestration code in C# is simple and runnable even before the training internals are added:

1using System;
2using System.Collections.Generic;
3using System.Linq;
4
5public class Program
6{
7    public static void Main()
8    {
9        var scores = new Dictionary<string, double>
10        {
11            ["A"] = 0.12,
12            ["B"] = 0.77,
13            ["C"] = 0.31
14        };
15
16        string predicted = scores.OrderByDescending(pair => pair.Value).First().Key;
17        Console.WriteLine(predicted);
18    }
19}

That example shows the decision rule: evaluate every binary model and take the label with the strongest result. The actual BPM score production happens in the models you train with Infer.NET.

Sketching a Binary BPM in Infer.NET

A typical Infer.NET BPM defines observed feature vectors, observed Boolean labels, and a prior over the weight vector. The code below shows the core binary model structure.

1using Microsoft.ML.Probabilistic.Distributions;
2using Microsoft.ML.Probabilistic.Math;
3using Microsoft.ML.Probabilistic.Models;
4
5int featureCount = 2;
6Vector[] features =
7{
8    Vector.FromArray(1.0, 0.2),
9    Vector.FromArray(0.9, 0.1),
10    Vector.FromArray(-1.1, -0.3),
11    Vector.FromArray(-0.8, -0.4)
12};
13bool[] labels = { true, true, false, false };
14
15Range item = new Range(features.Length);
16var x = Variable.Observed(features, item);
17var y = Variable.Observed(labels, item);
18
19var wPrior = VectorGaussian.FromMeanAndVariance(
20    Vector.Zero(featureCount),
21    PositiveDefiniteMatrix.Identity(featureCount)
22);
23Variable<Vector> w = Variable.Random(wPrior);
24
25using (Variable.ForEach(item))
26{
27    var score = Variable.InnerProduct(w, x[item]);
28    y[item] = Variable.GaussianFromMeanAndVariance(score, 1.0).IsPositive();
29}

For multiclass prediction, you repeat that training setup once per class, changing only the Boolean labels. For class "A", mark rows of class "A" as true and every other row as false. Do the same again for "B", then "C", and so on.

Alternatives to One-vs-Rest

One-vs-rest is usually the first answer because it is easy to implement and scales predictably. If classes are heavily imbalanced or often confused, one-vs-one is another option. That means training a classifier for every pair of classes and combining their votes, but it creates more models.

A softer probabilistic alternative is to build a custom latent-score model and normalize the scores across classes, similar in spirit to softmax classification. Infer.NET can express that kind of model, but it is no longer just a standard BPM. At that point you are designing a new probabilistic classifier, not simply extending the existing binary one.

Operational Considerations

Multiclass BPM systems need consistent feature extraction. Every one-vs-rest model must see the same preprocessing, the same feature ordering, and the same scale. If one model uses normalized inputs and another does not, the scores are not comparable.

You also need a decision policy for close scores. Some teams choose the highest probability no matter what. Others require a margin and return an "unknown" result when the top score is too weak.

Common Pitfalls

The most common mistake is expecting a single BPM instance to accept several labels directly. Standard BPM is a binary setup, so multiclass behavior has to be built around multiple binary models.

Another issue is interpreting raw scores from differently trained models without calibration. If one classifier is systematically overconfident, argmax selection can become biased. Validate scores on held-out data.

Finally, do not ignore class imbalance. In a one-vs-rest problem, the positive class may be much smaller than the combined negative set, which can distort learning unless you inspect the data carefully.

Summary

• BPM in Infer.NET is naturally a binary classifier.
• For more than two outcomes, one-vs-rest is the usual solution.
• Train one binary BPM per class and select the class with the strongest score.
• More advanced multiclass models are possible, but they are custom probabilistic designs rather than plain BPM.
• Consistent preprocessing and score validation matter as much as model structure.