Introduction

Restricted Boltzmann Machines can model binary note activations from MIDI data, but most TensorFlow errors in RBM projects come from data preparation rather than from the learning rule itself. If the piano-roll matrix has the wrong shape, dtype, or value range, the sampling code quickly fails with shape mismatches or unstable updates.

Convert MIDI Into A Binary Piano Roll

An RBM expects fixed-width visible vectors. For symbolic music, that usually means converting each time slice of a MIDI file into a vector of note activations, often one element per MIDI pitch.

Here is a minimal preprocessing function using pretty_midi:

1import numpy as np
2import pretty_midi
3
4
5def midi_to_binary_roll(path, fs=8):
6    midi = pretty_midi.PrettyMIDI(path)
7    roll = midi.get_piano_roll(fs=fs)   # shape: [128, time_steps]
8    binary = (roll > 0).astype("float32")
9    return binary.T                     # shape: [time_steps, 128]
10
11
12roll = midi_to_binary_roll("example.mid")
13print(roll.shape)
14print(roll.dtype)

Two details matter immediately:

• The output should be float32, not integers or objects.
• Each row should represent one training example with shape [n_visible].

If you feed TensorFlow a matrix shaped [128, time_steps] instead of [time_steps, 128], matrix multiplications in the RBM will be wrong.

A Minimal RBM In TensorFlow

TensorFlow does not provide a built-in high-level RBM layer, so you usually implement the weight updates manually with contrastive divergence.

1import tensorflow as tf
2
3
4class RBM(tf.Module):
5    def __init__(self, n_visible, n_hidden):
6        self.W = tf.Variable(tf.random.normal([n_visible, n_hidden], stddev=0.01))
7        self.v_bias = tf.Variable(tf.zeros([n_visible]))
8        self.h_bias = tf.Variable(tf.zeros([n_hidden]))
9
10    def hidden_prob(self, visible):
11        return tf.sigmoid(tf.matmul(visible, self.W) + self.h_bias)
12
13    def visible_prob(self, hidden):
14        return tf.sigmoid(tf.matmul(hidden, tf.transpose(self.W)) + self.v_bias)
15
16    def sample_binary(self, prob):
17        noise = tf.random.uniform(tf.shape(prob))
18        return tf.cast(noise < prob, tf.float32)
19
20    def contrastive_divergence(self, v0, learning_rate=0.01):
21        h0_prob = self.hidden_prob(v0)
22        h0_sample = self.sample_binary(h0_prob)
23        v1_prob = self.visible_prob(h0_sample)
24        v1_sample = self.sample_binary(v1_prob)
25        h1_prob = self.hidden_prob(v1_sample)
26
27        batch_size = tf.cast(tf.shape(v0)[0], tf.float32)
28        positive = tf.matmul(v0, h0_prob, transpose_a=True)
29        negative = tf.matmul(v1_sample, h1_prob, transpose_a=True)
30
31        self.W.assign_add(learning_rate * (positive - negative) / batch_size)
32        self.v_bias.assign_add(learning_rate * tf.reduce_mean(v0 - v1_sample, axis=0))
33        self.h_bias.assign_add(learning_rate * tf.reduce_mean(h0_prob - h1_prob, axis=0))

This example expects v0 to have shape [batch_size, 128] if you use the full MIDI pitch range as visible units.

Train On Batched Time Slices

Once the piano roll is prepared, training is straightforward:

1import tensorflow as tf
2
3roll = midi_to_binary_roll("example.mid")
4dataset = tf.data.Dataset.from_tensor_slices(roll).batch(32)
5
6rbm = RBM(n_visible=128, n_hidden=64)
7
8for epoch in range(5):
9    for batch in dataset:
10        rbm.contrastive_divergence(batch)

This works only if every batch has the same feature width. That is why converting MIDI into fixed 128-note vectors is so useful.

If you instead try to feed variable-length note event sequences directly, TensorFlow will often complain about irregular shapes or object arrays.

Why Errors Usually Happen

Most TensorFlow RBM errors fall into a few buckets.

The first is shape mismatch. If visible has the wrong orientation, tf.matmul raises an invalid argument error because the inner dimensions do not line up.

The second is dtype mismatch. MIDI processing libraries often return integer arrays, and older code examples sometimes mix NumPy default integers with TensorFlow float ops. Casting to float32 early avoids that entire class of errors.

The third is using TF1-era tutorials in modern TensorFlow. Older RBM examples rely on placeholders, sessions, and graph-building patterns that do not map cleanly to eager execution. If you copy that code into a current TensorFlow project, the API mismatch can look like an RBM bug when it is really a version mismatch.

Data Quality Still Matters

An RBM learns binary structure best when the input really is binary or close to binary. Raw velocity values, overlapping sustain artifacts, or inconsistent time quantization can make the model harder to train.

A practical cleanup pipeline is:

• Quantize time at a fixed sampling rate.
• Convert any positive velocity to 1.0.
• Keep the feature width fixed at 128 pitches.
• Batch time slices, not whole ragged songs.

That representation is simple, but it matches the math of Bernoulli visible units.

Common Pitfalls

The biggest mistake is feeding raw MIDI event objects directly into TensorFlow instead of converting them to numeric arrays first.

Another mistake is forgetting to transpose the piano roll. Many libraries return [pitch, time], while the RBM expects [sample, feature].

People also run into errors when they mix outdated TensorFlow 1 code with current TensorFlow 2 execution. If the tutorial uses sessions or placeholders, translate the ideas rather than copying the code literally.

Finally, do not expect an RBM to work well with arbitrary continuous-valued MIDI features unless you redesign the visible distribution. The standard Bernoulli setup assumes binary note activity.

Summary

• Convert MIDI to a fixed-width binary piano roll before training an RBM.
• Use rows shaped like [time_step, n_visible] so matrix multiplications line up.
• Cast data to float32 early to avoid dtype errors.
• Most RBM TensorFlow issues come from preprocessing and old tutorial code, not from the core learning rule.
• Keep the representation simple and binary when using a Bernoulli RBM.