tensorflow
fully_connected
neural_networks
machine_learning
model_weights

How to get weights from tensorflow fully_connected

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Introduction

In TensorFlow, a fully connected layer is usually a Dense layer. Its learnable parameters are the kernel, which is the weight matrix, and the bias vector. Once the layer has been built, reading those values is simple.

The main question is whether you want plain NumPy arrays, TensorFlow variables, or legacy TensorFlow 1.x tensors. In modern TensorFlow, layer.get_weights(), layer.kernel, and model.trainable_variables cover almost everything.

Read Weights with get_weights()

In tf.keras, get_weights() returns NumPy arrays. For a dense layer, the first array is the kernel and the second is the bias.

python
1import tensorflow as tf
2
3model = tf.keras.Sequential([
4    tf.keras.layers.Input(shape=(4,)),
5    tf.keras.layers.Dense(8, activation="relu", name="hidden"),
6    tf.keras.layers.Dense(3, name="output"),
7])
8
9layer = model.get_layer("hidden")
10weights, biases = layer.get_weights()
11
12print(weights.shape)
13print(biases.shape)
14print(weights[:2, :3])

If the input has size 4 and the layer has 8 units, the kernel shape is (4, 8) and the bias shape is (8,). That shape is the most useful quick sanity check when debugging layer construction.

Access TensorFlow Variables Directly

If you want TensorFlow variable objects instead of NumPy arrays, use the layer fields directly.

python
1layer = model.get_layer("hidden")
2
3print(layer.kernel.shape)
4print(layer.bias.shape)
5print(layer.kernel.numpy()[:2, :3])

This is useful when you want to inspect or compare weights during training, or when you are writing custom logging and visualization code.

You can also inspect every trainable variable in the model:

python
for variable in model.trainable_variables:
    print(variable.name, variable.shape)

That is often the fastest way to understand how TensorFlow named the parameters in a larger model.

It is also a convenient way to compare checkpoint contents against the currently loaded model structure when something seems to be missing or mapped to the wrong layer.

Build the Layer Before Inspecting It

The most common mistake is trying to read weights before the layer exists in a built state. TensorFlow does not create the kernel and bias until it knows the input shape.

This works immediately because the input shape is declared:

python
1model = tf.keras.Sequential([
2    tf.keras.layers.Input(shape=(4,)),
3    tf.keras.layers.Dense(8),
4])
5
6print(model.layers[0].get_weights())

If you create a layer manually, you may need to build it yourself:

python
1layer = tf.keras.layers.Dense(8)
2layer.build((None, 4))
3
4weights, biases = layer.get_weights()
5print(weights.shape, biases.shape)

Without that step, get_weights() can return an empty list because the variables do not exist yet.

Legacy TensorFlow 1.x Code

Older TensorFlow code often used tf.layers.dense or tf.contrib.layers.fully_connected. In that style, weights were usually inspected inside a session through the trainable-variable collection.

python
1import tensorflow.compat.v1 as tf
2tf.disable_eager_execution()
3
4x = tf.placeholder(tf.float32, shape=[None, 4])
5y = tf.layers.dense(x, 8, name="fc1")
6
7with tf.Session() as sess:
8    sess.run(tf.global_variables_initializer())
9    for variable in tf.trainable_variables():
10        print(variable.name, variable.shape)

That is still relevant for maintenance work, but new projects should generally stay with tf.keras.

Common Pitfalls

The biggest mistake is reading weights from an unbuilt layer. No input shape means no kernel or bias.

Another common issue is confusing layer.weights with layer.get_weights(). One gives TensorFlow variable objects. The other gives NumPy arrays.

It is also easy to assume frozen layers have no weights. They still do. trainable=False only stops updates during training.

Finally, avoid mixing TensorFlow 1.x examples into TensorFlow 2 code unless you are explicitly maintaining a legacy graph-based model.

That version boundary causes a surprising amount of confusion.

Summary

  • A TensorFlow fully connected layer is usually a Dense layer.
  • Use layer.get_weights() to read kernel and bias as NumPy arrays.
  • Use layer.kernel and layer.bias for direct TensorFlow variables.
  • Make sure the layer is built before inspecting it.
  • Use TensorFlow 1.x variable collections only for legacy code.

Related reading
Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

All Rights Reserved.