How to get weights from tensorflow fully_connected
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Introduction
In TensorFlow, a fully connected layer is usually a Dense layer. Its learnable parameters are the kernel, which is the weight matrix, and the bias vector. Once the layer has been built, reading those values is simple.
The main question is whether you want plain NumPy arrays, TensorFlow variables, or legacy TensorFlow 1.x tensors. In modern TensorFlow, layer.get_weights(), layer.kernel, and model.trainable_variables cover almost everything.
Read Weights with get_weights()
In tf.keras, get_weights() returns NumPy arrays. For a dense layer, the first array is the kernel and the second is the bias.
If the input has size 4 and the layer has 8 units, the kernel shape is (4, 8) and the bias shape is (8,). That shape is the most useful quick sanity check when debugging layer construction.
Access TensorFlow Variables Directly
If you want TensorFlow variable objects instead of NumPy arrays, use the layer fields directly.
This is useful when you want to inspect or compare weights during training, or when you are writing custom logging and visualization code.
You can also inspect every trainable variable in the model:
That is often the fastest way to understand how TensorFlow named the parameters in a larger model.
It is also a convenient way to compare checkpoint contents against the currently loaded model structure when something seems to be missing or mapped to the wrong layer.
Build the Layer Before Inspecting It
The most common mistake is trying to read weights before the layer exists in a built state. TensorFlow does not create the kernel and bias until it knows the input shape.
This works immediately because the input shape is declared:
If you create a layer manually, you may need to build it yourself:
Without that step, get_weights() can return an empty list because the variables do not exist yet.
Legacy TensorFlow 1.x Code
Older TensorFlow code often used tf.layers.dense or tf.contrib.layers.fully_connected. In that style, weights were usually inspected inside a session through the trainable-variable collection.
That is still relevant for maintenance work, but new projects should generally stay with tf.keras.
Common Pitfalls
The biggest mistake is reading weights from an unbuilt layer. No input shape means no kernel or bias.
Another common issue is confusing layer.weights with layer.get_weights(). One gives TensorFlow variable objects. The other gives NumPy arrays.
It is also easy to assume frozen layers have no weights. They still do. trainable=False only stops updates during training.
Finally, avoid mixing TensorFlow 1.x examples into TensorFlow 2 code unless you are explicitly maintaining a legacy graph-based model.
That version boundary causes a surprising amount of confusion.
Summary
- A TensorFlow fully connected layer is usually a
Denselayer. - Use
layer.get_weights()to read kernel and bias as NumPy arrays. - Use
layer.kernelandlayer.biasfor direct TensorFlow variables. - Make sure the layer is built before inspecting it.
- Use TensorFlow 1.x variable collections only for legacy code.
Related reading
- How to get weights in tf.layers.dense?
- How to give a constant input to keras
- How to graph tf.keras model in Tensorflow-2.0?
- How to handle large amouts of data in tensorflow?
- how to give the test size in stratified kfold sampling in python?
- How to graph grid scores from GridSearchCV?
- How to handle large amouts of data in tensorflow?
- How to handle RGB images in Keras
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.