IPython
logging module
Jupyter Notebook
Python programming
debugging

Get Output From the logging Module in IPython Notebook

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Introduction

Python logging inside a Jupyter or IPython notebook often behaves differently from a normal script because the kernel stays alive across multiple cell runs. That means handlers can accumulate, basicConfig() may appear to do nothing, and messages can vanish or duplicate. A good notebook setup creates one explicit logger and one explicit stream handler and then reuses them.

Why Logging Feels Strange in Notebooks

In a normal script, the process starts fresh each run. In a notebook, the kernel persists. If you configure logging in one cell, then run another cell that configures it again, you may end up with:

  • duplicate log lines from multiple handlers
  • no visible change because basicConfig() only configures once
  • mixed formatting from old state left in memory

That is why notebook logging should be more explicit than script logging.

A Clean Minimal Setup

This pattern works well in notebooks:

python
1import logging
2import sys
3
4logger = logging.getLogger("notebook-demo")
5logger.setLevel(logging.INFO)
6logger.propagate = False
7
8logger.handlers.clear()
9
10handler = logging.StreamHandler(sys.stdout)
11handler.setLevel(logging.INFO)
12handler.setFormatter(logging.Formatter("%(levelname)s | %(message)s"))
13
14logger.addHandler(handler)
15
16logger.info("hello from notebook logging")

Important choices here:

  • 'handlers.clear() prevents duplicate output when the cell is rerun'
  • 'sys.stdout sends the output into the notebook cell stream'
  • 'propagate = False avoids double logging through ancestor loggers'

Why basicConfig() Often Disappoints

People often try:

python
1import logging
2
3logging.basicConfig(level=logging.INFO)
4logging.info("test")

This can work once, but later notebook runs may not change anything because basicConfig() is intentionally conservative. If handlers already exist, it does very little.

In modern Python, you can force it:

python
1import logging
2
3logging.basicConfig(
4    level=logging.INFO,
5    format="%(levelname)s | %(message)s",
6    force=True,
7)
8
9logging.info("configured with force")

force=True is helpful in notebooks because it resets previous logging setup.

Capturing Logs from Your Own Modules

If you are testing a module inside a notebook, create a named logger in that module:

python
1import logging
2
3module_logger = logging.getLogger("myapp.service")
4module_logger.info("service started")

Then in the notebook, configure the root logger or the specific namespace you care about:

python
1import logging
2import sys
3
4root = logging.getLogger()
5root.handlers.clear()
6root.setLevel(logging.INFO)
7
8handler = logging.StreamHandler(sys.stdout)
9handler.setFormatter(logging.Formatter("%(name)s | %(levelname)s | %(message)s"))
10root.addHandler(handler)

That lets imported code send logs into notebook output cleanly.

Logging to a File as Well

For longer notebook sessions, file logging is useful because cell output is easy to lose.

python
1import logging
2
3logger = logging.getLogger("notebook-file")
4logger.setLevel(logging.INFO)
5logger.handlers.clear()
6
7file_handler = logging.FileHandler("notebook.log", mode="w")
8file_handler.setFormatter(logging.Formatter("%(asctime)s | %(levelname)s | %(message)s"))
9
10logger.addHandler(file_handler)
11logger.info("written to file")

You can combine a StreamHandler and a FileHandler if you want both interactive output and a persistent log file.

Common Notebook Debugging Pattern

For exploratory work, this setup is practical:

  1. clear handlers
  2. attach a stream handler to sys.stdout
  3. use a readable short formatter
  4. keep INFO for normal work and switch to DEBUG only when needed

That keeps notebook output readable without turning every cell into a wall of framework noise.

Common Pitfalls

  • Re-running logging setup cells without clearing old handlers.
  • Expecting basicConfig() to reconfigure logging after the first run.
  • Logging to stderr and then wondering why notebook output ordering looks odd.
  • Letting messages propagate to the root logger and getting duplicates.
  • Mixing library logger configuration and root logger configuration without a plan.

Summary

  • Notebook kernels keep logging state alive across cell runs.
  • Use explicit loggers and handlers instead of relying blindly on basicConfig().
  • Clear handlers before reconfiguring to avoid duplicate output.
  • Send logs to sys.stdout for predictable notebook display.
  • Use force=True with basicConfig() when you really want a reset.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

All Rights Reserved.