Python logging use milliseconds in time format
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
Python's standard logging module can include milliseconds in log timestamps, but the common mistake is trying to do it entirely through datefmt. The reliable built-in approach is to keep the date and time in %(asctime)s and append milliseconds with %(msecs)03d.
The Standard Millisecond Pattern
This is the most common working setup:
That produces output shaped like:
%(asctime)s provides the base timestamp, and %(msecs)03d adds a three-digit millisecond suffix.
Why %f Usually Does Not Work
Many developers expect this to be enough:
In standard logging formatters, that often does not behave like datetime.strftime. The logging module uses time.strftime semantics by default, so %f is not reliably interpreted as microseconds in the normal formatter path.
That is why %(msecs)03d is the safe built-in choice for millisecond precision.
Use the Same Pattern With Explicit Handlers
If you manage handlers yourself instead of using basicConfig, the same formatting idea applies.
This is useful when file logging and console logging need different formats.
Custom Formatter for More Control
If you want true microseconds, custom timezone logic, or a different timestamp implementation, override formatTime.
This is beyond the basic milliseconds case, but it shows where the built-in formatter stops and customization begins.
Milliseconds Are Most Useful in Dense Logs
Millisecond timestamps become valuable when many events share the same second. For example, API request handling, retries, and batch steps often generate logs too dense for second-level timestamps to be useful.
Without milliseconds, those two lines may appear identical in time even when they represent distinct steps in a short workflow.
Think About Time Zone Too
Precision alone is not enough in distributed systems. If logs are collected from multiple machines or containers, decide whether timestamps should be local time or UTC. That is a separate concern from milliseconds, but it becomes important as soon as you start treating logs as serious operational data.
If needed, a custom formatter can change the converter or build UTC timestamps explicitly.
Common Pitfalls
The most common mistake is relying on %f inside datefmt and expecting standard logging to emit milliseconds automatically. In most normal setups, that is not how the formatter works.
Another issue is calling basicConfig after logging has already been configured elsewhere. At that point the new format may appear to do nothing.
Developers also sometimes mix handlers with different timestamp formats, which makes logs hard to compare when debugging across outputs.
Finally, millisecond precision does not imply clock synchronization. If several machines log events, precision and time consistency are separate problems.
Summary
- The standard logging pattern for milliseconds is
%(asctime)s.%(msecs)03d. - Use
datefmtfor the main date and time portion, not as the sole millisecond mechanism. - '
%fusually requires a custom formatter if you want datetime-style behavior.' - The same approach works with both
basicConfigand explicit handlers. - Millisecond timestamps help most when many log events happen within the same second.
Related reading
- Q How to rewrite single path among many with the ingress-nginx
- Query EC2 tags from within instance
- RabbitMQ ** WARNING ** Mnesia is overloaded
- RabbitMQ connection through Nginx
- python max function using 'key' and lambda expression
- Python maximum recursion depth exceeded while calling a Python object
- RabbitMQ enagle feature flags before run server
- RabbitMQ how to create and restore backup

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.