Python logging best practices with structlog in production

<p>I need consistent structured logging across my Python application. Logs need to be machine-parseable JSON in production but human-readable in development, with context (request_id, user_id) propagated automatically.</p>

<p>structlog setup with context propagation:</p> <div class="highlight"><pre><span></span><code><span class="kn">import</span><span class="w"> </span><span class="nn">structlog</span> <span class="kn">import</span><span class="w"> </span><span class="nn">logging</span> <span class="kn">import</span><span class="w"> </span><span class="nn">sys</span> <span class="k">def</span><span class="w"> </span><span class="nf">configure_logging</span><span class="p">(</span><span class="n">json_logs</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span> <span class="n">log_level</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s1">'INFO'</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span> <span class="n">processors</span> <span class="o">=</span> <span class="p">[</span> <span class="n">structlog</span><span class="o">.</span><span class="n">contextvars</span><span class="o">.</span><span class="n">merge_contextvars</span><span class="p">,</span> <span class="c1"># Inject request-scoped context</span> <span class="n">structlog</span><span class="o">.</span><span class="n">stdlib</span><span class="o">.</span><span class="n">add_log_level</span><span class="p">,</span> <span class="n">structlog</span><span class="o">.</span><span class="n">stdlib</span><span class="o">.</span><span class="n">add_logger_name</span><span class="p">,</span> <span class="n">structlog</span><span class="o">.</span><span class="n">processors</span><span class="o">.</span><span class="n">TimeStamper</span><span class="p">(</span><span class="n">fmt</span><span class="o">=</span><span class="s1">'iso'</span><span class="p">,</span> <span class="n">utc</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span> <span class="n">structlog</span><span class="o">.</span><span class="n">processors</span><span class="o">.</span><span class="n">StackInfoRenderer</span><span class="p">(),</span> <span class="p">]</span> <span class="k">if</span> <span class="n">json_logs</span><span class="p">:</span> <span class="n">processors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">structlog</span><span class="o">.</span><span class="n">processors</span><span class="o">.</span><span class="n">JSONRenderer</span><span class="p">())</span> <span class="k">else</span><span class="p">:</span> <span class="n">processors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">structlog</span><span class="o">.</span><span class="n">dev</span><span class="o">.</span><span class="n">ConsoleRenderer</span><span class="p">())</span> <span class="n">structlog</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span> <span class="n">processors</span><span class="o">=</span><span class="n">processors</span><span class="p">,</span> <span class="n">wrapper_class</span><span class="o">=</span><span class="n">structlog</span><span class="o">.</span><span class="n">make_filtering_bound_logger</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">getLevelName</span><span class="p">(</span><span class="n">log_level</span><span class="p">)),</span> <span class="n">logger_factory</span><span class="o">=</span><span class="n">structlog</span><span class="o">.</span><span class="n">PrintLoggerFactory</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="p">),</span> <span class="n">cache_logger_on_first_use</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="p">)</span> <span class="c1"># Bind context for all logs in a request:</span> <span class="n">structlog</span><span class="o">.</span><span class="n">contextvars</span><span class="o">.</span><span class="n">bind_contextvars</span><span class="p">(</span> <span class="n">request_id</span><span class="o">=</span><span class="n">request_id</span><span class="p">,</span> <span class="n">user_id</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">current_user</span><span class="o">.</span><span class="n">id</span><span class="p">),</span> <span class="p">)</span> <span class="c1"># Usage throughout the codebase:</span> <span class="n">log</span> <span class="o">=</span> <span class="n">structlog</span><span class="o">.</span><span class="n">get_logger</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span> <span class="n">log</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s1">'trace.created'</span><span class="p">,</span> <span class="n">trace_id</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">trace</span><span class="o">.</span><span class="n">id</span><span class="p">),</span> <span class="n">title</span><span class="o">=</span><span class="n">trace</span><span class="o">.</span><span class="n">title</span><span class="p">)</span> <span class="n">log</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="s1">'rate_limit.exceeded'</span><span class="p">,</span> <span class="n">api_key_prefix</span><span class="o">=</span><span class="n">key</span><span class="p">[:</span><span class="mi">8</span><span class="p">])</span> <span class="n">log</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s1">'embedding.failed'</span><span class="p">,</span> <span class="n">trace_id</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">trace_id</span><span class="p">),</span> <span class="n">error</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">))</span> <span class="c1"># Clear context at end of request:</span> <span class="n">structlog</span><span class="o">.</span><span class="n">contextvars</span><span class="o">.</span><span class="n">clear_contextvars</span><span class="p">()</span> </code></pre></div> <p>Key points: - merge_contextvars automatically injects bound vars into every log line - JSON output in production for log aggregation (Datadog, Elasticsearch) - ConsoleRenderer in development for human-readable output - cache_logger_on_first_use: True improves performance - Use log.exception('msg') to include stack trace (equivalent to log.error with exc_info=True)</p>