Introduction

Elastic APM can connect traces across multiple Node services, but Kafka requires one extra step: you must propagate the trace context in message headers yourself. If you only install the APM agent and rely on HTTP auto-instrumentation, the Kafka handoff will break the trace.

The working setup is straightforward once you split it into two responsibilities: start the agent early in every service, then copy traceparent from producer to consumer.

Start the Agent Before Other Modules

Every Node service should start elastic-apm-node before loading Express, KafkaJS, database clients, or anything else the agent might instrument:

1// apm.js
2require("elastic-apm-node").start({
3  serviceName: process.env.ELASTIC_APM_SERVICE_NAME,
4  serverUrl: process.env.ELASTIC_APM_SERVER_URL,
5  environment: process.env.NODE_ENV || "development",
6});

Then import it first:

1// index.js
2require("./apm");
3
4const express = require("express");
5const app = express();
6
7app.get("/health", (_req, res) => {
8  res.send("ok");
9});
10
11app.listen(3000);

Use a unique serviceName for each process, such as checkout-api, orders-consumer, or inventory-service. If several services share the same name, Kibana groups them together and the trace becomes harder to read.

Producer: Put traceparent in Kafka Headers

If a service is already inside a request transaction, Elastic exposes the current trace context through apm.currentTraceparent. Put that value into the Kafka headers when you send the message:

1const apm = require("elastic-apm-node");
2const { Kafka } = require("kafkajs");
3
4const kafka = new Kafka({
5  clientId: "checkout-api",
6  brokers: ["localhost:9092"],
7});
8
9const producer = kafka.producer();
10
11async function publishOrderCreated(order) {
12  const span = apm.startSpan("kafka send order.created", "messaging");
13
14  try {
15    const traceparent = apm.currentTraceparent;
16
17    await producer.send({
18      topic: "order.created",
19      messages: [
20        {
21          value: JSON.stringify(order),
22          headers: traceparent ? { traceparent } : {},
23        },
24      ],
25    });
26  } finally {
27    if (span) span.end();
28  }
29}

If this code runs outside any active transaction, start one explicitly before producing the message. Otherwise the send span will exist, but it will not belong to a useful distributed trace.

Consumer: Start a Child Transaction

On the consumer side, read the Kafka header and start a transaction with childOf:

1const apm = require("elastic-apm-node");
2
3await consumer.run({
4  eachMessage: async ({ topic, message }) => {
5    const traceparent = message.headers?.traceparent?.toString();
6
7    const transaction = traceparent
8      ? apm.startTransaction(`consume ${topic}`, "messaging", { childOf: traceparent })
9      : apm.startTransaction(`consume ${topic}`, "messaging");
10
11    try {
12      const payload = JSON.parse(message.value.toString());
13      await processOrder(payload);
14      transaction.result = "success";
15    } catch (err) {
16      apm.captureError(err);
17      transaction.result = "failure";
18      throw err;
19    } finally {
20      transaction.end();
21    }
22  },
23});

Once that transaction is active, downstream HTTP calls from processOrder() can continue the same trace automatically if the receiving service also uses the agent correctly.

What the Full Trace Looks Like

A typical end-to-end flow is:

• 'api-gateway receives an HTTP request and starts an incoming transaction automatically'
• 'api-gateway publishes a Kafka message with the current traceparent'
• 'orders-consumer starts a child transaction from that header'
• 'orders-consumer calls another Node service over HTTP, and that HTTP hop continues the trace automatically'

When this is working, Kibana shows one distributed trace instead of isolated transactions.

Common Pitfalls

The most common mistake is starting the APM agent too late. If Express or KafkaJS loads before elastic-apm-node, instrumentation can be incomplete.

Another common problem is expecting Kafka context propagation to happen automatically. HTTP is mostly automatic; Kafka headers are not. You must write and read traceparent yourself.

Older examples sometimes use elastic-apm-traceparent. Current guidance is centered on the standard traceparent header, so mixed old and new services need extra care.

Finally, always end spans and transactions. Missing span.end() or transaction.end() leaves incomplete timing data and confusing traces in Kibana.

Summary

• Start the Elastic APM agent before other modules in every Node service.
• Give each service a unique serviceName.
• Put apm.currentTraceparent into Kafka message headers on the producer.
• Use the received header as childOf when starting the consumer transaction.
• If the trace breaks in Kibana, check agent startup order, header propagation, and transaction lifecycle first.