Application Insights, a part of Azure Monitor, is a highly versatile application performance management (APM) service that monitors live applications efficiently. One of its pivotal features includes the ability to correlate telemetry data from multiple sources to a single operation, using Operation IDs. This becomes particularly useful in distributed systems where components may run on different machines or processes.

However, linking telemetry from different processes or machines that do not use HTTP (which automatically supports correlation) requires a manual approach. Below, we explore how to manually (re)attach telemetry to an Application Insights Operation from another machine or process.

Understanding Operation and Telemetry Context

When telemetry is sent to Application Insights, it's associated with a TelemetryContext. This context includes the Operation ID (a GUID representing the operation across components), and potentially a Parent ID (representing a specific parent operation within the overall operation).

For enabling cross-component telemetry correlation in scenarios not covered automatically by Application Insights (such as non-HTTP based communications), you must manually propagate and set these context values.

Manual Propagation of Operation IDs

1. Extract and Transmit Context Data: Initially, when the operation starts, an Operation ID is generated. This ID and any other context data, like Parent ID, must be explicitly extracted and transmitted to the other machine or process. This often means serializing this context into a message or data format that is being sent to the other component.
2. Deserialize and Apply the Context: On the receiving end, this context must be deserialized from the incoming message and used to configure the TelemetryContext for that operation in that specific machine or process environment.

Example Scenario: Non-HTTP Messaging

Consider a scenario where two services communicate over a custom TCP-based protocol. Here’s how you could manually link the operations:

Sending Side:

1var operation = telemetryClient.StartOperation<DependencyTelemetry>("operation-name");
2try
3{
4    var message = new MyCustomMessage();
5    message.Properties["OperationId"] = operation.Telemetry.Context.Operation.Id;
6    message.Properties["ParentId"] = operation.Telemetry.Id;
7    
8    // Serialize and send the message over TCP
9    SendMessageOverTcp(message);
10}
11finally
12{
13    telemetryClient.StopOperation(operation);
14}

Receiving Side:

1var receivedMessage = ReceiveMessageOverTcp();
2if (receivedMessage.Properties.TryGetValue("OperationId", out var operationId) &&
3    receivedMessage.Properties.TryGetValue("ParentId", out var parentId))
4{
5    var operation = telemetryClient.StartOperation<RequestTelemetry>("operation-name");
6    operation.Telemetry.Context.Operation.Id = operationId;
7    operation.Telemetry.Context.Operation.ParentId = parentId;
8
9    try
10    {
11        // Perform operation
12    }
13    finally
14    {
15        telemetryClient.StopOperation(operation);
16    }
17}

Key Points in Operation Reattachment

Below is a table summarizing key considerations when manually reattaching telemetry to an App Insights Operation:

Additional Considerations

• Security: Ensure that the propagation of telemetry context does not expose sensitive information, and that the mechanism used for transmission is secure.
• Performance: Consider the overhead of adding telemetry propagation, particularly serialization and deserialization in performance-critical paths.
• Integration: In complex systems, ensure consistent implementation across all components to maintain reliable telemetry.

Conclusion

Manually reattaching operations across different machines or processes in non-HTTP environments, although complex, is crucial for maintaining a holistic view of application performance and troubleshooting in distributed systems. By ensuring correct propagation and assignment of the telemetry context, developers can provide deep insights into the health and activities of their applications across varied environments.