Introduction

MPI_Allreduce does not “send a structure” in the general sense. It performs a collective reduction, which means every process contributes a value, MPI combines those values with an operation, and the reduced result is returned to every process. If your data is a struct, you need both an MPI datatype that describes the struct layout and a reduction rule that explains how the fields should be combined.

First Ask Whether MPI_Allreduce Is the Right Operation

Before building a custom datatype, make sure MPI_Allreduce is actually what you need.

Use MPI_Allreduce when:

• every process contributes a value
• those values should be combined across ranks
• every process needs the same reduced result

If you only want to send a struct from one process to all others, MPI_Bcast is usually the correct collective. If you want one process to collect results, MPI_Reduce may be the better fit.

MPI_Allreduce is about reduction, not just transport.

Describe the Struct with an MPI Datatype

Suppose each process owns a struct with an id and a numeric value.

1#include <mpi.h>
2#include <stddef.h>
3
4typedef struct {
5    int id;
6    double value;
7} Data;
8
9MPI_Datatype create_data_type(void) {
10    MPI_Datatype type;
11    int block_lengths[2] = {1, 1};
12    MPI_Aint offsets[2] = {
13        offsetof(Data, id),
14        offsetof(Data, value)
15    };
16    MPI_Datatype types[2] = {MPI_INT, MPI_DOUBLE};
17
18    MPI_Type_create_struct(2, block_lengths, offsets, types, &type);
19    MPI_Type_commit(&type);
20    return type;
21}

This tells MPI how the struct is laid out in memory so it can move and reduce the data correctly.

Using offsetof is important because structure padding may exist between fields.

Define a Custom Reduction Operation

MPI still needs to know how to combine two Data values. For example, you might want the maximum id and the sum of value.

1#include <mpi.h>
2
3typedef struct {
4    int id;
5    double value;
6} Data;
7
8void reduce_data(void *invec, void *inoutvec, int *len, MPI_Datatype *datatype) {
9    Data *in = (Data *)invec;
10    Data *inout = (Data *)inoutvec;
11
12    for (int i = 0; i < *len; ++i) {
13        if (in[i].id > inout[i].id) {
14            inout[i].id = in[i].id;
15        }
16        inout[i].value += in[i].value;
17    }
18}

That function defines the reduction semantics field by field. MPI cannot guess this for arbitrary structs.

Use MPI_Allreduce with the Custom Pieces

Once you have both the datatype and the reduction operation, the collective call is straightforward.

1#include <mpi.h>
2#include <stdio.h>
3#include <stddef.h>
4
5typedef struct {
6    int id;
7    double value;
8} Data;
9
10MPI_Datatype create_data_type(void) {
11    MPI_Datatype type;
12    int block_lengths[2] = {1, 1};
13    MPI_Aint offsets[2] = {offsetof(Data, id), offsetof(Data, value)};
14    MPI_Datatype types[2] = {MPI_INT, MPI_DOUBLE};
15
16    MPI_Type_create_struct(2, block_lengths, offsets, types, &type);
17    MPI_Type_commit(&type);
18    return type;
19}
20
21void reduce_data(void *invec, void *inoutvec, int *len, MPI_Datatype *datatype) {
22    Data *in = (Data *)invec;
23    Data *inout = (Data *)inoutvec;
24
25    for (int i = 0; i < *len; ++i) {
26        if (in[i].id > inout[i].id) {
27            inout[i].id = in[i].id;
28        }
29        inout[i].value += in[i].value;
30    }
31}
32
33int main(int argc, char **argv) {
34    MPI_Init(&argc, &argv);
35
36    int rank;
37    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
38
39    Data local = {rank, rank + 0.5};
40    Data global;
41
42    MPI_Datatype data_type = create_data_type();
43    MPI_Op op;
44    MPI_Op_create(reduce_data, 1, &op);
45
46    MPI_Allreduce(&local, &global, 1, data_type, op, MPI_COMM_WORLD);
47
48    printf("rank %d sees id=%d value=%f\n", rank, global.id, global.value);
49
50    MPI_Op_free(&op);
51    MPI_Type_free(&data_type);
52    MPI_Finalize();
53    return 0;
54}

Every process receives the same reduced global result.

Why Built-in Ops Usually Do Not Work

Built-in operations such as MPI_SUM and MPI_MAX work on built-in datatypes and certain predefined combinations. They do not automatically understand custom structs with mixed field semantics.

That is why custom structs usually require a user-defined MPI_Op unless the problem can be rewritten as several separate all-reductions on primitive arrays.

In practice, separate reductions on primitive fields are sometimes simpler and easier to debug.

Common Pitfalls

A common mistake is trying to use MPI_Allreduce only to distribute one struct. That is a broadcast problem, not a reduction problem.

Another mistake is creating the custom datatype but forgetting to define a matching custom reduction operation.

Developers also sometimes assume struct fields are tightly packed and compute offsets manually. Use offsetof or MPI address utilities instead.

Finally, make sure your custom reduction is associative and meaningful for the intended collective semantics. If the rule is not well-defined, the result is not trustworthy.

Summary

• 'MPI_Allreduce is for reduction plus distribution, not just sending a struct.'
• Custom structs need an MPI datatype that matches their memory layout.
• Arbitrary structs also need a reduction rule, usually via a custom MPI_Op.
• If you only need distribution, prefer MPI_Bcast instead.
• Sometimes separate reductions on primitive fields are simpler than reducing a struct directly.