Introduction

Windows asynchronous I O in C is powerful, but many programs accidentally run synchronously even when ReadFile or WriteFile receives an OVERLAPPED argument. Most confusion comes from handle flags and completion strategy, not from the read call itself. If you pick one completion model and follow its rules consistently, behavior becomes predictable.

Why Code Executes Synchronously By Accident

Three mistakes cause most false async behavior.

• Handle not opened with FILE_FLAG_OVERLAPPED.
• Program waits immediately after each request, removing concurrency.
• One OVERLAPPED structure reused unsafely across concurrent operations.

If any of these happen, the code may still compile and even pass small tests, but throughput stays similar to blocking I O.

Minimal Correct Overlapped Read Pattern

This example uses event based completion for one file read.

1#include <windows.h>
2#include <stdio.h>
3
4int main(void) {
5    HANDLE hFile = CreateFileA(
6        "example.txt",
7        GENERIC_READ,
8        FILE_SHARE_READ,
9        NULL,
10        OPEN_EXISTING,
11        FILE_FLAG_OVERLAPPED,
12        NULL
13    );
14
15    if (hFile == INVALID_HANDLE_VALUE) {
16        printf("CreateFile failed: %lu\n", GetLastError());
17        return 1;
18    }
19
20    char buffer[128] = {0};
21    OVERLAPPED ov = {0};
22    ov.hEvent = CreateEventA(NULL, TRUE, FALSE, NULL);
23
24    if (!ov.hEvent) {
25        CloseHandle(hFile);
26        return 1;
27    }
28
29    BOOL ok = ReadFile(hFile, buffer, sizeof(buffer) - 1, NULL, &ov);
30    if (!ok) {
31        DWORD err = GetLastError();
32        if (err != ERROR_IO_PENDING) {
33            printf("ReadFile failed: %lu\n", err);
34            CloseHandle(ov.hEvent);
35            CloseHandle(hFile);
36            return 1;
37        }
38    }
39
40    WaitForSingleObject(ov.hEvent, INFINITE);
41
42    DWORD bytesRead = 0;
43    if (!GetOverlappedResult(hFile, &ov, &bytesRead, FALSE)) {
44        printf("GetOverlappedResult failed: %lu\n", GetLastError());
45    } else {
46        buffer[bytesRead] = '\0';
47        printf("Read %lu bytes: %s\n", bytesRead, buffer);
48    }
49
50    CloseHandle(ov.hEvent);
51    CloseHandle(hFile);
52    return 0;
53}

Key point: ERROR_IO_PENDING is not an error state here. It means operation started and will complete later.

Choosing A Completion Method

Windows gives several valid completion models. Pick based on workload shape.

• Event per request: simple and explicit, fine for low concurrency tools.
• Completion routine with alertable waits: useful but harder to reason about.
• I O completion ports: best for high concurrency servers.

For server workloads, IOCP is usually the right long term choice because one queue can coordinate many handles efficiently.

1/* IOCP initialization sketch */
2HANDLE iocp = CreateIoCompletionPort(INVALID_HANDLE_VALUE, NULL, 0, 0);
3if (!iocp) {
4    return 1;
5}
6
7/* Associate a file or socket handle with the port. */
8CreateIoCompletionPort(fileHandle, iocp, (ULONG_PTR)fileHandle, 0);
9
10DWORD bytes;
11ULONG_PTR key;
12LPOVERLAPPED pov;
13BOOL status = GetQueuedCompletionStatus(iocp, &bytes, &key, &pov, INFINITE);
14if (!status) {
15    DWORD err = GetLastError();
16    /* Handle completion failure. */
17}

Throughput Versus Latency Reality

Even with proper overlapped I O, you only gain throughput if multiple operations are in flight. Submitting one operation and waiting immediately gives little benefit over synchronous code.

A useful benchmark plan:

1. Measure single request latency in both modes.
2. Measure total work time at high concurrency.
3. Record CPU usage and context switch behavior.

If throughput does not improve, inspect wait points and request batching before changing APIs again.

Memory And Lifetime Rules

Each in flight operation needs stable buffers and its own OVERLAPPED memory until completion. Stack allocated structures that go out of scope early are a common source of corruption and intermittent crashes.

Also close handles only after all pending operations are resolved or canceled. Handle lifetime races are hard to debug in production.

Common Pitfalls

• Forgetting FILE_FLAG_OVERLAPPED on handle creation.
• Treating ERROR_IO_PENDING as fatal.
• Waiting right after submit and eliminating overlap.
• Reusing OVERLAPPED and buffers before completion.
• Mixing completion models in one module without clear ownership.

Summary

• Correct async Windows I O starts with overlapped handle creation.
• Use one completion model intentionally and apply it consistently.
• ERROR_IO_PENDING usually means success in progress.
• Real gains come from multiple concurrent in flight operations.
• Buffer and OVERLAPPED lifetime management is critical for correctness.