OCaml
Async programming
Concurrent write
Functional programming
OCaml libraries

Concurrent write with OCaml Async

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

With OCaml Async, concurrent writes are usually safe only when you define a clear serialization point. Async is cooperative, not magically thread-safe, so if multiple jobs write to the same file or socket, you should decide explicitly whether those writes must be ordered, buffered, or coordinated through a single writer loop.

Async Concurrency Is Cooperative

Async concurrency is built around Deferred.t values and a single-threaded scheduler. Multiple jobs can be pending at once, but they run cooperatively rather than preemptively. That means shared resources still need structure.

For writing, the key questions are:

  • are multiple jobs writing to the same destination
  • must writes stay in order
  • do you need to wait until bytes are flushed

If you skip those questions, “concurrent write” quickly turns into interleaved or poorly timed output.

Sequential Writes to One Writer.t

If order matters, the safest pattern is to keep one writer and serialize operations:

ocaml
1open Core
2open Async
3
4let write_lines writer lines =
5  Deferred.List.iter lines ~how:`Sequential ~f:(fun line ->
6    Writer.write_line writer line;
7    Writer.flushed writer)
8
9let () =
10  don't_wait_for begin
11    let%bind writer = Writer.open_file "output.txt" in
12    let%bind () = write_lines writer [ "one"; "two"; "three" ] in
13    Writer.close writer
14  end;
15  never_returns (Scheduler.go ())

This is not “parallel,” but it is often the correct answer because the destination is inherently sequential.

A Queue Is Better Than Many Call Sites Writing Directly

If many concurrent jobs want to write to the same destination, a queue or pipe is cleaner than letting all of them touch the writer directly:

ocaml
1open Core
2open Async
3
4let start_writer_loop file =
5  let reader, writer_pipe = Pipe.create () in
6  let%bind writer = Writer.open_file file in
7  don't_wait_for (
8    Pipe.iter reader ~f:(fun line ->
9      Writer.write_line writer line;
10      Writer.flushed writer)
11    >>= fun () ->
12    Writer.close writer
13  );
14  return writer_pipe
15
16let () =
17  don't_wait_for begin
18    let%bind pipe = start_writer_loop "output.txt" in
19    let jobs =
20      List.init 5 ~f:(fun i ->
21        Pipe.write pipe (sprintf "message %d" i))
22    in
23    let%bind () = Deferred.all_unit jobs in
24    Pipe.close pipe;
25    return ()
26  end;
27  never_returns (Scheduler.go ())

This pattern keeps concurrency in the producers while preserving a single ordered write path.

Distinguish Buffering from Flushing

Writer.write and Writer.write_line enqueue bytes into Async’s writer buffer. They do not mean the bytes are already on disk or on the socket. If you need durability or network ordering guarantees, wait for:

ocaml
Writer.flushed writer

That is one of the most important semantics in the whole topic. A write call schedules output; flushed tells you the buffered data has actually been handed off more fully.

Writing to Different Files Is Easier

If each concurrent job writes to its own file or its own connection, coordination is much simpler because there is no shared destination:

ocaml
1let write_file file content =
2  let%bind writer = Writer.open_file file in
3  Writer.write writer content;
4  let%bind () = Writer.flushed writer in
5  Writer.close writer

In that case you can run many writes concurrently with Deferred.all_unit, because each writer owns its own resource.

Common Pitfalls

The most common mistake is assuming that because Async supports many concurrent jobs, multiple jobs should all write to the same writer independently. Without a serialization strategy, ordering becomes hard to reason about.

Another pitfall is forgetting the difference between buffered and flushed writes. If the program exits or closes too early, queued data may not be written when you think it is.

It is also easy to overcomplicate the design. If the destination is one file and order matters, a single writer loop is usually the simplest correct solution.

Finally, do not confuse Async concurrency with OS-threaded parallel writes. Async makes I/O composition cleaner, but shared output resources still need explicit coordination.

Summary

  • In Async, concurrent producers should usually feed a single serialized writer for one shared destination.
  • Use Writer.write or Writer.write_line to enqueue bytes and Writer.flushed when completion matters.
  • A Pipe is a good coordination mechanism when many jobs need to submit writes.
  • Writing to separate files or sockets is easier because each resource can be owned independently.
  • The right design depends more on ordering and flush requirements than on the word “concurrent.”

Course illustration
Course illustration

All Rights Reserved.