Caching is a crucial aspect of modern distributed systems, offering significant performance improvements by storing copies of data or computations near the clients or consumers. However, managing cache coherency and designing an effective eviction policy across a distributed environment can be challenging. This article explores the design of a distributed cache using three powerful technologies: Caffeine for local caching, Infinispan for distributed caching, and RabbitMQ for messaging to synchronize and inform cache nodes about eviction decisions.

Understanding the Components

1. Caffeine: Caffeine is a high performance, near-optimal caching library based on Java. It is designed to be a more performant drop-in replacement for LinkedHashMap. Its eviction strategies are based on recency and frequency of use, making it suitable for high read and moderate update scenarios.
2. Infinispan: Infinispan is a distributed cache and key-value NoSQL data store software developed by Red Hat. It offers advanced functionality such as transactions, events, querying and processing, and can be scaled out linearly.
3. RabbitMQ: RabbitMQ is an open-source message-broker software that implements the Advanced Message Queuing Protocol (AMQP). It facilitates the asynchronous communication among distributed systems, ensuring that messages are delivered in a fast and reliable manner, which is crucial for coordinating cache states across different nodes.

Designing the Cache Architecture

Local Caching with Caffeine

Each node in the distributed system will incorporate a local cache built with Caffeine. This local cache will store the most frequently accessed data by the application running on the node, thus reducing latency and offloading the distributed cache.

1Caffeine<Object, Object> caffeineCache = Caffeine.newBuilder()
2    .expireAfterWrite(10, TimeUnit.MINUTES)
3    .maximumSize(10_000)
4    .build();

Distributed Caching with Infinispan

Infinispan will manage the global cache state across different nodes. It will provide a more consistent and resilient cache layer. Nodes communicate with each other to maintain the cache state, and when a node's local caffeine cache does not have the requested data, it can fetch it from the Infinispan layer.

1EmbeddedCacheManager manager = new DefaultCacheManager();
2manager.defineConfiguration("local", new ConfigurationBuilder()
3    .clustering().cacheMode(CacheMode.DIST_SYNC)
4    .build());
5Cache<String, String> cache = manager.getCache("local");

Using RabbitMQ for Cache Coherency

RabbitMQ plays a crucial role in maintaining cache consistency and informing other nodes about cache entries that should be evicted based on global policies or changes in the data layer.

Each node subscribes to a specific RabbitMQ topic that broadcasts messages pertaining to cache eviction. When a node receives a message that an item should be evicted, it can then evict that item from its local Caffeine cache.

1ConnectionFactory factory = new ConnectionFactory();
2factory.setHost("localhost");
3Connection connection = factory.newConnection();
4Channel channel = connection.createChannel();
5
6String queueName = channel.queueDeclare().getQueue();
7channel.queueBind(queueName, "evictions", "");
8
9Consumer consumer = new DefaultConsumer(channel) {
10    @Override
11    public void handleDelivery(String consumerTag, Envelope envelope,
12                               AMQP.BasicProperties properties, byte[] body) throws IOException {
13        String message = new String(body, "UTF-8");
14        caffeineCache.invalidate(message);
15    }
16};
17channel.basicConsume(queueName, true, consumer);

Global Eviction Policy

The global eviction policy decides which cache entries to evict across all nodes. It could be based on least frequently used (LFU), least recently used (LRU), or a custom strategy based on your application's needs.

Key Points Summary

Conclusion

Designing a distributed cache with a globally aware eviction policy requires combining different technologies that complement each other. Caffeine provides fast local caching, Infinispan offers robust distributed caching capabilities, and RabbitMQ ensures consistency and coherency across distributed components. This layered approach not only improves performance but also increases the fault tolerance and scalability of the system.