reload immediately on expiry

[koushikroy1989]

I am maintaining a cache of key value both are string. Each entry has its own expiry time. So I had this below working code where it reloads the data whenever any expired key is requested. As the loading of data in my case is time consuming and coming from external services, I see a need to reload the data in the background immediately whenever the expiry time passed. So when a request to get value comes after expiry, cache will be having a refreshed data already.

I am using Java 8 so heard somewhere the caffeine scheduler may not work. If you have a demo and any example of how best I can achieve this with or without scheduler that would be great. Suggestions to use a different cache framework - also welcomed.

My current code that expires the entry after a time but reloading happens only when the data is requested and adds up the external service call time to the get method.

` Caffeine.newBuilder().maximumSize(maxSize).recordStats().expireAfter(new Expiry<String, String>()
{

    @Override
    public long expireAfterCreate(@NonNull String key, @NonNull String value, long currentTime)
    {
        long expiresInSec = //decide expiry time for each entry and return with some buffer;
        return TimeUnit.SECONDS.toNanos(expiresInSec - 5);
    }

    @Override
    public long expireAfterUpdate(@NonNull String key, @NonNull String value, long currentTime,
            @NonNegative long currentDuration)
    {
        return currentDuration;
    }

    @Override
    public long expireAfterRead(@NonNull String key, @NonNull String value, long currentTime,
            @NonNegative long currentDuration)
    {
        return currentDuration;
    }

}).build();

`
Version - 2.8.0

[ben-manes]

The Scheduler can be used in Java 8, but you will have to supply your own thread. For example,

Scheduler scheduler = Scheduler.forScheduledExecutorService(es);
return Caffeine.newBuilder()
    .scheduler(scheduler)
    .expireAfter(...)
    .removalListener((key, value, cause) -> {
      if (cause == RemovalCause.EXPIRED) {
        load(key);
      }
    }).build();

In Java 9 introduced a global scheduler thread to avoid many ad hoc ones, which we expose through Scheduler.systemScheduler(). Since the 2.x release is Java 8 based, in that JVM the method falls back to returning Scheduler.disabledScheduler(). This just means you need to create and manage the lifecycle of a ScheduledExecutorService, and everything should function otherwise.

If you are needing to reload all expired data, then I am unsure if a bounded cache makes sense for you. A few other options to consider:

1. If all of the data should be in-memory, you can use an immutable map, reload it via a scheduled task, and swap the map instances.
2. Similar to (1), you might use a remote cache and periodically reload it, allowing the local caches to expire as is and fetch cheaply.
3. If you want to reload active content and allow inactive entries to expire, then use refreshAfterWrite with a shorter duration. This will trigger a reload when a read occurs on an entry that has exceeded the refresh threshold but has not yet been evicted due to expiration.

[koushikroy1989]

I don't think an expiring cache makes sense in your case, because you never want to evict an entry. Instead you want to reload it regularly on a fixed schedule. I would instead use a scheduled task.

var executor = Executors.newSingleThreadScheduledExecutor();
var cache = Caffeine.newBuilder().build(key -> /* load */);
executor.scheduleAtFixedRate(() -> {
  for (var key : cache.asMap().keySet()) {
    cache.refresh(key);
  }
}, /* initial delay */ 5, /* period */ 5, TimeUnit.SECONDS);

I understand your point. The reason why I am after this is because my each entry to the cache has its own expiry timings. That's why I am using below. Would that be possible to achieve using the scheduler example in your latest reply?

@Override public long expireAfterCreate(@NonNull String key, @NonNull String value, long currentTime) { long expiresInSec = //decide expiry time for each entry and return with some buffer; return TimeUnit.SECONDS.toNanos(expiresInSec - 5); }

[ben-manes]

If your cache is modestly sized, you might just schedule a new task per entry to support different timings.

The eviction means there can be a gap where the entry is removed before it is reloaded, which you would want to avoid. To leverage the cache would require some wrangling. Assuming v2.9 (where evictionListener replaced CacheWriter,

public class CaffeineTest {
  final Cache<Object, Object> cache;
  final Map<Object, Object> evicted;

  public CaffeineTest() {
    var executor = Executors.newSingleThreadScheduledExecutor();
    cache = Caffeine.newBuilder()
        .evictionListener((key, value, cause) -> onEviction(key, value))
        .scheduler(Scheduler.forScheduledExecutorService(executor))
        .expireAfterWrite(1, TimeUnit.SECONDS)
        .build();
    evicted = new ConcurrentHashMap<>();
  }
  
  public Object get(Object key) {
    return cache.get(key, k -> {
      Object resurrect = evicted.remove(key);
      return (resurrect == null) ? load(key) : resurrect;
    });
  }
  
  private Object load(Object key) {
    return key;
  }
  
  private void onEviction(Object key, Object value) {
    evicted.put(key, value);
    CompletableFuture.runAsync(() -> {
      cache.put(key, load(key));
      evicted.remove(key);
    });
  }
}

This stashes the evicted into a victim cache so that it can be resurrected, and then asynchronously reloads the entry. This way the caller can always obtain a value and expiration serves as a soft threshold to kick off a new load. wdyt?

[koushikroy1989]

v2.9

Thanks for all your help. Still at 2.8 so could not test the last example. I will try to bump up the version and test if feasible in my project.

[ben-manes]

In 2.8, this is nearly identical except would use CacheWriter. That is a slightly more powerful concept, but has holes which made it confusing. It was deprecated for removal in 2.9 and replaced with evictionListener so that the behavior is more explicit. Ideally you would upgrade, but if that is not feasible then you might take on a little debt by using the writer.

[krishna81m]

On a similar use case, we ended up using Refresh as it does a better job at guarantees but on access only if you want to return something while refreshing and expiration not guaranteed can take care of those that were not picked in refresh.