How to serve the old (expired) value in case of error?

[mickaeltr]

Hello, I am working on a project using Caffeine (2.9) with Spring Boot.

I'd like to implement a behaviour similar to the stale-if-error Cache-Control HTTP header:

The stale-if-error HTTP Cache-Control extension allows a cache to return a stale response when an error -- e.g., a 500 Internal Server Error, a network segment, or DNS failure -- is encountered, rather than returning a "hard" error. This improves availability.

The question was already raised on StackOverflow and got an answer that I could adapt to Caffeine. However it seems to me that it's not exactly behaving accordingly. As far as I could see, after the cache expiration, the old value is always returned on the first hit (then the refreshed value on the second hit).

Do you think what I am trying to achieve is possible or would it be a new feature? Thanks

---

Here is what I've done:

@Configuration
@EnableCaching
public class CacheConfig extends CachingConfigurerSupport {

    private static final Duration CACHE_DURATION = Duration.ofHours(1);

    @Bean
    @Override
    public CacheManager cacheManager() {
        var cacheManager = new CaffeineCacheManager();
        cacheManager.setCaffeine(
            Caffeine.newBuilder().refreshAfterWrite(CACHE_DURATION)
        );
        cacheManager.setCacheLoader(key -> {
            var cacheKey = (CacheKey) key;
            return cacheKey.method.invoke(cacheKey.target, cacheKey.params);
        });
        return cacheManager;
    }

    @Bean
    @Override
    public KeyGenerator keyGenerator() {
        return CacheKey::new;
    }

    private static class CacheKey extends SimpleKey {
        private final Object target;
        private final Method method;
        private final Object[] params;

        private CacheKey(Object target, Method method, Object... params) {
            super(params);
            this.target = target;
            this.method = method;
            this.params = params;
        }
    }
}

[ben-manes]

The guava style of returning if the refresh completed on the triggering thread was recently implemented in #688. Can you try master (snapshot or jitpack) to see if that fits your needs? The caller would need to return a completed future by overriding asyncReload.

[mickaeltr]

Thanks @ben-manes for your responsiveness. I've just tried with com.github.ben-manes:caffeine:d9f132cefe9a0833686354bd9986832e4f7d6beb and it seems like the behavior is still:

1. on first hit, loads the value and returns it
2. keeps returning the cached value
3. on first hit after the expiration, returns the stale value and reloads asynchronously
4. on second hit, returns the new value

[ben-manes]

Thanks @mickaeltr. Can you write an isolated unit test for me to debug with? That would avoid any gafs on my side by testing something different than what you're doing.

[ben-manes]

3. on first hit after the expiration, returns the stale value and reloads asynchronously

If it reloads asynchronously then it is fire-and-forget. Guava has the same except by default it runs on the calling thread,

  public ListenableFuture<V> reload(K key, V oldValue) throws Exception {
    checkNotNull(key);
    checkNotNull(oldValue);
    return Futures.immediateFuture(load(key));
  }

In Caffeine we have Caffeine.executor which defaults to ForkJoinPool.commonPool() so the operation is asynchronous by default.

If you want the same behavior as Guava's then you would override asyncReload to return a completed future,

caffeine/caffeine/src/main/java/com/github/benmanes/caffeine/cache/CacheLoader.java

Lines 192 to 205 in d9f132c

	default CompletableFuture<? extends V> asyncReload(
	K key, V oldValue, Executor executor) throws Exception {
	requireNonNull(key);
	requireNonNull(executor);
	return CompletableFuture.supplyAsync(() -> {
	try {
	return reload(key, oldValue);
	} catch (RuntimeException e) {
	throw e;
	} catch (Exception e) {
	throw new CompletionException(e);
	}
	}, executor);
	}

[mickaeltr]

I created reproduction tests:

• current behavior: https://github.com/mickaeltr/caffeine-stale-if-error/blob/main/src/test/java/com/example/demo/CacheTest.java#L27
• behavior I'd like to have: https://github.com/mickaeltr/caffeine-stale-if-error/blob/main/src/test/java/com/example/demo/CacheTest.java#L54

I'll try later to override the asyncReload as you suggested. Thanks again.

[ben-manes]

It looks like jitpack did not replace the Spring dependency so (in my IDE) it resolves to v2. I switched to Sonatype's repository for 3.1.0-SNAPSHOT. That passes the test_stale_if_error and fails test_stale_while_revalidate as desired.

patch.txt

[mickaeltr]

Thanks a lot! 🙌

I confirm it should work starting from version 3.1.0, with the following configuration:

import java.lang.reflect.Method;
import java.time.Duration;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.Executor;

import org.springframework.cache.CacheManager;
import org.springframework.cache.annotation.CachingConfigurerSupport;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.cache.caffeine.CaffeineCacheManager;
import org.springframework.cache.interceptor.KeyGenerator;
import org.springframework.cache.interceptor.SimpleKey;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.lang.NonNull;

import com.github.benmanes.caffeine.cache.CacheLoader;
import com.github.benmanes.caffeine.cache.Caffeine;

@Configuration
@EnableCaching
public class CacheConfig extends CachingConfigurerSupport {

    private static final Duration CACHE_DURATION = Duration.ofSeconds(1);

    @Bean
    @Override
    public CacheManager cacheManager() {
        var cacheManager = new CaffeineCacheManager();
        cacheManager.setCaffeine(Caffeine.newBuilder().refreshAfterWrite(CACHE_DURATION));
        cacheManager.setCacheLoader(new CacheLoader<>() {
            @Override
            public Object load(@NonNull Object key) throws Exception {
                var cacheKey = (CacheKey) key;
                return cacheKey.method.invoke(cacheKey.target, cacheKey.params);
            }

            @Override
            @NonNull
            public CompletableFuture<Object> asyncReload(
                @NonNull Object key,
                @NonNull Object oldValue,
                @NonNull Executor executor
            ) {
                try {
                    return CompletableFuture.completedFuture(load(key));
                } catch (Exception e) {
                    return CompletableFuture.failedFuture(e);
                }
            }
        });
        return cacheManager;
    }

    @Bean
    @Override
    public KeyGenerator keyGenerator() {
        return CacheKey::new;
    }

    private static class CacheKey extends SimpleKey {
        private final Object target;
        private final Method method;
        private final Object[] params;

        private CacheKey(Object target, Method method, Object... params) {
            super(params);
            this.target = target;
            this.method = method;
            this.params = params;
        }
    }
}

[ben-manes]

Released in 3.1.0