echocache

EchoCache

EchoCache is a generic, concurrency-safe Go caching library that prevents redundant computations under high load by combining a pluggable storage backend with singleflight deduplication: when many goroutines request the same missing key at the same time, only one refresh function is executed and the result is shared with all waiting callers.

Two caching modes are available:

Features

• Generic – works with any Go type (string, structs, slices, …).
• Singleflight deduplication – collapses concurrent misses for the same key into a single computation.
• Stale-while-revalidate – serve stale data instantly and refresh in the background.
• Pluggable backends – swap storage without changing application code:
  • In-memory LRU (NewLRUCache) – bounded, no expiry.
  • In-memory Expirable LRU (NewLRUExpirableCache) – bounded + per-entry TTL.
  • In-memory Single-entry (NewSingleCache) – single slot with TTL, zero allocations on hits.
  • Redis (NewRedisCache) – persistent, distributed, JSON serialisation.
  • NATS JetStream (NewNatsCache) – distributed KeyValue-backed cache.

Installation

go get github.com/logocomune/echocache

Requires Go 1.21+.

Quick Start

Synchronous cache (EchoCache)

package main

import (
    "context"
    "fmt"
    "time"

    "github.com/logocomune/echocache"
    "github.com/logocomune/echocache/store"
)

func main() {
    // Create a cache backed by a 128-entry in-memory LRU.
    cache := echocache.NewEchoCache[string](store.NewLRUCache[string](128))

    ctx := context.Background()
    key := "greeting"

    // First call: cache miss – the refresh function is executed.
    value, cached, err := cache.FetchWithCache(ctx, key, func(ctx context.Context) (string, error) {
        time.Sleep(50 * time.Millisecond) // simulate slow work
        return "Hello, World!", nil
    })
    fmt.Printf("value=%q cached=%v err=%v\n", value, cached, err)
    // Output: value="Hello, World!" cached=true err=<nil>

    // Second call: cache hit – refresh function is NOT executed.
    value, cached, err = cache.FetchWithCache(ctx, key, func(ctx context.Context) (string, error) {
        panic("should not be called")
    })
    fmt.Printf("value=%q cached=%v err=%v\n", value, cached, err)
    // Output: value="Hello, World!" cached=true err=<nil>
}

Concurrent deduplication

When many goroutines request the same key simultaneously, only one refresh is performed. All others wait and receive the same result.

package main

import (
    "context"
    "fmt"
    "sync"
    "time"

    "github.com/logocomune/echocache"
    "github.com/logocomune/echocache/store"
)

func main() {
    cache := echocache.NewEchoCache[string](store.NewLRUCache[string](128))
    ctx := context.Background()

    var wg sync.WaitGroup
    for i := range 10 {
        wg.Add(1)
        go func(id int) {
            defer wg.Done()
            start := time.Now()
            val, _, _ := cache.FetchWithCache(ctx, "shared-key", func(ctx context.Context) (string, error) {
                time.Sleep(200 * time.Millisecond)
                return "computed once", nil
            })
            fmt.Printf("goroutine %d: %q elapsed=%v\n", id, val, time.Since(start).Round(time.Millisecond))
        }(i)
    }
    wg.Wait()
    // All goroutines complete in ~200 ms, not 2 000 ms.
}

Stale-while-revalidate (EchoCacheLazy)

Return the cached value immediately (even if stale) and refresh silently in the background.

package main

import (
    "context"
    "fmt"
    "time"

    "github.com/logocomune/echocache"
    "github.com/logocomune/echocache/store"
)

type Weather struct {
    City        string
    Temperature float64
}

func main() {
    // LRU backend – no TTL; the lazy refresh interval controls staleness.
    backend := store.NewStaleWhileRevalidateLRUCache[Weather](128)
    cache := echocache.NewLazyEchoCache[Weather](backend, 5*time.Second)
    defer cache.ShutdownLazyRefresh()

    ctx := context.Background()
    refresh := func(ctx context.Context) (Weather, error) {
        // Simulate a slow external API call.
        time.Sleep(300 * time.Millisecond)
        return Weather{City: "Rome", Temperature: 22.5}, nil
    }

    // First call: cold cache – blocks until the value is fetched (~300 ms).
    w, _, _ := cache.FetchWithLazyRefresh(ctx, "weather:rome", refresh, 10*time.Second)
    fmt.Printf("%s: %.1f°C\n", w.City, w.Temperature)

    time.Sleep(11 * time.Second) // wait until the 10-second refresh interval expires

    // Second call: returns the stale value INSTANTLY and queues a background refresh.
    w, _, _ = cache.FetchWithLazyRefresh(ctx, "weather:rome", refresh, 10*time.Second)
    fmt.Printf("%s: %.1f°C (stale – refresh enqueued)\n", w.City, w.Temperature)
}

Using a Redis backend

import (
    "github.com/logocomune/echocache"
    "github.com/logocomune/echocache/store"
    "github.com/redis/go-redis/v9"
    "time"
)

rdb := redis.NewClient(&redis.Options{Addr: "localhost:6379"})

// Simple cache
cache := echocache.NewEchoCache[string](
    store.NewRedisCache[string](rdb, "myapp", 5*time.Minute),
)

// Lazy (stale-while-revalidate) cache
lazyBackend := store.NewStaleWhileRevalidateRedisCache[string](rdb, "myapp", 5*time.Minute)
lazyCache := echocache.NewLazyEchoCache[string](lazyBackend, 10*time.Second)
defer lazyCache.ShutdownLazyRefresh()

Using a NATS JetStream backend

import (
    "github.com/logocomune/echocache"
    "github.com/logocomune/echocache/store"
    "github.com/nats-io/nats.go"
    "github.com/nats-io/nats.go/jetstream"
)

nc, _ := nats.Connect(nats.DefaultURL)
js, _ := jetstream.New(nc)
kv, _ := js.CreateKeyValue(context.Background(), jetstream.KeyValueConfig{Bucket: "mycache"})

cache := echocache.NewEchoCache[string](store.NewNatsCache[string](kv, "prefix"))

Storage backends reference

Benchmarks

Run with:

go test ./... -bench=. -benchmem

Representative results on an AMD Ryzen 7 5800H (16 threads):

BenchmarkFetchWithCache_CacheHit-16              43965662    53.5 ns/op    0 B/op    0 allocs/op
BenchmarkFetchWithCache_CacheMiss-16              2335598   958.4 ns/op  319 B/op    5 allocs/op
BenchmarkFetchWithCache_Concurrent-16            46195622    56.7 ns/op    0 B/op    0 allocs/op
BenchmarkFetchWithLazyRefresh_CacheHit-16        15728220   159.3 ns/op    0 B/op    0 allocs/op
BenchmarkFetchWithLazyRefresh_CacheHitStale-16   24293737    99.0 ns/op    0 B/op    0 allocs/op

BenchmarkLRUCache_Get-16                         49176892    48.9 ns/op    0 B/op    0 allocs/op
BenchmarkLRUCache_Set-16                         44030736    56.8 ns/op    0 B/op    0 allocs/op
BenchmarkLRUExpirableCache_Get-16                26374305    86.9 ns/op    0 B/op    0 allocs/op
BenchmarkLRUExpirableCache_Set-16                20437982   117.7 ns/op    0 B/op    0 allocs/op
BenchmarkSingleCache_Get-16                      87536472    28.2 ns/op    0 B/op    0 allocs/op
BenchmarkSingleCache_Set-16                      26237246    93.0 ns/op    0 B/op    0 allocs/op

License

Distributed under the MIT license.