fastcache

FastCache

A production-ready, goroutine-safe in-memory key-value cache designed to handle 1M+ QPS with automatic memory management and LRU eviction.

✨ Features

• 🚀 High Performance: Optimized for 1M+ QPS with minimal latency (<1μs)
• 🔒 Thread-Safe: Goroutine-safe concurrent operations
• 💾 Memory Management: Automatic eviction with configurable limits
• ⚡ Non-blocking: Lock-free operations where possible
• ⏰ TTL Support: Automatic expiration of entries
• 🔄 LRU Eviction: Least Recently Used eviction policy
• 🎯 Sharded Design: Reduced lock contention through sharding
• 📊 Rich Monitoring: Comprehensive statistics and metrics
• 🏭 Production Ready: Battle-tested with extensive error handling
• 🎨 Zero Dependencies: Pure Go implementation

📦 Installation

go get github.com/nayan9229/fastcache

🚀 Quick Start

package main

import (
    "fmt"
    "time"
    "github.com/nayan9229/fastcache"
)

func main() {
    // Create cache with default settings (512MB, 1024 shards)
    cache := fastcache.New(fastcache.DefaultConfig())
    defer cache.Close()

    // Set a value
    cache.Set("user:123", map[string]interface{}{
        "name":  "John Doe",
        "email": "[email protected]",
    })

    // Get a value
    if value, exists := cache.Get("user:123"); exists {
        fmt.Printf("Found: %+v\n", value)
    }

    // Set with TTL
    cache.Set("session:abc", "session-data", 10*time.Minute)

    // Get statistics
    stats := cache.GetStats()
    fmt.Printf("Hit ratio: %.2f%%, Memory: %s\n", 
        stats.HitRatio*100, stats.MemoryUsage)
}

📊 Performance

Based on comprehensive benchmarking:

Tested on: Intel i7-10700K, 32GB RAM, Go 1.21

⚙️ Configuration

// High-performance configuration for 1M+ QPS
config := &fastcache.Config{
    MaxMemoryBytes:  512 * 1024 * 1024, // 512MB
    ShardCount:      1024,               // High concurrency
    DefaultTTL:      time.Hour,          // 1 hour default
    CleanupInterval: time.Minute,        // Cleanup frequency
}

cache := fastcache.New(config)

Pre-built Configurations

// For high-concurrency scenarios
cache := fastcache.New(fastcache.HighConcurrencyConfig())

// For memory-constrained environments
cache := fastcache.New(fastcache.LowMemoryConfig())

// Custom configuration
cache := fastcache.New(fastcache.CustomConfig(256, 512, 30*time.Minute))

🔧 API Reference

Core Operations

// Set with optional TTL
err := cache.Set(key string, value interface{}, ttl ...time.Duration)

// Get value
value, exists := cache.Get(key string)

// Delete key
deleted := cache.Delete(key string)

// Clear all entries
cache.Clear()

// Close cache
cache.Close()

Monitoring & Statistics

// Get comprehensive statistics
stats := cache.GetStats()
// Returns: entries, memory usage, hit/miss ratios, etc.

// Get detailed memory information
memInfo := cache.GetMemoryInfo()
// Returns: used, available, percentage, shard distribution

// Get performance metrics
perfMetrics := cache.GetPerformanceMetrics()
// Returns: operation counts, load balance, shard statistics

🏗️ Architecture

FastCache uses a sharded architecture to achieve high concurrency:

┌─────────────────────────────────────┐
│            FastCache                │
├─────────────────────────────────────┤
│  Shard 0  │  Shard 1  │  Shard N   │
│  ┌─────┐  │  ┌─────┐  │  ┌─────┐   │
│  │ Map │  │  │ Map │  │  │ Map │   │
│  │ LRU │  │  │ LRU │  │  │ LRU │   │
│  └─────┘  │  └─────┘  │  └─────┘   │
└─────────────────────────────────────┘

Key Features:

• FNV Hash Distribution: Even key distribution across shards
• Per-shard LRU: Independent LRU management
• RWMutex Locking: Multiple concurrent readers
• Atomic Counters: Lock-free statistics

🌐 Production Usage

API Server Integration

type APIServer struct {
    cache *fastcache.Cache
}

func (s *APIServer) GetUser(userID string) (*User, error) {
    // Try cache first
    if cached, exists := s.cache.Get("user:" + userID); exists {
        return cached.(*User), nil
    }
    
    // Cache miss - fetch from database
    user, err := s.fetchFromDB(userID)
    if err != nil {
        return nil, err
    }
    
    // Cache the result
    s.cache.Set("user:"+userID, user, 15*time.Minute)
    return user, nil
}

Monitoring Setup

// Periodic statistics logging
go func() {
    ticker := time.NewTicker(time.Minute)
    defer ticker.Stop()
    
    for range ticker.C {
        stats := cache.GetStats()
        log.Printf("Cache: entries=%d memory=%s hit_ratio=%.2f%%",
            stats.TotalEntries, stats.MemoryUsage, stats.HitRatio*100)
    }
}()

📁 Examples

The repository includes comprehensive examples:

# Basic usage
go run examples/basic/main.go

# API server integration
go run examples/api-server/main.go

# High concurrency testing
go run examples/high-concurrency/main.go

# Monitoring and metrics
go run examples/monitoring/main.go

🧪 Testing & Benchmarks

# Run all tests
make test

# Run benchmarks
make benchmark

# Run load tests
make load-test

# Generate coverage report
make test-coverage

# Run performance tests
make performance-test

Benchmark Results

$ make benchmark
BenchmarkSet-8              2000000    500 ns/op    64 B/op    1 allocs/op
BenchmarkGet-8              5000000    200 ns/op     0 B/op    0 allocs/op
BenchmarkMixed-8            1500000    800 ns/op    32 B/op    1 allocs/op
BenchmarkHighConcurrency-8  1200000   1000 ns/op    48 B/op    1 allocs/op

🛠️ Development

Prerequisites

• Go 1.21 or higher
• Make (optional, for convenient commands)

Setup

# Clone repository
git clone https://github.com/nayan9229/fastcache.git
cd fastcache

# Install development dependencies
make dev-deps

# Run quick development cycle
make quick

Available Make Targets

make help           # Show all available targets
make test           # Run tests
make benchmark      # Run benchmarks
make lint           # Run linter
make format         # Format code
make build          # Build examples
make docs           # Generate documentation

📈 Memory Management

FastCache automatically manages memory through:

1. Size Tracking: Real-time memory usage monitoring
2. LRU Eviction: Removes least recently used entries when memory limit is reached
3. TTL Cleanup: Background cleanup of expired entries
4. Configurable Limits: Flexible memory constraints

// Monitor memory usage
memInfo := cache.GetMemoryInfo()
if memInfo.Percent > 80.0 {
    log.Warn("Cache memory usage above 80%")
}

🔒 Thread Safety

All operations are goroutine-safe:

• Read Operations: Use RWMutex for concurrent reads
• Write Operations: Protected by exclusive locks
• Statistics: Atomic operations for counters
• Memory Management: Thread-safe eviction and cleanup

🚨 Error Handling

// Check for errors
if err := cache.Set("key", "value"); err != nil {
    if errors.Is(err, fastcache.ErrCacheClosed) {
        // Handle closed cache
    }
}

// Graceful shutdown
defer func() {
    if err := cache.Close(); err != nil {
        log.Printf("Error closing cache: %v", err)
    }
}()

📋 Best Practices

1. Choose Appropriate Shard Count

// High concurrency (1000+ goroutines)
config.ShardCount = 1024

// Medium concurrency (100-1000 goroutines)  
config.ShardCount = 256

// Low concurrency (<100 goroutines)
config.ShardCount = 64

2. Optimize TTL Values

// Fast-changing data
cache.Set(key, value, 1*time.Minute)

// Slow-changing data
cache.Set(key, value, 1*time.Hour)

// Static data (manual eviction)
cache.Set(key, value, 0)

3. Monitor Performance

stats := cache.GetStats()
if stats.HitRatio < 0.8 {
    // Consider increasing cache size or adjusting TTL
}

4. Handle Cache Misses Gracefully

value, exists := cache.Get(key)
if !exists {
    // Fetch from primary source
    value = fetchFromDatabase(key)
    cache.Set(key, value, defaultTTL)
}
return value

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Make your changes and add tests
4. Run the test suite: make test
5. Commit your changes: git commit -m 'Add amazing feature'
6. Push to the branch: git push origin feature/amazing-feature
7. Submit a pull request

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Inspired by various high-performance caching solutions
• Built with Go's excellent concurrency primitives
• Tested against real-world production workloads

📞 Support

• 🐛 Bug Reports: GitHub Issues
• 💡 Feature Requests: GitHub Discussions
• 📧 Email: [email protected]
• 📖 Documentation: GoDoc

⭐ If you find FastCache useful, please consider giving it a star on GitHub!