MemoryCache

To the time to life, rather than to life in time.

中文

Description

Minimalist in-memory KV storage, powered by HashMap and Minimal Quad Heap.

Cache Eviction Policy:

• Set evicts LRU keys on capacity overflow
• Periodic cycle evicts expired keys

Design

• Storage Data Limit: Limited by maximum capacity
• Expiration Time: Supported
• Cache Eviction Policy: LRU
• Persistent: None
• Locking Mechanism: Slicing + Mutual Exclusion Locking
• HashMap, Heap and LinkedList (excluding user KVs) implemented in pointerless technology

Advantage

• Simple and easy to use
• High performance
• Low memory usage
• Uses a 4-ary min-heap to maintain expiration times. Compared to binary heaps, this reduces tree height and improves insertion performance.

Methods

• Set : Set key-value pair with expiring time. If the key already exists, the value will be updated. Also the expiration time will be updated. exp<=0 means never expire.
• SetWithCallback : Similar to Set, but with a callback function. Triggered on capacity overflow, expiration, deletion, or clear.
• Get : Get value by key. If the key does not exist, the second return value will be false.
• GetTTL : Get the expiration time of the key.
• GetWithTTL : Get value by key. If the key exists, refreshes the expiration time.
• UpdateTTL : Update the expiration time of the key. d<=0 means never expire.
• Delete : Delete key-value pair by key.
• GetOrCreate : Get value by key. If the key does not exist, the value will be created.
• GetOrCreateWithCallback : Get value by key. If the key does not exist, the value will be created with an optional callback.
• Clear : Clear all caches. Triggers ReasonCleared callback for alive non-expired elements.
• Range : Iterate all key-value pairs in the cache.
• Len : Get the current number of cached elements (may include expired but un-cleaned elements).
• Stop : Stop the background goroutines, drain pending callbacks, and release resources. Do not use the instance after Stop.

Callbacks

Callbacks registered via SetWithCallback or GetOrCreateWithCallback are triggered on expiration, LRU eviction, explicit deletion, or Clear. Dispatch mode is controlled by WithAsyncCallback (default: async).

Async mode (WithAsyncCallback(true), default):

• Dispatched through an internal task queue (queues.Queue).
• Executed outside bucket locks — does not block cache operations or risk deadlocks from reentrant cache access.
• The *Element is a snapshot copied before the entry is recycled.
• Delivery is best-effort and unordered across keys; same hash shard is serialized (concurrency=1 per shard).
• Panics are recovered and logged (WithRecovery).
• Stop() drains pending callbacks (up to 30s timeout). After Stop, further mutations (if any) run callbacks synchronously so they are not silently dropped.

Sync mode (WithAsyncCallback(false)):

• Callback runs inline under the bucket lock before the entry is recycled.
• The *Element is a live reference valid only for the duration of the callback.
• Do not call back into MemoryCache from the callback — this may deadlock.
• No task queue is created; Stop() does not wait for callbacks.

Options

Option	Default	Description
WithBucketNum(num)	16	Number of storage buckets (auto-rounded to power of 2)
WithBucketSize(size, cap)	1000, 100000	Initial size and max capacity per bucket
WithInterval(min, max)	5s, 30s	Adaptive TTL check period
WithDeleteLimits(num)	1000	Max keys deleted per TTL check per bucket
WithCachedTime(enabled)	true	Enable time caching for lower time.Now() overhead
WithAsyncCallback(enabled)	true	Dispatch callbacks asynchronously via internal queue
WithSwissTable(enabled)	false	Use SwissTable instead of Go runtime map

Example

package main

import (
	"fmt"
	"time"

	"github.com/lxzan/memorycache"
)

func main() {
	mc := memorycache.New[string, any](
		// Set the number of storage buckets, y=pow(2,x)
		memorycache.WithBucketNum(128),

		// Set bucket size, initial size and maximum capacity (per bucket)
		memorycache.WithBucketSize(1000, 10000),

		// Set the expiration time check period.
		// If the number of expired elements is small, take the maximum value, otherwise take the minimum value.
		memorycache.WithInterval(5*time.Second, 30*time.Second),
	)
	defer mc.Stop() // Stop background goroutines and release resources

	mc.SetWithCallback("xxx", 1, time.Second, func(element *memorycache.Element[string, any], reason memorycache.Reason) {
		fmt.Printf("callback: key=%s, reason=%v\n", element.Key, reason)
	})

	val, exist := mc.Get("xxx")
	fmt.Printf("val=%v, exist=%v\n", val, exist)

	time.Sleep(2 * time.Second)

	val, exist = mc.Get("xxx")
	fmt.Printf("val=%v, exist=%v\n", val, exist)
}

Benchmark

• Pre-populated with 1,000,000 elements. benchtime=3s, parallel, 10 CPU cores.

goos: darwin
goarch: arm64
pkg: github.com/lxzan/memorycache/benchmark
cpu: Apple M1 Max
BenchmarkMemoryCache_Set-10           90382902        41.77 ns/op        4 B/op        0 allocs/op
BenchmarkMemoryCache_Get-10           79059558        50.51 ns/op        0 B/op        0 allocs/op
BenchmarkMemoryCache_SetAndGet-10     78810615        48.48 ns/op        0 B/op        0 allocs/op
BenchmarkMemoryCache_Delete-10       153390056        23.76 ns/op        0 B/op        0 allocs/op
BenchmarkMemoryCache_UpdateTTL-10    100000000        32.73 ns/op        0 B/op        0 allocs/op
BenchmarkRistretto_Set-10             68245581       673.1 ns/op        112 B/op        2 allocs/op
BenchmarkRistretto_Get-10             86053975        45.96 ns/op         17 B/op        1 allocs/op
BenchmarkRistretto_SetAndGet-10       27956532       110.6 ns/op         31 B/op        1 allocs/op
BenchmarkTheine_Set-10                 9754111       381.7 ns/op         22 B/op        0 allocs/op
BenchmarkTheine_Get-10                32876373       109.1 ns/op          0 B/op        0 allocs/op
BenchmarkTheine_SetAndGet-10          23014824       172.4 ns/op          0 B/op        0 allocs/op
PASS
ok      github.com/lxzan/memorycache/benchmark  103.750s