Fast, concurrent, evicting in-memory cache written to keep big number of entries without impact on performance. BigCache keeps entries on heap but omits GC for them. To achieve that operations on bytes arrays take place, therefore entries (de) serialization in front of the cache will be needed in most use cases.
Requires Go 1. 11 or newer.
import“github.com/allegro/bigcachecache, (_) ******************************:=bigcache. ********************* (NewBigCache) ******************** (bigcache.) ******************** (DefaultConfig) ********************11* time.Minute)) cache.Set(
********************** (my-unique-key) ********************** (byte) ****************** (********************** value
)entry, (_) ******************************:=cache. ************** (Get) ************** (
When cache load can be predicted in advance then it is better to use custom initialization because additional memory allocation can be avoided in that way.
import**************** (log“**************************************** github.com/allegro/bigcache)config(*******************:=bigcache.) ****************** (Config
)( number of shards (must be a power of 2)Shards:2016( time after which entry can be evictedLifeWindow:* time.Minute, ( Interval between removing expired entries (clean up).(**************** (If set to(**************** (Setting toCleanWindow: (5) ******************** (time.) ******************* Minute, ( rps * lifeWindow, used only in initial memory allocationMaxEntriesInWindow:127511**************************************************************************************************************************,
(
max entry size in bytes, used only in initial memory allocationMaxEntrySize:, ( prints information about additional memory allocationVerbose: true, ( cache will not allocate more memory than this limit, value in MB
( if value is reached then the oldest entries can be overridden for the new ones
( 0 value means no size limitHardMaxCacheSize:050212, ( callback fired when the oldest entry is removed because of its expiration time or no space left( for the new entry, or because delete was called. A bitmask representing the reason will be returned.( Default value is nil which means no callback and it prevents from unwrapping the oldest entry.OnRemove:nil, ( OnRemoveWithReason is a callback fired when the oldest entry is removed because of its expiration time or no space left( for the new entry, or because delete was called. A constant representing the reason will be passed through.( Default value is nil which means no callback and it prevents from unwrapping the oldest entry.( Ignored if OnRemove is specified.OnRemoveWithReason:nil, }cache, (initErr**********************:=bigcache. ********************* NewBigCache(config)ifinitErr!=nillog.Fatal(initErr ) } cache.Set(
********************** (my-unique-key) ********************** (byte) ****************** (********************** value
) if**************** (entry, (err) :=cache.Get(my-unique-key”
; err==nil{ fmt.Println(string(entry)) }
Three caches were compared: bigcache, (freecacheand map. Benchmark tests were made using an i7 – (K CPU @ 4.) ****************************************************************************************************************************************************** (GHz with) *********************************************************************************************************************************** (GB of RAM on Ubuntu) ************************************************************************************************************************************************************************************************************************************************************** (LTS) 5.2.****************************************************************************************************************************** – – – generic).
Benchmarks source code can be found here
Writes and reads
go version go version go1. linux / amd (************************************************************************************************************************** gotest-bench=. -benchmem -benchtime=4s. / … -timeout m goos: linux goarch: amd pkg: github.com/allegro/bigcache/v2/caches_bench BenchmarkMapSet-8 (****************************************************************************************** (ns / op) ********************************************************************************************************** B / op 3 allocs / op BenchmarkConcurrentMapSet-8 (************************************************************************************ (ns / op) ************************************************************************************************ B / op 8 allocs / op BenchmarkFreeCacheSet-8 (**************************************************************************************** (ns / op) ************************************************************************************************** (B / op 2 allocs / op BenchmarkBigCacheSet-8 (****************************************************************************************** (ns / op) **************************************************************************************************** B / op 2 allocs / op BenchmarkMapGet-8 34222654 (ns / op) B / op 1 allocs / op BenchmarkConcurrentMapGet-8 (****************************************************************************************** (ns / op) B / op 2 allocs / op BenchmarkFreeCacheGet-8 ****************************************************************** (ns / op) ************************************************************************************************************** B / op 2 allocs / op BenchmarkBigCacheGet-8 (****************************************************************************************** (ns / op) ********************************************************************************************************** B / op 4 allocs / op BenchmarkBigCacheSetParallel-8 (********************************************************************************************** ns / op B / op 3 allocs / op BenchmarkFreeCacheSetParallel-8 (************************************************************************************** (ns / op) ********************************************************************************************** B / op 3 allocs / op BenchmarkConcurrentMapSetParallel-8 (********************************************************************************************** ns / op
************************************************************************************ B / op 6 allocs / op BenchmarkBigCacheGetParallel-8 (********************************************************************************************************************. 1 ns / op B / op 4 allocs / op BenchmarkFreeCacheGetParallel-8 (************************************************************************************************ ns / op B / op 3 allocs / op BenchmarkConcurrentMapGetParallel-8 (********************************************************************************************** ns / op B / op 2 allocs / op PASS ok github.com/allegro/bigcache/v2/caches_bench ************************************************************************ (s) ********************
Writes and reads in bigcache are faster than in freecache. Writes to map are the slowest.
GC pause time
go version go version go1. linux / amd (************************************************************************************************************************** go run caches_gc_overhead_comparison.go Number of entries: GC pause
forbigcache: 1. ms GC pauseforfreecache: 5. 01575879 ms GC pauseformap: 9. (ms)
Test shows how long are the GC pauses for caches filled with 23 mln of entries. Bigcache and freecache have very similar GC pause time.
Memory usage
You may encounter system memory reporting what appears to be an exponential increase, however this is expected behavior. Go runtime allocates memory in chunks or ‘spans’ and will inform the OS when they are no longer required by changing their state to ‘idle’. The ‘spans’ will remain part of the process resource usage until the OS needs to repurpose the address. Further reading available
How it works
BigCache relies on optimization presented in 1.5 version of Go ( (issue -) **********************************************************************
). This optimization states that if map without pointers in keys and values is used then GC will omit its content. Therefore BigCache usesmap [uint64] uintkeys are hashed and values are offsets of entries.
Entries are kept in bytes array, to omit GC again. Bytes array size can grow to gigabytes without impact on performance because GC will only see single pointer to it.
********************** (my-unique-key) ********************** (byte) ****************** (********************** value
)entry, (_) ******************************:=cache. ************** (Get) ************** (
When cache load can be predicted in advance then it is better to use custom initialization because additional memory allocation can be avoided in that way.
)( number of shards (must be a power of 2)Shards:2016( time after which entry can be evictedLifeWindow:* time.Minute, ( Interval between removing expired entries (clean up).(**************** (If set to(**************** (Setting toCleanWindow: (5) ******************** (time.) ******************* Minute, ( rps * lifeWindow, used only in initial memory allocationMaxEntriesInWindow:127511*; err==nil{ fmt.Println(string(entry)) }*************************************************************************************************************************
max entry size in bytes, used only in initial memory allocation,
(MaxEntrySize:, ( prints information about additional memory allocationVerbose: true, ( cache will not allocate more memory than this limit, value in MB( if value is reached then the oldest entries can be overridden for the new ones
( 0 value means no size limitHardMaxCacheSize:050212, ( callback fired when the oldest entry is removed because of its expiration time or no space left( for the new entry, or because delete was called. A bitmask representing the reason will be returned.( Default value is nil which means no callback and it prevents from unwrapping the oldest entry.OnRemove:nil, ( OnRemoveWithReason is a callback fired when the oldest entry is removed because of its expiration time or no space left( for the new entry, or because delete was called. A constant representing the reason will be passed through.( Default value is nil which means no callback and it prevents from unwrapping the oldest entry.( Ignored if OnRemove is specified.OnRemoveWithReason:nil, }cache, (initErr**********************:=bigcache. ********************* NewBigCache(config)ifinitErr!=nillog.Fatal(initErr ) } cache.Set(********************** (my-unique-key) ********************** (byte) ****************** (********************** value
) if**************** (entry, (err) :=cache.Get(my-unique-key”
**************************** (LifeWindow) &**************** Three caches were compared: bigcache, (freecacheand map. Benchmark tests were made using an i7 – (K CPU @ 4.) ****************************************************************************************************************************************************** (GHz with) *********************************************************************************************************************************** (GB of RAM on Ubuntu) ************************************************************************************************************************************************************************************************************************************************************** (LTS) 5.2.****************************************************************************************************************************** – – – generic).
Benchmarks source code can be found here
Writes and reads
go version go version go1. linux / amd (************************************************************************************************************************** gotest-bench=. -benchmem -benchtime=4s. / … -timeout m goos: linux goarch: amd pkg: github.com/allegro/bigcache/v2/caches_bench BenchmarkMapSet-8 (****************************************************************************************** (ns / op) ********************************************************************************************************** B / op 3 allocs / op BenchmarkConcurrentMapSet-8 (************************************************************************************ (ns / op) ************************************************************************************************ B / op 8 allocs / op BenchmarkFreeCacheSet-8 (**************************************************************************************** (ns / op) ************************************************************************************************** (B / op 2 allocs / op BenchmarkBigCacheSet-8 (****************************************************************************************** (ns / op) **************************************************************************************************** B / op 2 allocs / op BenchmarkMapGet-8 34222654 (ns / op) B / op 1 allocs / op BenchmarkConcurrentMapGet-8 (****************************************************************************************** (ns / op) B / op 2 allocs / op BenchmarkFreeCacheGet-8 ****************************************************************** (ns / op) ************************************************************************************************************** B / op 2 allocs / op BenchmarkBigCacheGet-8 (****************************************************************************************** (ns / op) ********************************************************************************************************** B / op 4 allocs / op BenchmarkBigCacheSetParallel-8 (********************************************************************************************** ns / op B / op 3 allocs / op BenchmarkFreeCacheSetParallel-8 (************************************************************************************** (ns / op) ********************************************************************************************** B / op 3 allocs / op BenchmarkConcurrentMapSetParallel-8 (********************************************************************************************** ns / op
************************************************************************************ B / op 6 allocs / op BenchmarkBigCacheGetParallel-8 (********************************************************************************************************************. 1 ns / op B / op 4 allocs / op BenchmarkFreeCacheGetParallel-8 (************************************************************************************************ ns / op B / op 3 allocs / op BenchmarkConcurrentMapGetParallel-8 (********************************************************************************************** ns / op B / op 2 allocs / op PASS ok github.com/allegro/bigcache/v2/caches_bench ************************************************************************ (s) ********************
Writes and reads in bigcache are faster than in freecache. Writes to map are the slowest.
GC pause time
go version go version go1. linux / amd (************************************************************************************************************************** go run caches_gc_overhead_comparison.go Number of entries: GC pause
forbigcache: 1. ms GC pauseforfreecache: 5. 01575879 ms GC pauseformap: 9. (ms)
Test shows how long are the GC pauses for caches filled with 23 mln of entries. Bigcache and freecache have very similar GC pause time.
Memory usage
You may encounter system memory reporting what appears to be an exponential increase, however this is expected behavior. Go runtime allocates memory in chunks or ‘spans’ and will inform the OS when they are no longer required by changing their state to ‘idle’. The ‘spans’ will remain part of the process resource usage until the OS needs to repurpose the address. Further reading available
How it works
BigCache relies on optimization presented in 1.5 version of Go ( (issue -) **********************************************************************
). This optimization states that if map without pointers in keys and values is used then GC will omit its content. Therefore BigCache usesmap [uint64] uintkeys are hashed and values are offsets of entries.Entries are kept in bytes array, to omit GC again. Bytes array size can grow to gigabytes without impact on performance because GC will only see single pointer to it.
GIPHY App Key not set. Please check settings