Skip to content

Commit

Permalink
Allow unbounded storage if maxSize or ttl set
Browse files Browse the repository at this point in the history 
This also prevents setting size=0 if maxSize is set, since that is a
recipe for disaster.

At least one of max, maxSize, or ttl MUST be set, to prevent unbounded
growth of the cache.  And really, without ttlAutopurge, it's effectively
unsafe and unbounded in that case anyway, *especially* if allowStale is
set.  This is potentially "unsafe at any speed" territory, so it emits a
process warning in that case.

If max is not set, then regular Array is used to track items, without
setting an initial Array capacity.  This will often perform much worse,
but in many cases, it's not so bad.  Bigger hazard is definitely
unbounded memory consumption.

Fix: #208
  • Loading branch information
@isaacs
isaacs committed Mar 17, 2022
1 parent c5707e3 commit c29079d
Show file tree
Hide file tree
Showing 9 changed files with 184 additions and 42 deletions.
90 changes: 88 additions & 2 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ const options = {
// the number of most recently used items to keep.
// note that we may store fewer items than this if maxSize is hit.

max: 500, // <-- mandatory, you must give a maximum capacity
max: 500, // <-- Technically optional, but see "Storage Bounds Safety" below

// if you wish to track item size, you must provide a maxSize
// note that we still will only keep up to max *actual items*,
Expand Down Expand Up @@ -112,7 +112,11 @@ If you put more stuff in it, then items will fall out.
may be stored if size calculation is used, and `maxSize` is exceeded.
This must be a positive finite intger.

This option is required, and must be a positive integer.
At least one of `max`, `maxSize`, or `TTL` is required. This must be a
positive integer if set.

**It is strongly recommended to set a `max` to prevent unbounded growth
of the cache.** See "Storage Bounds Safety" below.

* `maxSize` - Set to a positive integer to track the sizes of items added
to the cache, and automatically evict items in order to stay below this
Expand All @@ -121,6 +125,13 @@ If you put more stuff in it, then items will fall out.
Optional, must be a positive integer if provided. Required if other
size tracking features are used.

At least one of `max`, `maxSize`, or `TTL` is required. This must be a
positive integer if set.

Even if size tracking is enabled, **it is strongly recommended to set a
`max` to prevent unbounded growth of the cache.** See "Storage Bounds
Safety" below.

* `sizeCalculation` - Function used to calculate the size of stored
items. If you're storing strings or buffers, then you probably want to
do something like `n => n.length`. The item is passed as the first
Expand Down Expand Up @@ -193,6 +204,17 @@ If you put more stuff in it, then items will fall out.

This may be overridden by passing an options object to `cache.set()`.

At least one of `max`, `maxSize`, or `TTL` is required. This must be a
positive integer if set.

Even if ttl tracking is enabled, **it is strongly recommended to set a
`max` to prevent unbounded growth of the cache.** See "Storage Bounds
Safety" below.

If ttl tracking is enabled, and `max` and `maxSize` are not set, and
`ttlAutopurge` is not set, then a warning will be emitted cautioning
about the potential for unbounded memory consumption.

Deprecated alias: `maxAge`

* `noUpdateTTL` - Boolean flag to tell the cache to not update the TTL when
Expand Down Expand Up @@ -467,6 +489,70 @@ ignored.
* `tail` Internal ID of most recently used item
* `free` Stack of deleted internal IDs

## Storage Bounds Safety

This implementation aims to be as flexible as possible, within the limits
of safe memory consumption and optimal performance.

At initial object creation, storage is allocated for `max` items. If `max`
is set to zero, then some performance is lost, and item count is unbounded.
Either `maxSize` or `ttl` _must_ be set if `max` is not specified.

If `maxSize` is set, then this creates a safe limit on the maximum storage
consumed, but without the performance benefits of pre-allocation. When
`maxSize` is set, every item _must_ provide a size, either via the
`sizeCalculation` method provided to the constructor, or via a `size` or
`sizeCalculation` option provided to `cache.set()`. The size of every item
_must_ be a positive integer.

If neither `max` nor `maxSize` are set, then `ttl` tracking must be
enabled. Note that, even when tracking item `ttl`, items are _not_
preemptively deleted when they become stale, unless `ttlAutopurge` is
enabled. Instead, they are only purged the next time the key is requested.
Thus, if `ttlAutopurge`, `max`, and `maxSize` are all not set, then the
cache will potentially grow unbounded.

In this case, a warning is printed to standard error. Future versions may
require the use of `ttlAutopurge` if `max` and `maxSize` are not specified.

If you truly wish to use a cache that is bound _only_ by TTL expiration,
consider using a `Map` object, and calling `setTimeout` to delete entries
when they expire. It will perform much better than an LRU cache.

Here is an implementation you may use, under the same [license](./LICENSE)
as this package:

```js
// a storage-unbounded ttl cache that is not an lru-cache
const cache = {
data: new Map(),
timers: new Map(),
set: (k, v, ttl) => {
if (cache.timers.has(k)) {
clearTimeout(cache.timers.get(k))
}
cache.timers.set(k, setTimeout(() => cache.del(k), ttl))
cache.data.set(k, v)
},
get: k => cache.data.get(k),
has: k => cache.data.has(k),
delete: k => {
if (cache.timers.has(k)) {
clearTimeout(cache.timers.get(k))
}
cache.timers.delete(k)
return cache.data.delete(k)
},
clear: () => {
cache.data.clear()
for (const v of cache.timers.values()) {
clearTimeout(v)
}
cache.timers.clear()
}
}
```

## Performance

As of January 2022, version 7 of this library is one of the most performant
Expand Down
59 changes: 44 additions & 15 deletions index.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ const shouldWarn = code => typeof process === 'object' &&

const warn = (code, what, instead, fn) => {
warned.add(code)
process.emitWarning(`The ${what} is deprecated. Please use ${instead} instead.`, 'DeprecationWarning', code, fn)
const msg = `The ${what} is deprecated. Please use ${instead} instead.`
process.emitWarning(msg, 'DeprecationWarning', code, fn)
}

const isPosInt = n => n && n === Math.floor(n) && n > 0 && isFinite(n)
Expand Down Expand Up @@ -60,7 +61,7 @@ class ZeroArray extends Array {

class Stack {
constructor (max) {
const UintArray = getUintArray(max)
const UintArray = max ? getUintArray(max) : Array
this.heap = new UintArray(max)
this.length = 0
}
Expand All @@ -75,7 +76,7 @@ class Stack {
class LRUCache {
constructor (options = {}) {
const {
max,
max = 0,
ttl,
ttlResolution = 1,
ttlAutopurge,
Expand All @@ -85,7 +86,7 @@ class LRUCache {
disposeAfter,
noDisposeOnSet,
noUpdateTTL,
maxSize,
maxSize = 0,
sizeCalculation,
} = options

Expand All @@ -97,17 +98,17 @@ class LRUCache {
stale,
} = options instanceof LRUCache ? {} : options

if (!isPosInt(max)) {
throw new TypeError('max option must be an integer')
if (max !== 0 && !isPosInt(max)) {
throw new TypeError('max option must be a nonnegative integer')
}

const UintArray = getUintArray(max)
const UintArray = max ? getUintArray(max) : Array
if (!UintArray) {
throw new Error('invalid max value: ' + max)
}

this.max = max
this.maxSize = maxSize || 0
this.maxSize = maxSize
this.sizeCalculation = sizeCalculation || length
if (this.sizeCalculation) {
if (!this.maxSize) {
Expand Down Expand Up @@ -141,7 +142,7 @@ class LRUCache {
this.noDisposeOnSet = !!noDisposeOnSet
this.noUpdateTTL = !!noUpdateTTL

if (this.maxSize) {
if (this.maxSize !== 0) {
if (!isPosInt(this.maxSize)) {
throw new TypeError('maxSize must be a positive integer if specified')
}
Expand All @@ -161,6 +162,20 @@ class LRUCache {
this.initializeTTLTracking()
}

// do not allow completely unbounded caches
if (this.max === 0 && this.ttl === 0 && this.maxSize === 0) {
throw new TypeError('At least one of max, maxSize, or ttl is required')
}
if (!this.ttlAutopurge && !this.max && !this.maxSize) {
const code = 'LRU_CACHE_UNBOUNDED'
if (shouldWarn(code)) {
warned.add(code)
const msg = 'TTL caching without ttlAutopurge, max, or maxSize can ' +
'result in unbounded memory consumption.'
process.emitWarning(msg, 'UnboundedCacheWarning', code, LRUCache)
}
}

if (stale) {
deprecatedOption('stale', 'allowStale')
}
Expand Down Expand Up @@ -221,9 +236,17 @@ class LRUCache {
this.calculatedSize = 0
this.sizes = new ZeroArray(this.max)
this.removeItemSize = index => this.calculatedSize -= this.sizes[index]
this.addItemSize = (index, v, k, size, sizeCalculation) => {
const s = size || (sizeCalculation ? sizeCalculation(v, k) : 0)
this.sizes[index] = isPosInt(s) ? s : 0
this.requireSize = (k, v, size, sizeCalculation) => {
if (sizeCalculation && !size) {
size = sizeCalculation(v, k)
}
if (!isPosInt(size)) {
throw new TypeError('size must be positive integer')
}
return size
}
this.addItemSize = (index, v, k, size) => {
this.sizes[index] = size
const maxSize = this.maxSize - this.sizes[index]
while (this.calculatedSize > maxSize) {
this.evict()
Expand All @@ -241,7 +264,12 @@ class LRUCache {
}
}
removeItemSize (index) {}
addItemSize (index, v, k, size, sizeCalculation) {}
addItemSize (index, v, k, size) {}
requireSize (k, v, size, sizeCalculation) {
if (size || sizeCalculation) {
throw new TypeError('cannot set size without setting maxSize on cache')
}
}

*indexes ({ allowStale = this.allowStale } = {}) {
if (this.size) {
Expand Down Expand Up @@ -389,6 +417,7 @@ class LRUCache {
sizeCalculation = this.sizeCalculation,
noUpdateTTL = this.noUpdateTTL,
} = {}) {
size = this.requireSize(k, v, size, sizeCalculation)
let index = this.size === 0 ? undefined : this.keyMap.get(k)
if (index === undefined) {
// addition
Expand All @@ -400,7 +429,7 @@ class LRUCache {
this.prev[index] = this.tail
this.tail = index
this.size ++
this.addItemSize(index, v, k, size, sizeCalculation)
this.addItemSize(index, v, k, size)
noUpdateTTL = false
} else {
// update
Expand All @@ -414,7 +443,7 @@ class LRUCache {
}
this.removeItemSize(index)
this.valList[index] = v
this.addItemSize(index, v, k, size, sizeCalculation)
this.addItemSize(index, v, k, size)
}
this.moveToTail(index)
}
Expand Down
6 changes: 6 additions & 0 deletions tap-snapshots/test/deprecations.js.test.cjs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,5 +49,11 @@ Array [
"LRU_CACHE_METHOD_del",
Function get del(),
],
Array [
"TTL caching without ttlAutopurge, max, or maxSize can result in unbounded memory consumption.",
"UnboundedCacheWarning",
"LRU_CACHE_UNBOUNDED",
Function LRUCache(classLRUCache),
],
]
`
Loading

0 comments on commit c29079d

Please sign in to comment.