A foreword about memory

This is a document to advise the user to be cautious about using large threshold amounts with this project.

By default Scrap has a default threshold that allows any file under 5 megabytes to be loaded into the memory for 10 minutes. It is recommended that the user in the event that a limited amount of free memory is available that one of the solutions below is performed to ensure stability:

• Increase the amount of memory
• Enable "evict keys" in your redis configuration to dispose old values
• Decrease the threshold to support smaller files.

In the event that your system runs out of memory, and is entirely dependent on your system configuration, redis will:

• Slow down as your swapfile is consumed, and your disk will be used as a fallback
• Die via The OOM Killer

Reccomendations

System stability is crucial in any environment.
Our suggestion is to tune your kernel's swappiness to an amount that will safely use physical memory and fail over to swapping when enough is not available. (Without a swap file, your system may begin to kill processes in an attempt to free up memory.)

Please also configure your redis installation to evict keys that are outdated.

What about encrypted files?

Encrypted files are stored uniquely, it is very unlikely due to the process of HMAC that two hashes will match. With this being said if multiple encrypted files are accessed simultaneously, redis may become saturated with a large amount of unique files.

Currently, encrypted data when cached in memory remains encrypted and will consume more space than the actual filesize itself. The data stream will only be decrypted when the user requests it.

In a future commit, it is likely we will disable the cache for encrypted files.