Redis过期策略和内存淘汰策略

官方译文

Set a memory usage limit to the specified(指定) amount of bytes.

设置指定的内存字节数限制。

When the memory limit is reached Redis will try to remove keys
according to the eviction(收回、收回) policy(策略) selected (see maxmemory-policy).

一旦内存使用达到上限,Redis会根据选定的回收策略(参见:maxmemmory-policy)删除key

If Redis can’t remove keys according(根据,依据) to the policy, or if the policy is set to ‘noeviction’, Redis will start to reply with errors to commands that would use more memory, like SET, LPUSH, and so on, and(转折) will continue to reply to read-only(只读) commands like GET.

如果因为删除策略Redis无法删除key,或者策略设置为 “noeviction”,Redis会回复需要更多内存的错误信息给命令终端。例如,SET,LPUSH等等,但持续响应只读命令,例如get。

This option is usually useful when using Redis as an LRU or LFU cache, or to set a hard memory limit for an instance (using the ‘noeviction’ policy).

在使用Redis作为LRU缓存,或者为实例设置了硬性内存限制的时候(使用 “noeviction” 策略)的时候,这个选项通常事很有用的。

WARNING: If you have replicas attached to an instance with maxmemory on,
the size of the output buffers needed to feed(满足,向…提供) the replicas are subtracted(减去、去掉)
from the used memory count, so that network problems / resyncs will
not trigger a loop where keys are evicted, and in turn the output
buffer of replicas is full with DELs of keys evicted triggering the deletion
of more keys, and so forth until the database is completely emptied.

警告:如果有多个slave连上达到内存上限时,master为同步slave的输出缓冲区所需内存不计算在使用内存中。
这样当移除key时,就不会因网络问题 / 重新同步事件触发移除key的循环,反过来slaves的输出缓冲区充满了key被移除的DEL命令,
这将触发删除更多的key,直到这个数据库完全被清空为止。

In short(总而言之)… if you have replicas attached it is suggested that you set a lower
limit for maxmemory so that there is some free(空闲) RAM on the system for replica
output buffers (but this is not needed if the policy is ‘noeviction’).

总之,如果你需要添加多个slave,建议你设置一个最低内存限制,这样的话,就有空闲的内存为slave的输出缓存区(但是最大内存策略设置为”noeviction”的话就没必要了).

maxmemory
maxmemory 内存大小(字节数)
MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
is reached. You can select among(在…中) five behaviors:
最大内存策略:当内存大小达到最大限制时,redis实在下面五中行为中选择:
volatile-lru -> Evict using approximated LRU among the keys with an expire set.
根据LRU算法删除设置过期时间的key
allkeys-lru -> Evict any(任何的,一些) key using approximated LRU.
根据LRU算法删除任何key
volatile-lfu -> Evict using approximated LFU among the keys with an expire set.
根据LFU算法删除设置过期时间的key
allkeys-lfu -> Evict any key using approximated LFU.
根据LFU删除任何key
volatile-random -> Remove a random key among the ones with an expire set.
随机移除设置过过期时间的key
allkeys-random -> Remove a random key, any key.
随机移除任何key
volatile-ttl -> Remove the key with the nearest expire time (minor TTL)
移除即将过期的key(minor TTL)
noeviction -> Don’t evict anything, just return an error on write operations.
当内存不足以容纳新写入数据时,新写入操作会报错。

LRU means Least(最少,最小) Recently Used

LRU是指最少被使用的

LFU means Least Frequently(最频繁地,时常地) Used

LFU是指最不常被使用的

Both LRU, LFU and volatile-ttl are implemented using approximated
randomized algorithms(运算法则).

Note: with any of the above policies, Redis will return an error on write
operations, when there are no suitable keys for eviction.

记住:使用以上的回收策略,在没有合适的回收键,将在写的操作上返回一个错误信息。

At the date of writing these commands are: set setnx setex append
incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd
sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby
zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby
getset mset msetnx exec sort

这些写的命令有:set setnx setex append……(上面的这些Redis操作命令)
The default is:
maxmemory-policy noeviction
缓存粗略 策略名字

LRU, LFU and minimal TTL algorithms are not precise algorithms but approximated
algorithms (in order to save(节约、保存) memory), so you can tune it for speed or
accuracy. For default Redis will check five keys and pick the one that was
used less recently, you can change the sample size using the following
configuration directive.

LRU、LFU和最小TTL算法的实现都不是很精确,但是很接近(为了节约内存),所以你可以用样本量做检测。
默认Redis会检查5个key然后取最旧的那个,你可以通过下面的配置指令来设置样本的个数。

The default of 5 produces good enough results. 10 Approximates very closely
true LRU but costs more CPU. 3 is faster but not very accurate(精确,准确).

默认值为5,数字越大结果越精确但是会消耗更多CPU,3是最快的但不是最精确的。
maxmemory-samples 5

Starting from Redis 5, by default a replica will ignore its maxmemory setting
(unless(除非) it is promoted(促进、提升) to master after a failover(故障切换) or manually(手动地,手动地)). It means
that the eviction of keys will be just handled by the master, sending the
DEL commands to the replica as keys evict in the master side.

从Redis5版本开始,在默认的从节点上,将忽略最大内存限制(除非从节点在故障切换或者手动地切换为主节点)。
这将意味着删除命令通过在主节点实现,然后将命令发送给从节点。

This behavior ensures(确保) that masters and replicas stay consistent, and is usually
what you want, however if your replica is writable, or you want the replica to have
a different memory setting, and you are sure all the writes performed to the
replica are idempotent, then you may change this default (but be sure to understand
what you are doing).

此行为确保主节点和从节点保持一致,并且是您所希望的状态,但是,如果您的从节点是可写的,或者您希望从节点具有不同的内存设置,
并且您确定对从节点执行的所有写入都是等幂的,则可以更改此默认值(但请确保了解您正在执行的操作)

Note that since the replica by default does not evict, it may end using more
memory than the one set via maxmemory (there are certain buffers that may
be larger on the replica, or data structures may sometimes take more memory and so
forth). So make sure you monitor(监控) your replicas and make sure they have enough
memory to never hit a real out-of-memory condition before the master hits
the configured maxmemory setting.

注意,由于默认情况下从节点不会主动回收,因此它可能会使用更多内存,甚至大于通过maxmemory设置的内存(有些缓冲区可能
在从节点上更大,或者数据结构有时可能占用更多内存等等)。
因此,你要确保你的从节点监控器和有足够大的内存空间,确保在主节点在未达到最大内存限制前,从节点不会出现内存不足的情况。
replica-ignore-maxmemory yes
配置项 配置值(Yes/No)

过期删除策略

定时过期:

1.每个设置过期时间的key都需要创建一个定时器,到过期时间就会立即清除。

2.该策略可以立即清除过期的数据,对内存很友好;

3.但是会占用大量的CPU资源去处理过期的数据,从而影响缓存的响应时间和吞吐量。

惰性过期:

1.只有当访问一个key时,才会判断该key是否已过期,过期则清除。

2.该策略可以最大化地节省CPU资源,却对内存非常不友好。

3.可能出现大量的过期key没有再次被访问,从而不会被清除,占用大量内存。

定期过期:

1.每隔一定的时间,会扫描一定数量的数据库的expires字典中一定数量的key,并清除其中已过期的key。

2.该策略是前两者的一个折中方案。通过调整定时扫描的时间间隔和每次扫描的限定耗时,可以在不同情况下使得CPU和内存资源达到最优的平衡效果。

3.expires字典会保存所有设置了过期时间的key的过期时间数据,其中,key是指向键空间中的某个键的指针,value是该键的毫秒精度的UNIX时间戳表示的过期时间。键空间是指该Redis集群中保存的所有键。

如何配置过期策略

1.惰性过期策略为内置策略,无需配置

2.定期删除策略

1
2
配置redis.conf 的hz选项,默认为10 (即1秒执行10次,100ms一次,值越大说明刷新频率越快,最Redis性能损耗也越大
第二、配置redis.conf的maxmemory最大值,当已用内存超过maxmemory限定时,就会触发主动清理策略

内存淘汰策略

  • volatile-lru -> Evict using approximated LRU among the keys with an expire set.
    根据LRU算法删除设置过期时间的key

  • allkeys-lru -> Evict any(任何的,一些) key using approximated LRU.
    根据LRU算法删除任何key

  • volatile-lfu -> Evict using approximated LFU among the keys with an expire set.
    根据LFU算法删除设置过期时间的key

  • allkeys-lfu -> Evict any key using approximated LFU.
    根据LFU删除任何key

  • volatile-random -> Remove a random key among the ones with an expire set.
    随机移除设置过过期时间的key

  • allkeys-random -> Remove a random key, any key.
    随机移除任何key

  • volatile-ttl -> Remove the key with the nearest expire time (minor TTL)
    移除即将过期的key(minor TTL)

  • noeviction -> Don’t evict anything, just return an error on write operations.
    当内存不足以容纳新写入数据时,新写入操作会报错。

配置内存淘汰策略

1
maxmemory-policy noeviction

持久化对过期策略的影响

  • RDB持久化

1.持久化key之前,会检查是否过期,过期的key不进入RDB文件.

2.数据载入数据库之前,会对key先进行过期检查,如果过期,不导入数据库(主库情况)

  • AOF持久化

1.从内存数据库持久化数据到AOF文件:
当key过期后,还没有被删除,此时进行执行持久化操作(该key是不会进入aof文件的,因为没有发生修改命令)

当key过期后,在发生删除操作时,程序会向aof文件追加一条del命令(在将来的以aof文件恢复数据的时候该过期的键就会被删掉)

2.从AOF重写到内存
重写时,会先判断key是否过期,已过期的key不会重写到aof文件

主从复制对过期策略的影响

1.默认情况下,从节点是不做数据过期处理的,可以通过replica-ignore-maxmemory yes
决定开启是否在从节点处理过期处理策略。

2主从复制默认的情况下,都是在主节点实现,主节点将对应的del命令发送给从节点实现,从节点执行del命令。