Redis 7.x 系列【16】持久化机制之 AOF

有道无术，术尚可求，有术无道，止于术。

本系列Redis 版本 7.2.5

源码地址：https://gitee.com/pearl-organization/study-redis-demo

1. 概述

默认情况下，Redis 使用RDB持久化，但如果进程出现问题或电源中断，可能会导致几分钟的写入数据丢失（具体取决于配置的保存点）。

Append Only File（AOF）是另一种持久化模式，它提供了更好的持久性保证。AOF以日志的形式来记录每个写操作，将Redis执行过的所有写指令记录下来（读操作不记录），只许追加文件但不可以改写文件，Redis启动之初会读取该文件重新构建数据。换言之，重启的话就根据日志文件的内容，将写指令从前到后执行一次以完成数据的恢复工作。

执行流程：

客户端的请求写命令会被追加（append）到AOF缓冲区内。
AOF缓冲区根据AOF持久化策略（always, everysec, no）将操作同步（sync）到磁盘的AOF文件中。
- always：始终同步，每次Redis的写入都会立刻记入日志，性能较差但数据完整性比较好。
- everysec：每秒同步，每秒记入日志一次，如果宕机，本秒的数据可能丢失。
- no：自己不进行更新日志的行为，而是等操作系统自动调度的时候，再去刷磁盘。
AOF文件大小超过重写策略或手动重写时，会对AOF文件进行重写（rewrite），压缩AOF文件容量。
Redis服务重启时，会重新加载（load）AOF文件中的写操作达到数据恢复的目的。

优点：

备份机制更稳健，丢失数据概率更低。
可读的日志文本，通过操作AOF文件，可以处理误操作。

缺点：

比起RDB占用更多的磁盘空间。
恢复备份速度要慢。
每次读写都同步的话，有一定的性能压力。
存在个别Bug，造成不能恢复。

2. 配置项

redis.conf 中 AOF 相关的配置如下：

############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check https://redis.io/topics/persistence for more information.

appendonly no

# The base name of the append only file.
#
# Redis 7 and newer use a set of append-only files to persist the dataset
# and changes applied to it. There are two basic types of files in use:
#
# - Base files, which are a snapshot representing the complete state of the
#   dataset at the time the file was created. Base files can be either in
#   the form of RDB (binary serialized) or AOF (textual commands).
# - Incremental files, which contain additional commands that were applied
#   to the dataset following the previous file.
#
# In addition, manifest files are used to track the files and the order in
# which they were created and should be applied.
#
# Append-only file names are created by Redis following a specific pattern.
# The file name's prefix is based on the 'appendfilename' configuration
# parameter, followed by additional information about the sequence and type.
#
# For example, if appendfilename is set to appendonly.aof, the following file
# names could be derived:
#
# - appendonly.aof.1.base.rdb as a base file.
# - appendonly.aof.1.incr.aof, appendonly.aof.2.incr.aof as incremental files.
# - appendonly.aof.manifest as a manifest file.

appendfilename "appendonly.aof"

# For convenience, Redis stores all persistent append-only files in a dedicated
# directory. The name of the directory is determined by the appenddirname
# configuration parameter.

appenddirname "appendonlydir"

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".

# appendfsync always
appendfsync everysec
# appendfsync no

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync no". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.

no-appendfsync-on-rewrite no

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes

# Redis can create append-only base files in either RDB or AOF formats. Using
# the RDB format is always faster and more efficient, and disabling it is only
# supported for backward compatibility purposes.
aof-use-rdb-preamble yes

# Redis supports recording timestamp annotations in the AOF to support restoring
# the data from a specific point-in-time. However, using this capability changes
# the AOF format in a way that may not be compatible with existing AOF parsers.
aof-timestamp-enabled no

2.1 appendonly

appendonly 用于控制是否启用AOF持久化。

appendonly no

被设置为 no（默认）时，Redis 不会使用 AOF 持久化机制。

2.2 appendfilename

appendfilename 用于配置 AOF 文件的基础名称。

appendfilename "appendonly.aof"

Redis 7 及更高版本使用一组仅追加文件进行AOF，文件主要有两种类型：

基础文件：是数据集在文件创建时完整状态的快照。基础文件可以是 RDB（二进制序列化）格式或 AOF（文本命令）格式。
增量文件：包含在上一个文件之后对数据集应用的其他命令。

此外，还使用清单文件来跟踪文件的创建顺序，以及它们应该被应用的顺序。

Redis 根据特定的模式创建AOF文件的名称。文件名的前缀基于appendfilename配置参数，后面跟着关于序列和类型的额外信息。

例如，如果 appendfilename被设置为appendonly.aof，那么可能会产生以下文件名：

appendonly.aof.1.base.rdb 作为基础文件。
appendonly.aof.1.incr.aof, appendonly.aof.2.incr.aof 作为增量文件。
appendonly.aof.manifest 作为清单文件。

2.3 appenddirname

appenddirname 配置AOF文件的存储目录。

appenddirname "appendonlydir"

2.4 appendfsync

appendfsync 用于配置AOF缓冲区将操作同步（sync）到磁盘的AOF文件中的策略。

# appendfsync always
appendfsync everysec
# appendfsync no

fsync() 是一个系统调用，它告诉操作系统将数据实际写入磁盘，而不是等待更多的数据进入输出缓冲区。不同的操作系统对 fsync() 的实现可能有所不同，一些系统会立即将数据写入磁盘，而另一些系统则会尽快尝试这样做。

Redis 支持三种不同的 fsync 模式：

no：不进行 fsync 调用，让操作系统自行决定何时将数据刷新到磁盘。这种方式速度最快，但可能存在数据丢失的风险。
always：每次写入 AOF 日志后都进行 fsync 调用。这是最慢但最安全的方式，因为每次写入都会立即被刷新到磁盘。
everysec：每秒只进行一次 fsync 调用。这是速度和数据安全性之间的一个折中方案。

默认值是 everysec，因为它通常是在速度和数据安全性之间最好的折中方案。你可以根据自己的需求调整这个设置。如果你能接受一些数据丢失的风险并且追求更好的性能，可以考虑使用 no 模式（但请注意，如果你能接受数据丢失，那么默认的基于快照的持久化模式可能更合适）。相反，如果你追求更高的数据安全性，即使这意味着性能会降低，可以使用 always 模式。

2.5 no-appendfsync-on-rewrite

no-appendfsync-on-rewrite 用于配置在BGSAVE或BGREWRITEAOF正在进行时，是否阻止主进程调用fsync()。

no-appendfsync-on-rewrite no

当AOF的fsync策略设置为always或everysec时，如果有一个后台保存进程（后台保存或AOF日志后台重写）正在对磁盘进行大量I/O操作，在某些Linux配置下，Redis可能会在fsync()调用上阻塞过长时间。目前这个问题没有修复方法，因为即使在不同的线程中执行fsync也会阻塞我们的同步write调用。

默认为no，不阻止主进程fsync()，还是会把数据往磁盘里刷，但是遇到重写操作，可能会发生阻塞，数据安全，但是性能降低。

当设置为yes时，在BGSAVE或BGREWRITEAOF正在进行时，主进程将不会执行fsync调用。这意味着在这段时间内，Redis的持久性与appendfsync no设置时相同，即数据不会立即同步到磁盘。在最坏的情况下，你可能会丢失最多30秒（使用默认Linux设置）的AOF日志数据。因此，如果你对延迟很敏感，可以将其设置为yes。但是，从持久性的角度来看，将其保持为no（默认值）是最安全的选择。

2.6 auto-aof-rewrite-percentage、auto-aof-rewrite-min-size

AOF采用文件追加方式，文件会越来越大为避免出现此种情况，新增了重写机制，当AOF文件的大小超过所设定的阈值时，Redis会fork出一条新进程来将文件重写，也是先写临时文件最后再重命名。

重写过程会创建一个新的 AOF 文件，其中只包含能够恢复当前数据状态所必需的最少命令，从而减小 AOF 文件的大小。

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

auto-aof-rewrite-percentage 用于设置一个百分比阈值，当 AOF 文件的大小相对于上一次重写后的大小增长了超过这个百分比时，Redis 就会自动触发 BGREWRITEAOF 命令来重写 AOF 文件。设置为 100（即 100%），那么当 AOF 文件的大小增长了一倍时触发重写。

auto-aof-rewrite-min-size 用于设置一个 AOF 文件重写所需的最小大小（以字节为单位）。只有当 AOF 文件的大小既超过了指定的百分比（ auto-aof-rewrite-percentage ），又超过了 auto-aof-rewrite-min-size 指定的最小大小时，Redis 才会触发 AOF 文件的重写。

2.7 aof-load-truncated

aof-load-truncated 用于配置如果 AOF 文件末尾被截断时的相关处理策略。

aof-load-truncated yes

在 Redis 启动过程中，当 AOF 数据被加载回内存时，可能会发现 AOF 文件末尾被截断。例如，在运行 Redis 的操作系统崩溃时，尤其是当 ext4 文件系统被挂载但没有使用 data=ordered 选项。然而，当 Redis 本身崩溃或中止但操作系统仍然正常工作时，这种情况不会发生。

当发生这种情况时，Redis 可以选择报错退出，或者加载尽可能多的数据（现在是默认行为）并启动，如果 AOF 文件末尾被截断。以下选项控制这种行为。

aof-load-truncated yes：会加载一个被截断的 AOF 文件，并且 Redis 服务器会开始记录日志以通知用户该事件。
aof-load-truncated no：服务器会报错并拒绝启动。当选项设置为 no 时，用户需要使用 “redis-check-aof” 工具修复 AOF 文件后再重新启动服务器。