Redis is a high-performance key-value database that is widely used in projects, and every developer should master this skill. In today’s article, we’ll introduce Redis persistence in detail.

As we all know, Redis is memory-based and reads and writes data from and to memory, but the data does not disappear when we restart the service, which is ensured by Redis persistence.

Redis provides two persistence methods, RDB and AOF, which can be used separately or simultaneously, or in some cases, neither.

I. RDB

The RDB persistence method is to save the state at a certain moment to disk in the form of a file by generating an in-memory snapshot, which is called an RDB file.

When we restart the service, we can directly read this file to recover the data, this persistence method can recover the data quickly, but when the service is down, all the data written after the last snapshot file is created will be lost, and the data security is poor.

The Redis service generates snapshot files in the following cases

  • The client executes the save or bgsave command.
  • When shutting down the service with the shutdown command, the save logic is executed first.
  • When the conditions set by the save configuration item in the redis configuration file are met.

1-1 Save and bgsave commands

The save command creates a snapshot file through the main thread, and the main thread cannot process other client requests until the processing is completed, so this command should be used with caution.

The bgsave command creates the snapshot file through the fork’s sub-thread, and the main thread can continue to process other clients’ requests.

1-2 Save configuration item

RDB file generation rules can also be configured using the save configuration item, which in redis 7.0 version is described as follows:

1
2
save <seconds> <changes> [<seconds> <changes> ...]
save 3600 1 300 100 60 10000

The meaning of the above configuration is

  • Generate a snapshot after 3600 seconds with one change
  • Generate snapshot after 300 seconds with 100 changes
  • Generate snapshot after 60 seconds with 10,000 changes

This configuration needs to be configured according to the amount of lost data allowed by the business system.

  • save Conditions for generating RDB files
  • stop-writes-on-bgsave-error
  • rdbcompression whether to compress string type data when executing RDB snapshot
  • rdbchecksum Whether to enable checksum of RDB file content
  • dbfilename File name Default is dump.rdb
  • rdb-del-sync-files
  • dir file path
  • rdb-save-incremental-fsync

II. AOF

RDB persistence method may lose all the data written since the last snapshot file was created, if we have high security requirements for the system, we can use AOF such persistence.

AOF persistence is achieved by means of an append command. After Redis executes the received write command, it records this command as text in the AOF file. When you need to restore data, you execute all the commands contained in the AOF file from start to finish. This persistence method is turned off by default in Redis, and can be turned on with the appendonly configuration item.

2-1 Write back strategy

Disk operations are a relatively time-consuming operation, and updating the disk after each execution of a command will definitely affect Redis performance. In Redis, the log is first written to the aof_buf buffer, and then a specific write back strategy is used according to the appendfsync configuration. Three write back strategies are provided in Redis, as follows:

  • always Synchronous write back: Write the log to disk immediately after executing a write command. High reliability, basically no data loss; higher performance impact.
  • everysec write-back every second: writes the contents of the buffer to disk every second. Moderate performance, up to 1s data loss in case of downtime.
  • no OS-controlled write back: The OS decides when to write the buffer contents back to disk. Good performance, more data is lost during downtime.

The specific choice of which write back policy needs to be decided according to the performance and reliability requirements of our system. If you want to get as high performance as possible, choose no; if you want to get as much reliability as possible, choose always; if you want to allow a little data loss, but also want the performance not to be affected too much, choose everysec.

2-2 AOF rewriting

Over time, an AOF file gets larger and larger, and performance on both reading and writing to that file deteriorates. To solve this problem, an AOF rewrite mechanism is provided in Redis.

Performing a Redis rewrite involves regenerating an AOF file based on the current state of the Redis database, that is, reading all of the library’s key-value pairs and then generating a command for each key-value pair to be logged. Why does this make the file smaller? For example, if I call a set operation 10 times on a key, the original AOF file will record 10 commands, while the rewritten file will only record the last set log.

The conditions are specified in the Redis configuration file and are as follows.

1
2
3
auto-aof-rewrite-percentage 100

auto-aof-rewrite-min-size 64mb

3-3 Hybrid Persistence

RDB and AOF have their own advantages and disadvantages. Is there any way to balance the fast data recovery of RDB files with the reliability of AOF data? Hybrid persistence is available in Redis version 4.0.

Hybrid persistence means that when the AOF file is rewritten, the current state of the Redis database is written to the AOF file in RDB format, and the data is restored when Redis is restarted in both RDB and AOF parsing.

  • appendonly Whether to enable AOF function Default is no
  • appendfilename AOF file name Default is appendonly.aof
  • appendfsync write to disk policy default is everysec
  • no-appendfsync-on-rewrite Default is no
  • auto-aof-rewrite-percentage File size growth rate to be met for AOF rewrites Default is 100
  • auto-aof-rewrite-min-size The minimum file size required for AOF rewrites is 64MB by default.
  • aof-load-truncated Whether to write back to disk when performing RDB generation or AOF rewrites Default is no
  • aof-use-rdb-preamble Whether to enable hybrid persistence Default yes

III.Summary

In today’s article, we introduce the persistence methods in Redis, which are all theoretical and do not require rote memorization. I hope you can understand a few things

What persistence methods Redis provides.
The differences between RDB and AOF persistence and the advantages and disadvantages of each.
The write back strategy, rewrite and hybrid persistence of AOF logs.