We only delete caches, we don't update them. Once the data in the DB is modified, we delete the corresponding cache directly instead of updating it.
Let's see how to delete the cache in the right order.
- Delete the cache first, then update the DB
Let's look at the case of two concurrent requests. A requests to update the data and deletes the cache first, then B requests to read the data, at this time the cache has no data, it will load the data from DB and write back to the cache, then A updates the DB, then at this time the data in the cache will remain dirty until the cache expires or there is a new request to update the data. As shown in the figure
Update the DB first, then delete the cache
A request to update the DB first, then B request to read the data, at this time the return is the old data, at this time can be considered as A request has not been updated, the final consistency, can be accepted, and then A deleted the cache, subsequent requests will get the latest data, as shown in the figure
Let's look at the normal request flow again.
- The first request updates the DB and deletes the cache
- The second request reads the cache and has no data, so it reads the data from the DB and writes it back to the cache
- Subsequent read requests can all read directly from the cache
Let's look at what happens with DB queries, assuming there are seven columns of data in the row record ABCDEFG.
A request that queries only part of the column data, such as a request for ABC, CDE or EFG among them, as in the figure
Query a single complete row of records, as shown ![redis-cache-07](. /../resource/redis-cache-07.png)
Query multiple rows of records with some or all columns, as in ![redis-cache-08](. /../resource/redis-cache-08.png)
For the above three cases, first, we do not use partial queries because partial queries cannot be cached, and once cached, there is no way to locate what data needs to be deleted once the data is updated; second, for multi-row queries, according to the actual scenario and needs, we will establish the corresponding mapping from query conditions to primary keys in the business layer; and for single-row complete record queries, go-zero has a built-in complete cache management approach. So the core principle is: go-zero must cache complete row records.
Let's detail the three built-in cache handling scenarios for go-zero.
- Primary key-based cachingPRIMARY KEY (``id``)
This is relatively the easiest cache to handle, just use the primary key as the key to cache row records in redis.
- Caching based on unique indexes
In the database design, if you look up data by index, the engine will first look up the primary key in the index->primary key tree and then look up the row records by the primary key, which introduces an indirect layer to solve the problem of index-to-row record correspondence. The same principle is used in go-zero's cache design.
Index-based caching is divided into single-column unique indexes and multi-column unique indexes.
But for go-zero, single-column and multi-column are just different ways to generate cache keys, the control logic behind them is the same. Then go-zero has built-in cache management to better control data consistency issues, and also built-in to prevent cache breakthroughs, penetrations, and avalanches (these were carefully discussed during the gopherchina conference, see the subsequent gopherchina sharing video).
In addition, go-zero has built-in statistics for cache accesses and access hits, as follows.
You can see more detailed statistics, so that we can analyze the cache usage. For cases where the cache hit rate is very low or the request volume is very small, we can remove the cache, which can also reduce the cost.
The single column unique index is as follows.UNIQUE KEY `product_idx` (`product`)
A multi-column unique index is as follows.UNIQUE KEY `vendor_product_idx` (`vendor`, `product`)
The concrete implementation code is as follows.
Take method here is to go through the
key to get the data from the cache first, if you get it, return it directly, if not, then go through the
query method to the
DB to read the full row and write it back to the cache, and then return the data. The whole logic is still relatively simple to understand.
Let's look at the implementation of
Take in detail.
The logic of
Take is as follows.
- Find the data from the cache using the key
- If found, return the data
- If not found, use the query method to read the data
- After reading, call c.SetCache(key, v) to set the cache
doTake code and explanation are as follows.
Because this block is more complex, I have marked the response blocks and logic with different colors.
block 2 is actually the same as the primary key-based cache, so here we focus on the logic of
The block 1 part of the code block is divided into two cases.
The primary key can be found from the cache through the index, so the logic of
block 2will be done directly with the primary key, and the logic of the cache based on the primary key will be followed as above.
The primary key cannot be found from the cache through the index.
- Search the complete row from DB by index, if there is
- When the complete row record is found, the cache of primary key to complete row record and index to primary key will be written to
redisat the same time.
- Return the required row data
- Search the complete row from DB by index, if there is
Let's look at a practical example
All of the above automatic cache management code is available via goctl, and our team's internal
CRUD and cache are basically generated via goctl, which saves a lot of development time, and the cache code itself is very error-prone, and even with good code experience, it's hard to get it right every time, so we recommend using automatic cache code generation tools to avoid errors whenever possible.