Wow… it’s been a while. Work has been busy, so haven’t written anything and need to get my momentum back.
I have, however, been working on some really interesting back-end systems, and felt inspired to write something on caching.
What’s the problem?
Large, complex systems run a lot of complex, expensive calculations and reports on big sets of data. As a system’s data and user base grows, the effect of unnecessary operations will cause scalability problems. Consider the following system handling a request:
Every client connecting to this service goes through the same steps: connecting to the service, getting the data, generating the response. If there are only a few users and the query only takes a second, then there is no problem. But over time the duration of the GetData() call increases with the volume of data, and then we will find ourselves in the situation below:
What is caching?
Caching is the process of storing data at a more local level to avoid recalculating/re-fetching unnecessarily and to provide better system responsiveness and availability. This becomes more relevant the greater the number concurrent users you have in your system.
The most basic type of cache is a Read-through Cache, where data is stored on route from the data source and re-used for subsequent requests over a period of time. If we apply this to the previous scenario, you can see that we can achieve a 75% reduction in the number of calls to the database:
This benefit will be amplified by the many users, as the longer the data can be re-used, the lower the hit will be to the database.
Cache does not have to be read-only - you can also update the cache when you update the data source. This is called a Write-through Cache, where data is updated in memory en route to the data source.
The benefit of this is that it gives us the data persistence, combined with the performance boost of it being in memory. If the system crashes the data can be restored to the cache when it starts up, and if the database is unavailable the data can still be retained in the cache.
Consider the below:
The database is the primary bottleneck in most systems as it is a remote server running an IO-bound data repository. To get the maximum performance from your system, you need to minimise the load on the database. This approach goes a long way toward that goal, as it is only read from and updated once for four complete transactions.
Consider this as it scales to larger systems and you can immediately see the savings.
By maximising the number of scenarios like this in your system, you can scale much more linearly and not be as affected by higher load. If the goal is to minimise the load on the database while at the same time maintaining data integrity, this is a reliable approach.
It does, however, come at the cost of additional complexity. Subtle bugs caused by dirty data - data that is no longer up to date with what is in the data source - can cause odd and unexpected problems where the cause can be difficult to determine. Symptoms can include unknown IDs appearing in drop-down lists or being unable to action something on a server because it has been deleted. It’s important to keep these in mind when developing and diagnosing faults in systems that use caching.
This covers the most basic caching scenarios. Caching is by far the cheapest way to gain significant performance gains for minimal effort, and is something all developers should be aware of.