How does server side caching help?

Server side caching using in memory data structure store like redis or appserver memory can make both reads and writes faster.

Reads can be faster because

• Persitent data stores are much slower compared to memory stores.(This performance difference is magnified by high concurrency)
• Fetching Cached results of complex db queries and other compurations will lead to much lower latency. So allready stored data is preffered over data generation.
• If the orginal data source is far awary then cache can help reducing network latency.
  • This concept is often used in global deployments.
• In memory distributed caches are easier to scale up and scale out compared to persistent data storage like rdmbs where beoynd a point you cannot keep adding more servers to scale out(especially when there are joins). 
• In conclusion while sharding , pagination , proper indexing can improve db response speed but some times without server side caching getting optimal performance is difficult .

Writes can be faster when the data is written to cache synchronously but written to primary data store asynchronously. Note that server side cache is a shared cache like CDN , loadbalancer, apigateway.

Client vs Server side caching

A client side cache is a browser cache built into the browser. Relatively static server responses need not be downloaded again and again from the server with the help of a client side cache.

Applications can use client side caching, server side caching or both.Use client side caching if you want to allow clients (typically browsers) to cache data locally on the users computer. Ie it is a private cache. The benefits are that the clients may not request your APIs until the cache expires. On the other hand you can't invalidate this cache because it is stored on the client side. Use this cache for non-dynamic/not frequently changing data. For instance an api which returns the recent transactions executed by a user is not cachable as the data changes frequently. A user may execute a transaction and come back to transactions page to view his recent transactions. If the browser is caching the recent transactions then the latest transaction may not be visible . A example of cacheable data can be recommendations on ecommerce page. Even if the user is seeing stale data for some time it might be ok based on the applications requirements. The same data can be cachable or not cachable in different contexts. For example, during online checkout, you need the authoritative price of an item, so caching might not be appropriate. On other pages, however, the price might be a few minutes out of date without a negative impact on users. Note that browser cache key is a combination of the request method and resource URI. URI consists of scheme, authority, path, query. Querstring parameters are basically inputs based on which resourse is fetched. Note that some browsers use URI as primary cache key and not method as only GET responses are cached.

Advantages of client side caching are

• reduces the number of http requests the web browser makes to webserver
• reduces page load time
• reduces network cost as fewer hits are going to server
• reduces processing cost (on server side , serverless functions may charge based on cpu usage)
• reduces the load on the server side.

Note that client side caching can also refer to in memory cache held by appserver. Read more here https://redis.io/topics/client-side-caching.

Data properties to be considered choosing caching strategy

• Consider caching data that is read frequently .
• Data criticality
  • Memory cache should not be the authoratative store as if the cache crashes critical data will be lost.
  • Do not use write behind (a cache writing pattern)
• Data volume
  • Cache write pattern like write through will need larger caches.
  • Cache reading pattern like lazy loading requires smaller cache size.

When to load data to cache

• Lazy loading
  • The advantage of lazy loading pattern is that  the cache contains only data that the application actually requests, which helps in keeping the cache size cost-effective. On the other hand becuase of cache miss there can be latency.
• Cache warming
  • The application proactively loads data into cache which will ensure that the first visitor does not get a cache-miss. Ie the application is proactively preparing the cache for visitors. 
  • In a cache,traffic is not usually evenly spread across all the data. For instance the home page of an app is likely to be accessed more frequently. Cache warming should be only used for limited data sets , where read performance is critical , otherwise the cache size will become really large.
  • For high traffic sites cache warming is usually not required.

Writing technques when using cache

• Write through :
  • Data is written to cache and primary data store synchronously. The significance here is not the order in which it happens or whether it happens in parallel.  The significance is that completion is only confirmed after the data has been written to the cache and the primary data store.
  • Advantages of this appraoch 
    • The application ensures that the cache is kept synchronized with the database.
    • If the data criticality is high the data is allready saved in primary db hence even if cache fails there is no data loss. 
  • Disadvange : 
    • The write performance will be slower.
    • The cache will be loaded with data which the application has never requested.ie large more expensive cache..
• Write behind / write back
  • Modified cache entries are asynchronously written to the data source after a configured delay . Completion confirmation is sent before primary data store write. This helps in better write performance but should not be used when data criticality is high.  Resilience to data loss can be added in this pattern by duplicating writes to 2 caches. Write behind also reduces load on database as you can restrict the rate of updates via a queue.
• Write around
  • In this pattern, data is written only to the primary data store without writing to the cache. Completion response is sent as soon as the data is written to the primary data store . Advantage is that the cache will not be flooed with data that may not subsequently be re-read. The Disdvantage is that data in cache and primary data store will not be in sync. Only when data in cache expires ,will it be read from the primary data store.

Managing cache size

Caches can implement an expiration policy that invalidates data and removes it from the cache if it's not accessed for a specified period by the application. For caching to be effective, the expiration policy should match the pattern of access for app that uses the data . If the expiration period is too short then this can cause apps to repeatedly retrieve data from the data store and if the expiration period too long that the cached data can become stale or the cache size will increase depending on caching pattern used.

Lru caches are used to manage cache size as least recently used data is evicted from cache.

Common caching patterns

• Cache aside :
  • The app tallks both with the cache and the primary datastore
  • Cache doesn't interact with the database at all.
  • The cache is "kept aside" as a faster and more scalable in-memory data store. 
  • Advantages of cache aside
    • One advantage of cash aside is that the data model in cache can be different from the data model in primary data store. For example the response generated as a result of multiple queries can be cached. Cache can be introduced for api caching at any point of time.
    • This pattern is easier to code.
  • For reading data normally lazy read is used.ie  
    • Try to read from cache
    • If data not there , read form primary data store and then update cache.
    • Example code for reading data in cache aside

/*CACHE ASIDE PATTERN
LAZY LOAD  */
//get from cache
data = cache.get(key) 

if (data== null) {   //entry not found in cache

   // read from primary data store
   data = primary_data_store.get(key) 
  
   //update cache,even if data is null/{} this should be done so that cache key exists in redis with empty data
   cache.put(key, data)  

}

 Example code for writing data in cache aside

/*example code for writing data in cache aside*/

data= new data();
//data is put in cache
cache.put(key, data) 

/* synchronous write to primary data store , you can also write asynchnously which will improve write performance but data can be lost*/
primary_data_store.put(key, data) 

Read through / write through cache

• The cache acts as if it is the primary data store.
• The application only interacts with the cache . Primary data store reading and writing is delegated to cache.
• The calling application is not aware of database.
• Data is always retrieved from cache using key.
• The read and write patterns are
  • read through (lazy loading)
  • write through or write behind.
• Advantage
  • since calling app is not app is not aware of primary data store the code is cleaner
• Disadvantage
  • harder to implement , It requires writing a provider plugin for database fetch.  Cache aside is easier to implement.
  • Another disadvantage is , as compared to cash aside is that cache aside permits different data model in cache and primary datastore.

How to choose caching key

Caching key can be a static string. If paginated data has to be cached key could contain page number and limits information. In spring framework @Cacheable attribute by default would create cache key using all method parameters.  Note that The default key generation strategy changed with the release of Spring 4.0. Earlier versions of Spring used a key generation strategy that, for multiple key parameters, only considered the hashCode() of parameters and not equals(); this could cause unexpected key collisions (see SPR-10237 for background). The new 'SimpleKeyGenerator' uses a compound key for such scenarios. Read more about spring caching here https://www.baeldung.com/spring-cache-tutorial.

API Gateway caching 

API Gateway's aside from providing core functionality of request routing and api compostion provide edge services like api caching. For instance amazon api gateway can be configured to cache rest end point responses for a specified time to live (TTL ) period. Note that while browsers can also cache api response, the response cached by API Gateway is server side caching once the api response is there in cache , it can be used by different client browsers.Ie API Gateway is a shared cache. Typically useful for api which are commonly used across users, eg offer of sale. By default most edge servers cache content based on entire resource path and query string. This behaviour will typically be modifiable (cache keys can be configured to include a querystring or portion of it) . Similar to browser cache invalidation can be an issue. The TTL has to be carefully chosen.

The thundering herd effect (aka dog piling)

In case of a cache-miss , multiple queries hit the database in parallel. This can happen under high load if a cache key expires because of TTL. Adding some randomness to TTL may ensure that the number of keys expiring within a time window is less.

Design challenges

• Restart of redis can destory cache if persistence is not enabled. Let us say is a system is adding customer's transactions in an ecommerto portal to redis cache. It does not want restart of redis to result in incomplete transaction list to be sent to clients of data. When it a new transaction is created determination has to be done if only new transaction or all transaction list has to be sent to redis cache at the time of cache update. But sending absolute data can lead to race conditions if two threads try to make absolute updates. The most recent all transactions list may be overwritten as we do not have control over order of execution of threads.