Appservers are typically stateless and state is typically stored in persistent data stores like RDBMS. To handle requests app server can retrieve state from persistent datastores(some times through complex queries) , process the state and then generate response which can be slow. If the generated response is stored  in memory data structure store like redis (so that next read can be handled via allready stored response), response processing and consequently response latency can be reduced.

How does server side caching help?

Server side caching using in memory data structure store like redis or appserver memory can make both reads and writes faster.

Reads can be faster because

Writes can be faster when the data is written to cache synchronously but written to primary data store asynchronously. Note that server side cache is a shared cache like CDN , loadbalancer, apigateway.

Client vs Server side caching

A client side cache is a browser cache built into the browser. Relatively static server responses need not be downloaded again and again from the server with the help of a client side cache.

Applications can use client side caching, server side caching or both.Use client side caching if you want to allow clients (typically browsers) to cache data locally on the users computer. Ie it is a private cache. The benefits are that the clients may not request your APIs until the cache expires. On the other hand you can't invalidate this cache because it is stored on the client side. Use this cache for non-dynamic/not frequently changing data. For instance an api which returns the recent transactions executed by a user is not cachable as the data changes frequently. A user may execute a transaction and come back to transactions page to view his recent transactions. If the browser is caching the recent transactions then the latest transaction may not be visible . A example of cacheable data can be recommendations on ecommerce page. Even if the user is seeing stale data for some time it might be ok based on the applications requirements. The same data can be cachable or not cachable in different contexts. For example, during online checkout, you need the authoritative price of an item, so caching might not be appropriate. On other pages, however, the price might be a few minutes out of date without a negative impact on users. Note that browser cache key is a combination of the request method and resource URI. URI consists of scheme, authority, path, query. Querstring parameters are basically inputs based on which resourse is fetched. Note that some browsers use URI as primary cache key and not method as only GET responses are cached.

Advantages of client side caching are

Note that client side caching can also refer to in memory cache held by appserver. Read more here

Data properties to be considered choosing caching strategy

When to load data to cache

Writing technques when using cache

Managing cache size

Caches can implement an expiration policy that invalidates data and removes it from the cache if it's not accessed for a specified period by the application. For caching to be effective, the expiration policy should match the pattern of access for app that uses the data . If the expiration period is too short then this can cause apps to repeatedly retrieve data from the data store and if the expiration period too long that the cached data can become stale or the cache size will increase depending on caching pattern used.

Lru caches are used to manage cache size as least recently used data is evicted from cache.

Common caching patterns

//get from cache
data = cache.get(key) 

if (data== null) {   //entry not found in cache

   // read from primary data store
   data = primary_data_store.get(key) 
   //update cache,even if data is null/{} this should be done so that cache key exists in redis with empty data
   cache.put(key, data)  


 Example code for writing data in cache aside

/*example code for writing data in cache aside*/

data= new data();
//data is put in cache
cache.put(key, data) 

/* synchronous write to primary data store , you can also write asynchnously which will improve write performance but data can be lost*/
primary_data_store.put(key, data) 

Read through / write through cache

How to choose caching key

Caching key can be a static string. If paginated data has to be cached key could contain page number and limits information. In spring framework @Cacheable attribute by default would create cache key using all method parameters.  Note that The default key generation strategy changed with the release of Spring 4.0. Earlier versions of Spring used a key generation strategy that, for multiple key parameters, only considered the hashCode() of parameters and not equals(); this could cause unexpected key collisions (see SPR-10237 for background). The new 'SimpleKeyGenerator' uses a compound key for such scenarios. Read more about spring caching here

API Gateway caching 

API Gateway's aside from providing core functionality of request routing and api compostion provide edge services like api caching. For instance amazon api gateway can be configured to cache rest end point responses for a specified time to live (TTL ) period. Note that while browsers can also cache api response, the response cached by API Gateway is server side caching once the api response is there in cache , it can be used by different client browsers.Ie API Gateway is a shared cache. Typically useful for api which are commonly used across users, eg offer of sale. By default most edge servers cache content based on entire resource path and query string. This behaviour will typically be modifiable (cache keys can be configured to include a querystring or portion of it) . Similar to browser cache invalidation can be an issue. The TTL has to be carefully chosen.

The thundering herd effect (aka dog piling)

In case of a cache-miss , multiple queries hit the database in parallel. This can happen under high load if a cache key expires because of TTL. Adding some randomness to TTL may ensure that the number of keys expiring within a time window is less.

Design challenges