Resiliency with Azure Caching

Using Azure caching can be easy when everything works though there have been many times when everything has not worked.  This post is applicable to dedicated caching (previously known as caching preview) and shared caching.  So how do you build in resiliency to your caching?

First, let’s assume data is being cached from a SQL Azure database, though what is outlined in this post is applicable to other data stores as well. First off let’s outline what could go wrong.

1) The cache becomes unavailable (could be because of a variety of reasons)

2) The database become unavailable

There are many reasons for number one to happen and the easiest way to handle it is to fall back to retrieving from the database.  You can have code that looks something like the below:

public RealyCoolCacheItem GetRealyCoolCacheItem(string key)
{
     try
     {
          var cacheItem = GetCacheItem(key);

          if (cacheItem == null)
          {
               var reallyCoolItem = RetrieveReallyCoolCacheItemFromDb(key);

          if (reallyCoolItem != null)
          {
               DataCache.Put(key, reallyCoolItem, TimeSpan.FromHours(2));
          }

               return reallyCoolItem;
          }
          else
          {
               return (RealyCoolCacheItem)cacheItem.Value;
          }
     }
     catch (Exception ex)
     {
          //Log the Error
     }

     //Would end up here if working with the cache throws
     return RetrieveReallyCoolCacheItemFromDb(key);
}

The second situation of a database going down is a bit more interesting.  The common direction is to have a fail over database though this is a big stick for a simple problem.  The cache is used to read frequently used data.

The solution Unified Logging has implemented is to serialized cache data to blob storage each time it is retrieved from the database overwriting the existing blob if necessary.  If the retrieval from the database fails it then reads the last blob written and things keep working.  This is how the submission endpoints of Unified Logging keep up and running when these undesirable events occur.

private RealyCoolCacheItem RetrieveReallyCoolCacheItemFromDb()
{
     try
     {
          //Retrieve from DB HERE
          var reallyCoolDbItem = magicDb.GetReallyCoolItem();

          //Save the item to the secondary failover datastore
          //In this case the secondary store is blob storage
          // RetrieveReallyCoolCacheItemFromDb_Failover is a constant
          failover.Save(RetrieveReallyCoolCacheItemFromDb_Failover, reallyCoolDbItem);

          return reallyCoolDbItem;
     }
     catch (Exception ex)
     {
           //Log the error

          //The retrieval from the db has failed so get the item form the secondary datastore
          return failover.Retrieve<RealyCoolCacheItem>(RetrieveReallyCoolCacheItemFromDb_Failover);
     }
}

At this point you are probably saying, all well and good but what is “failover”.  It is a simple class which implements a simple interface which has Save and Retrieve.

public interface ICacheFailover
{
      void Save<T>(string key, T serializableObject) where T : class;

      T Retrieve<T>(string key) where T : class;
}

Comments are closed.