Distributed Caching using Redis Server with .NET/C# Client

Introduction

In this article, I would like to describe my experience with installing and configuring Redis server in most compact way. Also, I would like to do a brief overview of usage Redis hashes and lists in .NET/C# client.

In this article:

• Install Redis server
• Protecting the server
• Configuring server replication
• Access cache from C# application
• Using ASP.NET session state with redis
• Sample usage of the Redis Sets, Lists and transactions
• Description of the attached sources

Redis Installation

Download binaries from https://github.com/dmajkic/redis/downloads (win32 win64 direct link) unpack archive to the application directory (e.g. C:\Program Files\Redis)

Download Redis service compiled from https://github.com/kcherenkov/redis-windows-service/downloads, then copy to the program folder (e.g. C:\Program Files\Redis. If config file is missing, download it and copy to the application directory as well. Example of the valid Redis config file is athttps://raw.github.com/antirez/redis/2.6/redis.conf.

When you have the full set of the application files (like it is shown on the image below),

navigate to the application directory and run the following command:

Where:

• %name% — name of service instance, example: redis-instance;
• %binpath% — path to this project EXE file, example: C:\Program Files\Redis\RedisService_1.1.exe;
• %configpath% — path to redis configuration file, example: C:\Program Files\Redis\redis.conf;

Example:

" \"C:\Program Files\Redis\redis.conf\""

It should look like this:

Make sure that you have enough privileges to start the service. After installation, check that the service was created successfully and is running now:

Alternatively, you can use installer, created by someone (I’ve not tried): https://github.com/rgl/redis/downloads.

Redis Server Protection: Password, IP Filtering

The primary way to protect Redis server is to set IP filtering using Windows firewall or properties of the active network connection. Additional protection can be set using redis password. It needs to update the Redisconfig file (redis.conf) in the following way:

First, find the line:

Remove the # symbol in the beginning and replace foobared with new password:

Then restart Redis Windows service!!!

When instantiating the client, use constructor with a password:

RedisClient client = new RedisClient(serverHost, port, redisPassword);

Redis Server Replication (master – slave configuration)

This technique allows creation copy of the server data into the synchronized copy, this means that each time when master is modified, slave server gets notification and is automatically synchronized. Mostly replication is used for read (but not write) scalability or data redundancy and for the server failover. Setup two instances of Redis (two services on the same or different servers), then configure one of them as slave. To make Redis server instance to be slave of another server, change the config file in this way:

Find the line below:

replace with:

(specify real IP of the master server, and port in case you customized it). If master is configured to require password (authentication), change redis.conf as it is shown below, find line:

remove the # symbol in the beginning and replace <master-password> with master password, to be like that:

Now this Redis instance can be used as a readonly synchronized copy of the master server.

Using Redis Cache from the C# Code

To use Redis in C# run the Manage NuGet packages addon, find ServiceStack.Redis pack, and install it.

Sample of using Set/Get methods directly from the instantiated client:

string elementKey = "testKeyRedis";

using (RedisClient redisClient = new RedisClient(host))
{
      if (redisClient.Get<string>(elementKey) == null)
      {
           // adding delay to see the difference
           Thread.Sleep(5000); 
           // save value in cache
           redisClient.Set(elementKey, "some cached value");
      }
      // get value from the cache by key
      message = "Item value is: " + redisClient.Get<string>("some cached value");
 }

Typed entity sets are more interesting and practical, because they operate with exact types of objects. In the code sample below, there are two classes defined Phone, and Person – owner of the phone. Each phone instance has a reference to the owner. This code demonstrates how we can add, remove or find items in the cache by criteria:

{
   public int Id { get; set; }
   public string Model { get; set; }
   public string Manufacturer { get; set; }
   public Person Owner { get; set; }
}

public class Person
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string Surname { get; set; }
    public int Age { get; set; }
    public string Profession { get; set; }
}

using (RedisClient redisClient = new RedisClient(host))
{
     IRedisTypedClient<phone> phones = redisClient.As<phone>();
     Phone phoneFive = phones.GetValue("5");
     if (phoneFive == null)
     {
          // make a small delay
          Thread.Sleep(5000);
          // creating a new Phone entry
          phoneFive = new Phone
          {
               Id = 5,
               Manufacturer = "Motorolla",
               Model = "xxxxx",
               Owner = new Person
               {
                    Id = 1,
                    Age = 90,
                    Name = "OldOne",
                    Profession = "sportsmen",
                    Surname = "OldManSurname"
               }
          };
          // adding Entry to the typed entity set
          phones.SetEntry(phoneFive.Id.ToString(), phoneFive);
     }
     message = "Phone model is " + phoneFive.Manufacturer;
     message += "Phone Owner Name is: " + phoneFive.Owner.Name;
}

In the example above, we instantiate the typed client IRedisTypedClient, which works with specific type of cached objects: Phone type.

ASP.NET Session State with Redis

To configure ASP.NET session state with redis provider, add a new file to your web project, namedRedisSessionStateProvider.cs, copy code from https://github.com/chadman/redis-service-provider/raw/master/RedisProvider/SessionProvider/RedisSessionProvider.cs, then add or change the following section in the configuration file (sessionState tag has to be inside system.web tag), or you can download attached sources and copy code.

customprovider="RedisSessionStateProvider" cookieless="false">
      <providers>
        <add name="RedisSessionStateProvider" writeexceptionstoeventlog="false" 
        type="RedisProvider.SessionProvider.CustomServiceProvider" 
        server="qablog.qaitdevlabs.com" port="6379" password="pasword">
      </add> </providers>
</sessionstate>

NOTE, that password is optional, based on the server authentication. It must be replaced with real value, or removed, if Redis server doesn’t require authentication. server attribute and port also have to be replaced according to concrete values (default port is 6379). Then in the project, you can use the session state:

public class MvcApplication1 : System.Web.HttpApplication
{
    protected void Application_Start()
    {
        //....
    }

    protected void Session_Start()
    {
        Session["testRedisSession"] = "Message from the redis ression";
    }
}

In the Home controller:

{
    public ActionResult Index()
    {
       //...
       ViewBag.Message = Session["testRedisSession"];
       return View();
    }
//...
}

Result:

ASP.NET output cache provider with redis can be configured in the similar way.

Redis Sets and Lists

The major note is that Redis lists implement IList<T> while Redis sets implement ICollection<T>. Let’s see how we can use them.

Lists are mostly used when it needs to separate different categories of objects of the same type. For example, we have “most selling phones” and “old collection” two lists of phones:

string host = “qablog.qaitdevlabs.com”;

using (var redisClient = new RedisClient(host))
{
    //Create a 'strongly-typed' API that makes all Redis Value operations to apply against Phones
    IRedisTypedClient<phone> redis = redisClient.As<phone>();

    IRedisList<phone> mostSelling = redis.Lists["urn:phones:mostselling"];
    IRedisList<phone> oldCollection = redis.Lists["urn:phones:oldcollection"];

    Person phonesOwner = new Person
        {
            Id = 7,
            Age = 90,
            Name = "OldOne",
            Profession = "sportsmen",
            Surname = "OldManSurname"
        };

    // adding new items to the list
    mostSelling.Add(new Phone
            {
                Id = 5,
                Manufacturer = "Sony",
                Model = "768564564566",
                Owner = phonesOwner
            });

    oldCollection.Add(new Phone
            {
                Id = 8,
                Manufacturer = "Motorolla",
                Model = "324557546754",
                Owner = phonesOwner
            });

    var upgradedPhone  = new Phone
    {
        Id = 3,
        Manufacturer = "LG",
        Model = "634563456",
        Owner = phonesOwner
    };

    mostSelling.Add(upgradedPhone);

    // remove item from the list
    oldCollection.Remove(upgradedPhone);

    // find objects in the cache
    IEnumerable<phone> LGPhones = mostSelling.Where(ph => ph.Manufacturer == "LG");

    // find specific
    Phone singleElement = mostSelling.FirstOrDefault(ph => ph.Id == 8);

    //reset sequence and delete all lists
    redis.SetSequence(0);
    redisClient.Remove("urn:phones:mostselling");
    redisClient.Remove("urn:phones:oldcollection");
}

Redis sets are useful when it needs to store associated sets of data and gather statistical information, for example answer -> queustion, votes for an answer or question. Let’s say that we have questions and answers, it needs to store them in the cache for better performance. Using Redis, we can do it this way:

/// <summary>

/// Gets or sets the Redis Manager. The built-in IoC used with ServiceStack autowires this property.
/// </summary>
IRedisClientsManager RedisManager { get; set; }
/// <summary>
/// Delete question by performing compensating actions to 
/// StoreQuestion() to keep the datastore in a consistent state
/// </summary>
/// <param name="questionId">
public void DeleteQuestion(long questionId)
{
    using (var redis = RedisManager.GetClient())
    {
        var redisQuestions = redis.As<question>();

        var question = redisQuestions.GetById(questionId);
        if (question == null) return;

        //decrement score in tags list
        question.Tags.ForEach(tag => redis.IncrementItemInSortedSet("urn:tags", tag, -1));

        //remove all related answers
        redisQuestions.DeleteRelatedEntities<answer>(questionId);

        //remove this question from user index
        redis.RemoveItemFromSet("urn:user>q:" + question.UserId, questionId.ToString());

        //remove tag => questions index for each tag
        question.Tags.ForEach("urn:tags>q:" + tag.ToLower(), questionId.ToString()));

        redisQuestions.DeleteById(questionId);
    }
}

public void StoreQuestion(Question question)
{
    using (var redis = RedisManager.GetClient())
    {
        var redisQuestions = redis.As<question>();

        if (question.Tags == null) question.Tags = new List<string>();
        if (question.Id == default(long))
        {
            question.Id = redisQuestions.GetNextSequence();
            question.CreatedDate = DateTime.UtcNow;

            //Increment the popularity for each new question tag
            question.Tags.ForEach(tag => redis.IncrementItemInSortedSet("urn:tags", tag, 1));
        }

        redisQuestions.Store(question);
        redisQuestions.AddToRecentsList(question);
        redis.AddItemToSet("urn:user>q:" + question.UserId, question.Id.ToString());

        //Usage of tags - Populate tag => questions index for each tag
        question.Tags.ForEach(tag => redis.AddItemToSet
        ("urn:tags>q:" + tag.ToLower(), question.Id.ToString()));
    }
}

/// <summary>
/// Delete Answer by performing compensating actions to 
/// StoreAnswer() to keep the datastore in a consistent state
/// </summary>
/// <param name="questionId">
/// <param name="answerId">
public void DeleteAnswer(long questionId, long answerId)
{
    using (var redis = RedisManager.GetClient())
    {
        var answer = redis.As<question>().GetRelatedEntities<answer>
        (questionId).FirstOrDefault(x => x.Id == answerId);
        if (answer == null) return;

        redis.As<question>().DeleteRelatedEntity<answer>(questionId, answerId);

        //remove user => answer index
        redis.RemoveItemFromSet("urn:user>a:" + answer.UserId, answerId.ToString());
    }
}

public void StoreAnswer(Answer answer)
{
    using (var redis = RedisManager.GetClient())
    {
        if (answer.Id == default(long))
        {
            answer.Id = redis.As<answer>().GetNextSequence();
            answer.CreatedDate = DateTime.UtcNow;
        }

        //Store as a 'Related Answer' to the parent Question
        redis.As<question>().StoreRelatedEntities(answer.QuestionId, answer);
        //Populate user => answer index
        redis.AddItemToSet("urn:user>a:" + answer.UserId, answer.Id.ToString());
    }
}

public List<answer> GetAnswersForQuestion(long questionId)
{
    using (var redis = RedisManager.GetClient())
    {
        return redis.As<question>().GetRelatedEntities<answer>(questionId);
    }
}

public void VoteQuestionUp(long userId, long questionId)
{
    //Populate Question => User and User => Question set indexes in a single transaction
    RedisManager.ExecTrans(trans =>
    {
        //Register upvote against question and remove any downvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:q>user+:" + questionId, userId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:q>user-:" + questionId, userId.ToString()));

        //Register upvote against user and remove any downvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:user>q+:" + userId, questionId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:user>q-:" + userId, questionId.ToString()));
    });
}

public void VoteQuestionDown(long userId, long questionId)
{
    //Populate Question => User and User => Question set indexes in a single transaction
    RedisManager.ExecTrans(trans =>
    {
        //Register downvote against question and remove any upvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:q>user-:" + questionId, userId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:q>user+:" + questionId, userId.ToString()));

        //Register downvote against user and remove any upvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet"urn:user>q-:" + userId, questionId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:user>q+:" + userId, questionId.ToString()));
    });
}

public void VoteAnswerUp(long userId, long answerId)
{
    //Populate Question => User and User => Question set indexes in a single transaction
    RedisManager.ExecTrans(trans =>
    {
        //Register upvote against answer and remove any downvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:a>user+:" + answerId, userId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:a>user-:" + answerId, userId.ToString()));

        //Register upvote against user and remove any downvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:user>a+:" + userId, answerId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:user>a-:" + userId, answerId.ToString()));
    });
}

public void VoteAnswerDown(long userId, long answerId)
{
    //Populate Question => User and User => Question set indexes in a single transaction
    RedisManager.ExecTrans(trans =>
    {
        //Register downvote against answer and remove any upvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:a>user-:" + answerId, userId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:a>user+:" + answerId, userId.ToString()));

        //Register downvote against user and remove any upvotes if any
        trans.QueueCommand(redis => 
        redis.AddItemToSet("urn:user>a-:" + userId, answerId.ToString()));
        trans.QueueCommand(redis => 
        redis.RemoveItemFromSet("urn:user>a+:" + userId, answerId.ToString()));
    });
}

public QuestionResult GetQuestion(long questionId)
{
    var question = RedisManager.ExecAs<question>
    (redisQuestions => redisQuestions.GetById(questionId));
    if (question == null) return null;

    var result = ToQuestionResults(new[] { question })[0];
    var answers = GetAnswersForQuestion(questionId);
    var uniqueUserIds = answers.ConvertAll(x => x.UserId).ToHashSet();
    var usersMap = GetUsersByIds(uniqueUserIds).ToDictionary(x => x.Id);

    result.Answers = answers.ConvertAll(answer =>
        new AnswerResult { Answer = answer, User = usersMap[answer.UserId] });

    return result;
}

public List<user> GetUsersByIds(IEnumerable<long> userIds)
{
    return RedisManager.ExecAs<user>(redisUsers => redisUsers.GetByIds(userIds)).ToList();
}

public QuestionStat GetQuestionStats(long questionId)
{
    using (var redis = RedisManager.GetReadOnlyClient())
    {
        var result = new QuestionStat
        {
            VotesUpCount = redis.GetSetCount("urn:q>user+:" +questionId),
            VotesDownCount = redis.GetSetCount("urn:q>user-:" + questionId)
        };
        result.VotesTotal = result.VotesUpCount - result.VotesDownCount;
        return result;
    }
}

public List<tag> GetTagsByPopularity(int skip, int take)
{
    using (var redis = RedisManager.GetReadOnlyClient())
    {
        var tagEntries = redis.GetRangeWithScoresFromSortedSetDesc("urn:tags", skip, take);
        var tags = tagEntries.ConvertAll(kvp => new Tag { Name = kvp.Key, Score = (int)kvp.Value });
        return tags;
    }
}

public SiteStats GetSiteStats()
{
    using (var redis = RedisManager.GetClient())
    {
        return new SiteStats
        {
            QuestionsCount = redis.As<question>().TypeIdsSet.Count,
            AnswersCount = redis.As<answer>().TypeIdsSet.Count,
            TopTags = GetTagsByPopularity(0, 10)
        };
    }
}

Conclusion. Optimization Caching in Application with Fast Local Cache

Since Redis doesn’t store data locally (no local replication), it might make sence to optimize performance by storing some light or user – dependent objects in the local cache (to skip serialization to string and client – server data transfering). For example, in the web application, it is better to use ‘System.Runtime.Caching.ObjectCache‘ for light objects, which are user dependent and used frequently by the application. Otherwise, when object has common usage, large size it must be saved in the distributed Rediscache. Example of the user dependent objects – profile information, personalization information. Common objects – localization data, information shared between different users, etc.