We have a legacy application of about ten years old, which uses a database for caching. Each time a request came in, the database was queried to look for that request with the specific input parameters, and if found, the cached version of the data was taken. This worked fine, except when there was a lot of traffic, where it became a bottleneck.
To remove this bottleneck, we needed another way of caching; and so we decided to replace it with memcached. This article describes my first test with setting up and using memcached to see how it works.
Referring to the definition “Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.”, we can say that memcached is like a short-term memory for your data. You can store simple and more complex objects in it, which makes your web site respond a lot faster than when it would have to get that data from a database each time. You can specify the amount of memory it can use – which means that if the memory is full, older data gets purged in a least recently used order.
It’s important to understand that servers are disconnected from each other, which means that if you have more than one server, each server is unaware of the others, so they are not synchronized. As such, the effect of adding more servers is that you will simply get more capacity. This means the client must have the intelligence to decide what server to use (Hashing/Key Distribution), as each of your data will be stored only once on exactly one of the memcached servers.
First we need to install memcached on the server. On the project site there are no packages, only sources; but you can download a windows package from here. I took the memcached 1.4.5 win64.zip version, which contains 2 files after you unzip it:
So copy these files somewhere on the server. In my case I used my development computer itself as the server.
Now you have to options: run it as a console application or as a service. Running it as a console application is fine if you are doing R&D, but for real development you should install it as a service of course.
It seems that the latest version cannot be run as a service anymore, it should be wrapped in something else to make it run as a service. Anyway, for my test, I’m just going to run it by starting the memcached executable with some parameters. So run the elevated command prompt (run as administrator), go to the location where you copied memcached and start it as follows:
memcached -p 12345 -m 64 –vv [ENTER]
- p 12345 specifies the portnumber the memcached instance will listen to
- m 64 specifies the amount of RAM in Mb that memcached will use for storage
- vv controls verbosity (the amount of information that is logged)
The result will be a running memcached instance:
To test this, open a new command prompt, and type:
telnet 127.0.0.1 12345 [ENTER]
You will see an empty prompt:
set mymessage 1 0 11 [ENTER]
Hello world [ENTER]
With this command, we have added a new key mymessage, with 1 as flag (arbitrary metadata), with 0 as the expiry (never expires), and with value which is 11 bytes long. If it succeeds, STORED will be displayed:
To get it back, type:
get mymessage [ENTER]
And you will get back the value again:
This proves that the memcached instance is running and working properly.
In order to make use of memcached server, we need a .NET client API that allows us to easily access the memcached server’s functionality. It seems that the enyim client is one of the most popular ones with a rich feature set, so I decided to go with this one. Just go to the downloads section, get the latest one (at this time that’s Enyim.Caching.2.9.zip) and unzip it. You should have this:
Copy this to your references or libraries folder, and from your visual studio project add a reference to Enyim.Caching.dll:
The next thing to do is to add a global MemcacheClient instance to you application. From the documentation:
“Important! MemcachedClient is a “heavy object”, as in creating and initializing the client is quite expensive. (It has a socket pool inside, needs to calculate the tables for the consistent hashing, etc.) So, DO NOT create a client every time you want to perform an operation (well, you can, but 1) you need to Dispose() it after you finished, and 2) this will slow down your application considerably”
So best practice is to initialize the instance only once, for example in a web application you would typically do this in the Application_Start method of global.asax; but we’re using a simple console application, so:
static void Main(string args)
using (var client = new MemcachedClient())
// Store message
client.Store(StoreMode.Set, "mymessage", "Hello world");
// Get message back from cache
var message = client.Get<string>("mymessage");
Before we can use the client, we have to configure it, so extend your application’s config file with:
<add address="127.0.0.1" port="12345"/>
If you build the application, you may get two build errors:
When you have this, it means probably that the target framework of your project is wrong. To fix it, right click your project and select properties; and make sure .NET Framework 4 is selected:
Build again, debug it and check whether message contains the correct value:
This proved that memcached is quite easy to setup.
What about the key?
In the previous example the key was short and simple. But in reality it could happen that keys are big – too big. Memcached allows keys of a maximum size of 250 characters (for a reason). In the case your keys are too big, you are probably better of generating a hash (which is unique) and use that hash as the key. Here’s some code that does that for you:
/// Calculte the SHA1 hash of the given text.
/// <param name="text">The text to calculate the hash for.</param>
/// <returns>Hash of specified text.</returns>
private static string HashString(string text)
using (var sha1 = new SHA1CryptoServiceProvider())
var encoder = new UTF8Encoding();
hash = Convert.ToBase64String(sha1.ComputeHash(encoder.GetBytes(text)));