According to https://memcached.org, Memcached is a free and open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load. Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

Memcached is very simple to setup, however you must be sure the service is secure and optimized for your workload. The last thing you need after installation is all your data to be exposed publicly (as is the default in some Operating Systems) or your instance to be used to attack other websites: see Memcrashed Major Amplification Attack.

In this article, I will walk you through installation, including security and optimization. We will then talk about how to monitor your Memcached instance.

Setup

Installation of the memcached service is super simple. I recommend you also install the client side library so we can monitor the service and see what it is doing

# Ubuntu
root@server:~# apt-get install memcached libmemcached-tools
# CentOS / RHEL
root@server:~# yum install memcached

Once installed, set the service to start on boot, and start the service on Red Hat/CentOS based systems. On Debian/Ubuntu based systems this step is done without your approval.

# Enable service start on boot
root@server:~# systemctl enable memcached
Synchronizing state of memcached.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable memcached
# Start Service
root@server:~# systemctl start memcached

You should now see the service listening on the default port 11211. Depending on the Operating System you are running, memcached may be listening on localhost, or on all interfaces. Having memcached open to the public is a severe security risk, so we will fix that in a moment

# Ubuntu
root@server:~# ss -pntl | grep 11211
LISTEN   0         128               127.0.0.1:11211            0.0.0.0:*        users:(("memcached",pid=1721,fd=26))
# CentOS
[root@web1 /]# ss -pntl | grep 11211
LISTEN     0      0            *:11211                    *:*                   users:(("memcached",pid=20486,fd=26))
LISTEN     0      0           :::11211                   :::*                   users:(("memcached",pid=20486,fd=27))

If you plan to use Memcached with PHP, you need to install the interface so PHP knows how to talk to Memcached. There are two packages available, php-memcache and php-memcached. You should install php-memcached, however at this time of writing either should work.

# Ubuntu
root@server:~# apt-get install php-memcached
# CentOS / RHEL
root@server:~# yum install php-memcached

Once you install the PHP module, you need to restart your PHP handler, typically either Apache or php-fpm

# Verify module is loaded
root@server:~# php -m | grep -i memcache
memcached

# Restart the PHP handler.  Pick the command that applies to your server
root@server:~# systemctl restart apache2    <--- mod_php on Ubuntu
root@server:~# systemctl restart httpd      <--- mod_php on CentOS / RHEL
root@server:~# systemctl restart php-fpm    <--- php-fpm

These steps should have your application up and running with Memcached. Now lets take a look at ensuring the service is secure, and then we will optimize it further.

Security

Depending on your OS, Memcached could be listening on its public interface. If you don’t have firewall rules in place to block traffic to the port, another server anywhere in the world can simply dump the contents of Memcached with a simple command

[root@server /]# memcached-tool 192.168.1.1:11211 dump

If your using Memcached for sessions, there will be data you do not want malicious user’s getting their hands on, such as email addresses, usernames, and anything else related to that session.

Not too long ago (as of this writing) there was an exploit where a remote server could send a small UDP packet to Memcached and request the response to go to another external server. The response of the request could be HUGE in comparison to the size of the request. Malicious server’s could essentially send a request to your server to send huge DDOS attack to a victim. This attack became known as the Memcrashed Major Amplification Attack.

Many service providers stepped up and blocked such network activity at edge nodes of data centers, however we still want your user’s information to be safe.

To keep your user’s information safe, we need Memcached to listen locally if you are on a single server environment, or on a private network if you have multiple servers talking to the one Memcached instance. You can change the interface Memcached is bound in the global configuration file.

# Ubuntu
root@server:~# grep "\-l" /etc/memcached.conf
-l 127.0.0.1
# CentOS / RHEL
[root@db1 /]# grep OPTIONS /etc/sysconfig/memcached
OPTIONS="-l 192.168.1.1"

You can see in the Ubuntu example we are setting Memcached to listen locally, while in the CentOS example we are telling Memcached to listen on a local protected network.

If you only have a public network network and multiple server’s that need to talk to the instance, you will need to use firewall rules to limit access to port 11211. Configuring the firewall is outside the scope of this article, but as a general rule of thumb, you need to default to blocking all traffic to port 11211 and whitelist trusted IP adresses to that port.

If you have a strict firewall currently in place, you may also need to permit networks should you be configuring Memcached to listen on an internal network.

Optimization

As far as optimizing Memcached, there are not many options we have available to fine tune. There are really only two you need to concern yourself with.

Ubuntu - /etc/memcached.conf

# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
# Note that the daemon will grow to this size, but does not start out holding this much
# memory
-m 64

# Limit the number of simultaneous incoming connections. The daemon default is 1024
# -c 1024 

CentOS - /etc/sysconfig/memcached

MAXCONN="1024"
CACHESIZE="256"

First we will talk about cachesize. The cachesize is the maximum amount of memory (in MB) that will be used for object storage. The default being 64 MB on Debian based systems and 256 MB on Red Hat based systems. If your running a small to medium sized website and your using Memcached for session storage, this may be more than enough to fit all your data. You can review the details of the Memcached instance by passing it the ‘stats’ command using either echo or the memcached-tool in CentOS based Operating Systems.

[root@server /]# echo stats | nc 192.168.1.1 11211 | grep " bytes "
STAT bytes 118006
[root@server /]# memcached-tool 192.168.1.1:11211 stats | grep " bytes "
                   bytes      118006

We can see in the above example we are using much less than a MB, so we have no need to increase the cachesize.

If you are using Memcached for page cache, you may need to increase the cachesize dramatically depending on how much cache you are storing. Let your website build up its cache for some time and check how many bytes you are using, if your close to the cachesize limit, increase the cachesize and restart Memcache, keeping in mind that will clear the contents currently in Memcached.

Another way to determine if your cachesize is low, is if you see a large number of evicted objects. The eviction value in the stats, is the number of times Memcached had to remove an object from the cache before it expired. There are other reasons that can cause an eviction, so were not going to go to far down the rabbit hole with this value.

[root@server /]# echo stats | nc 192.168.1.1 11211 | grep "eviction"
STAT evictions 0

This bring us to maxconn, or the maximum number of simultaneous connections to Memcached. The default is 1024, which should be high enough for most small to medium sized websites. If you start seeing errors about being unable to connect to Memcache during high traffic periods, this value may be your problem. If you are running a large website spanning multiple servers, you should set this number to be around the value as the maximum number of PHP processes that can run at the same time. So if your running php-fpm with a max_children value of 200 across 10 servers, you may need to increase maxconn to 2000 so all the PHP processes can connect during high traffic periods.

For most use cases, you can leave maxconn at its default 1024.

Monitoring

We talked briefly about one way to monitor your Memcached instance in the optimization section. You can pass commands directly to the instance with the echo command, or even connect to it via netcat or telnet. On CentOS based systems, the daemon installation comes with a nice tool called memcached-tool. The following are examples of getting the Memcached stats in different ways

# Connecting to the port via netcat and running commands
root@server:~# nc 127.0.0.1 11211
stats
STAT pid 1721
STAT uptime 8606
STAT time 1528922161
STAT version 1.5.6 Ubuntu
STAT libevent 2.1.8-stable
STAT pointer_size 64
STAT rusage_user 0.671516
STAT rusage_system 0.671516
STAT max_connections 1024
STAT curr_connections 1
STAT total_connections 4
STAT rejected_connections 0
STAT connection_structures 2
STAT reserved_fds 20
STAT cmd_get 0
STAT cmd_set 0
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 0
STAT get_misses 0
STAT get_expired 0
STAT get_flushed 0
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 19
STAT bytes_written 3793
STAT limit_maxbytes 67108864
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT time_in_listen_disabled_us 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 524288
STAT hash_is_expanding 0
STAT slab_reassign_rescues 0
STAT slab_reassign_chunk_rescues 0
STAT slab_reassign_evictions_nomem 0
STAT slab_reassign_inline_reclaim 0
STAT slab_reassign_busy_items 0
STAT slab_reassign_busy_deletes 0
STAT slab_reassign_running 0
STAT slabs_moved 0
STAT lru_crawler_running 0
STAT lru_crawler_starts 4335
STAT lru_maintainer_juggles 8654
STAT malloc_fails 0
STAT log_worker_dropped 0
STAT log_worker_written 0
STAT log_watcher_skipped 0
STAT log_watcher_sent 0
STAT bytes 0
STAT curr_items 0
STAT total_items 0
STAT slab_global_page_pool 0
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT evicted_active 0
STAT evictions 0
STAT reclaimed 0
STAT crawler_reclaimed 0
STAT crawler_items_checked 0
STAT lrutail_reflocked 0
STAT moves_to_cold 0
STAT moves_to_warm 0
STAT moves_within_lru 0
STAT direct_reclaims 0
STAT lru_bumps_dropped 0
END

# echo a command to netcat
root@server:~# echo stats | nc 127.0.0.1 11211
STAT pid 1721
STAT uptime 5890
STAT time 1528919445
... [output truncated] ...

# memcached-tool
[root@server /]# memcached-tool 127.0.0.1:11211 stats
STAT pid 1721
STAT uptime 5890
STAT time 1528919446
... [output truncated] ...

There are a number of commands you can pass to the instance. The following is the official memcached documentation on commands accepted: https://github.com/memcached/memcached/wiki/Commands

The following commands are available for the memcached-tool command

[root@db1 /]# memcached-tool 127.0.0.1:11211 help
Usage: memcached-tool <host[:port] | /path/to/socket> [mode]

       memcached-tool 10.0.0.5:11211 display    # shows slabs
       memcached-tool 10.0.0.5:11211            # same.  (default is display)
       memcached-tool 10.0.0.5:11211 stats      # shows general stats
       memcached-tool 10.0.0.5:11211 dump       # dumps keys and values

When reviewing the memcached statistics, there is a lot of information presented. The following chat explains the meaning of each value. Chart Source

Third Party Monitoring

There are a few third party monitoring tools available to you. I will list a few common tools to explore if you are interested

Conclusion

Memcached is a simple and effective in memory object storage service. Memcached can be configured on the application for session and/or page cache, and can also be configured for session storage in the default php.ini configuration file.

Memcached can also be configured for failover and automatic redundancy with multiple instances, however that is outside the scope of this article.