Lewis Lebentz

How to protect against an XMLRPC attack

Over the last week I started to notice some alerts coming in from a website uptime monitoring service I use called Pingdom. At first it seemed normal, over the last few months the uptime of my site had been around 99.99% which is very good, so a few instances of downtime weren't unexpected. I thought my VPS host DigitalOcean may have been having some issues, or some Ubuntu updates may have caused a temporary spike in RAM.

However, the downtime alerts continued, sometimes for just a minute at a time, but occasionally for a few hours! The alerts eventually became annoying, so I decided to investigate.

I checked the logs for Apache, MySQL and the auth logs at the locations below:

I could see many login attempts from IP addresses coming from all over the world. This is normal, as my site is publicly accessible, but I installed a daemon called fail2ban to add some extra protection. Fail2ban scans your logs for authentication errors and bans the IP for a certain amount of time, this helps prevent brute force attacks.

The downtime continued despite installing fail2ban, though I could see it was now successfully blocking some attacks, mainly from Hong Kong.

I received the next alert whilst I was on my Mac, so I quickly SSH'd into my server and everything appeared to be normal I checked how much RAM was in use using this command:

free -m

I only have a low spec server, but the memory usage was almost maxed out. I used the below command to check which processes were running:

top

I could see some apache2 processes with a mysqld process seemed to be using up all the resources.

I raised a support ticket with DigitalOcean to ask if it looked like I was the target of a DDoS attack according to their logs, I received a response after two minutes and after some further investigation we discovered this is the apache2 log files:

46.166.139.20 - - [06/Dec/2015:17:59:01 -0500] "POST /xmlrpc.php HTTP/1.0" 200 560 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"

This was repeated constantly, multiple times per second until it killed my server.

I did some research on the xmlrpc.php file and found a blog post by Sucuri detailing the discovery of a new WordPress attack. The hacker creates a script to send POST requests to that file, attempting to brute force a login. Each HTTP request can contain hundreds of credentials! Usually brute force attacks can be blocked by rate limiting, but this method allows you attempt thousands of login attempts in just a few seconds.

So, someone from the Netherlands was attempting to brute force my WordPress account, but DoS'ing my server too due to the sheer amount of requests coming in.

How can I fix it?

Well, the easiest way and the most effective method for me has been to modify the .htaccess file to block requests to xmlrpc.php this instantly stopped my server going down, as Apache no longer has to process the requests.

All you need to do is open up the root directory of your web server and type in:

nano .htaccess

and add this code to the bottom of the file:

# Block access to xmlrpc.php
<Files xmlrpc.php>
 order allow,deny
 deny from all
 ErrorDocument 403 127.0.0.1
</Files>

This will redirect the attacker to start hitting himself with all the requests and will take the load off of your server!

I would like to thank DigitalOcean for the fast support, and would also recommend setting up Pingdom and New Relic to monitor uptime and performance (both have free plans).

–

Lewis Lebentz