Valued Hosting Clients,
Most of you are aware our Wordpress server ONEFASTPUPPY was hacked Saturday morning. The purpose of this email is to tell you what happened, the immediate action we took, how the hacker got in, and how we intend to prevent similar occurrences in the future.
First, I want to apologize for it taking me several days to get this email out to you. Just hours after cleaning up our recent web server hack, I was hit by a human virus that took me off my feet for a few days. So goes life.
Next, I apologize for its length. To be clear, there is nothing here you MUST read. Within the week, you’ll receive an email about changes to our Hosting Terms of Service. That one is important – watch for it!
The first site hacked showed up on our radar just after 7:15 am, moments after it was hacked. Over the course of the next 2-1/2 hours, sites began to fail, showing a “hacked by” message instead of the site itself.
Within minutes, I was tending to the first site to fall, in an attempt to discover how the hacker got in and plug that hole before more damage was wrought. Unfortunately, the exploit moved quickly and I found myself in a race, unable to stay ahead of the damage. Meanwhile, Faraz, our Lead Tech, determined and then researched the hack signature and discovered how to fix the damage done to sites and lock the hacker out.
Ben, our Hostmaster, was brought in to make changes to the server itself, part of our lock out strategy, while I set to restore sites from backups and Faraz worked the list of those that needed more than a backup restore.
Within 14 hours, 195 sites were restored, most losing zero data. A small handful of sites lost 4- 24 hours data; this was due to having to roll to a backup that had pre-dated the hack. Over the next 48 hours, as errors were reported on half a dozen sites, they were resolved.
How did the hacker gain access?
A site owner, who hosts with us but does not use us as their webmaster, allowed Wordpress and plugins to become outdated. The hacker exploited a known issue in an outdated plugin to get in.
Once in, they deposited code in a sidebar widget of the site, which
This exploit is known as defacement, meaning no Wordpress files (your text, photos, member info, etc.) were changed. The only changes made were to the Wordpress database in the following locations: the Wordpress title, charset, and widgets.
- renamed the site
- changed the charset from UTF-7 to UTF-8
- changed permissions on the wp-config.php file
- destroyed the sidebar(s) and all widgets
- left the hack message
This is important for several reasons I won’t go into; what you want to know is that your data was NOT compromised.
Once all sites were safely restored and we were sure the server was locked down, Ben, Faraz and myself looked at what was needed to prevent similar such occurrences in the future.
It was agreed that the server, ONEFASTPUPPY, is running top-level security and no additional changes were required.
We found that we employ robust security measures on the sites we maintain, and adequate security on those we webmaster, even if when there is no maintenance agreement. While no site is “hack-proof” all of these sites are at a level of security we are proud to represent. No changes are needed other than to encourage more site owners to employ a maintenance plan for their own financial protection.
However, I have not required the small handful of site owners hosting with us while using other webmasters, to hold to the same standards. I never intended to host sites other than those beachdog.com builds and/or maintains. Our beachdog.com Hosting Terms of Service haven’t kept pace as we allowed a few of these sites onto our server. For this, you have my personal apology and I am absorbing all cleanup costs (with the exception of the offending site).
Not surprisingly, updates to our Hosting Terms of Service will be heading to each of you within the week. The biggest change will be detailing the requirement for site owners to keep Wordpress, plugins, and other scripts and/or programs current.
If you are on a maintenance plan, we do this on your behalf – nothing for you to think about.
If we webmaster for you but you are not on a maintenance plan, we make these updates as requested by you. We also make them without regard to their effect on your site if doing so is necessary for server-wide security. This is why maintenance plans are such a good deal; if something breaks, we fix it no charge. Without the plan, you pay for that fixing time.
Site owners not keeping software current will be made accountable for all cleanup fees should their site be responsible for compromising the server. Had this been in force for this particular hack, the bill to the offending site owner would have been in excess of $5,000 due to the number of hours and technicians working after-hours and overtime. This is why I continue to profess our $275/year basic maintenance plan is not only smart business, but also financial security.
Please, take a few moments to click around your site to ensure all is well. Any hack-related issues reported before August 5th will be taken care of at no charge.
If you are not on a maintenance plan, please consider one. More info:
Thanks in advance for your patience. Between the hack and my getting hit by a bug, things are a bit backed up over here. All requests are being triaged and worked as quickly as possible by the team. If your need is urgent, be sure to tell us.
I take your trust very seriously and welcome your feedback.
Keleigh Schwartz, President