Just how bad is it? Really bad. I will include links to references so that you can see for yourself the impact and nature of these breaches. Let’s take a look at breaches just in the past few months:

• 1.29 million Sega accounts
• 100 million or more Sony accounts
• Potetnailly the email accounts of over 2,500 companies serviced by Epsilon
• 360,083 bank accounts at Citigroup
• Tens of thousands of accounts at Codemasters
• 25,000 accounts or more at Square Enix
• 280,000 accounts at Honda
• 1.2 million accounts at the Texas Comptroller’s office
• 10,000 credit cards at the St. George Bank in Australia
• 114,000 accounts of iPad 3G owners
• 200,000 accounts or more at Bethesda Softworks
• $500,000 worth of Bitcoin currency
• 8.63 million patients’ information at the National Health Service Facility in London (UK’s largest employer)
• 18,000 Bioware accounts
• 40 million or more RSA SecurID tokens issued by EMC to over 30,000 companies and government agencies, including half of US banks that use SecurID tokens

This list doesn’t even include the numerous other sites and companies that have been attacked in this time period as well: the Labour Party, CNN, Automatic Data Processing (ADP), Lockheed Martin, the US Senate, the CIA, the IMF, PBS, Epic Games, L-3 Communications, Google, and almost 50 others.

The personal information compromised in these breaches include anything from social security numbers to credit card numbers to just email addresses, usernames, and passwords. Covering up these blunders costs companies millions. The case of the Texas comptroller alone has already cost $1.8 million. As one states, 2011 is set to be the worst year ever for security breaches.

The groups Anonymous and LulzSec have been linked to a number of recent attacks, though certainly the network of hackers is much larger. In response to the significant amount of online plundering, legislation known as the Data Security and Breach Notification Act will require companies to notify authorities and customers within 24 hours of a breach. Hopefully, new legislation will be a catalyst to better security policies.

It has been said that the only secure computer is one that is not on a network. Though it is true that most businesses could not sustain sophisticated attacks, like the one on EMC’s RSA, most of the attacks are not sophisticated. The majority of attacks focus on basic loopholes like SQL injection, security loopholes in servers that haven’t been upgraded (Sony), unencrypted data (Sony, Citigroup, etc), and passing data insecurely through URLs (Citigroup).

Even keeping up to date on the latest security breaches on Yahoo Pipes, the Web Hacking Incident Database, or the DataLossDB, can give network administrators insight into what security loopholes to look for in their own networks. To avoid a PR nightmare and a huge cleanup bill, organizations need to take their online security much more serious.

While quite stealthy, or maybe not so stealthily, this paper is clearly an advertisement, a 20 page advertisement at that, aimed at crackers (or “security researchers” or whatever they want to call themselves) who want to make money cracking software, asking them to contribute their skills for monetary rewards to iDefense and TippingPoint (owned by 3com). For proof, just look page four, there is a chart right in the middle of the page detailing how the best hack in a quarter can pull in 10 grand! While the lamest hack, of the top five, only earns $2000. So, technically, having the most leet hack for every quarter for four quarters would earn you $40,000 a year, better than a manager at Burger King!

So, you got your xsploitn’ skillz and you’re ready to go, but who are these companies you’ll be working for? iDefense specializes in reselling the information provided to them via their exclusive subscriber service. They also provide their paranoid readership with special workarounds to use until a vendor releases a patch. Mainly only government agencies and financial institutions with money to blow sign up for this type of service. TippingPoint sells Intrusion Detection System (IDS) products that use the information provided to them to supply their IDS systems with signatures that will block the offending exploit. Corporate behemoths on the Fortune 500 list are some of their customers. These two companies don’t rely on paid 0day exploits for their only their only fear tactic. They publish vulnerability reports and even IDS signatures for “public” vulnerabilities.

The report goes on to tell us about how its hard to get crackers, who normally shroud themselves in anonymity, are sometimes hard to work with because they, not surprisingly, don’t trust the companies. Ironically, this 20 page advertisement, tells us that most security researchers working with these companies are recruited by word of mouth. They also tell us that they advertise at cool shows like BlackHat and DEFCON. Then they bemoan how hard it is to work with companies and “ethical issues.” Apparently vendors seem to have a problem with them paying for hacks from people who could be “malicious.” Also, the industry cries about how they are encouraging people to find vulnerablitites.

Near the end of their report the acknowledge the truth that everyone already knows. Why would a person with a profitable 0day exploit just submit it to them for a mere chance at $10,000 when they could just sell it to 5 people on IRC somewhere in China or Russian for $4000 each and be guaranteed to make twice as much? They also seem to poke fun of Microsoft and their Trustworthy Computing Initiate with this amusing swipe, “If clients lose confidence in a vendor’s ability to produce secure technology, the damage done to a vendor’s corporate reputation can be translated into lost sales. It is for this very reason that Microsoft has spent billions of dollars to launch their Trustworthy Computing Initiative .” Everyone knows how worthless the resulting Trusted Platform Module (TPM).

While I do see some benefit from these products, I can’t shake the feeling that this is just another form of blackmail or extortion. The problem is, that these services create a group of companies that are privy to this information while everyone else has to wait. This creates two worlds in which “private” vulnerability information sold by these companies is made available to their clients, while companies relying on “public” vulnerabilities could conceivably be attacked and exploited by the companies with access to the “private” vulnerability information. Maybe a non-profit that did the same while contributing the exploit information to normal vulnerability channels like cert would be something I could believe in. If these products end the world of closed source hardware and software then I’m all for it, otherwise I hope they sink to the depths never to return. For that matter is what these companies are doing even legal?!

Before we jump into that, a quick definition, courtesy of Wikipedia:

A distributed denial of service attack (DDoS) occurs when multiple systems flood the bandwidth or resources of a targeted system, usually one or more web servers. These systems are compromised by attackers using a variety of methods, though most commonly it’s due to malware or trojan attacks, either pre-scheduled or triggered by an external event.

There are a number of ways to deal with a DDoS attack, but to find out best practices, I checked with a top sysadmin, who offered this advice based on a recent experience he had with a client site:

As you may know, one of our ecommerce customers suffered a devastating DDoS attack which started early Friday morning and lasted until we finally contracted with a DDoS mitigation service late Saturday night. The service was implemented by pointing the “A” record for the domain to their server. The cost of the service was $350 per month plus $150 setup.

The effects of the attack stopped instantly once DNS resolved to their IP.

There are several of these companies around. All seem to have about the same price structure for the same services. I didn’t do much research but choose the first one to respond on a Saturday. All likely had support available on the weekend but sales staff apparently get time off.

Gathering the information on the attack has been somewhat difficult since during the attack our server was virtually shut down. In fact the only way we were able to get access to shell was to change the DNS to point to a different server then establish an ssh login and run “top” or something similar to keep it open when we switched it back.

It is interesting to note that the attack followed DNS according to the TTL set. We had had it down to 10 seconds as we were in the process of moving the account from one server to another when the attack occurred. The attack followed the DNS within 10 seconds or less. There was very little residual attack activity after the DNS switched and that stopped within a minute or two.

So here is what we think we encountered based on information from the colo support’s anecdotal observations and information from the mitigation service after we blocked the attack. It is interesting to note that the mitigation service does not log activity so the information they provided is from spot observations rather than reliable metrics.

1. Incoming IPs were estimated at reaching as much as 100/sec. Each IP attempted to open between 5 to 25 connections

2. IPs were from all around the net but a sufficient number were from the US so that trying to isolate by country was useless (the customer was not regional but does business across the US).

3. At any one time the number of unique IPs was between three and four hundred. Since the software on the mitigation servers expires the IPs it blocks after 15 minutes and we did not see many instances of the same IPs recurring, the IP pool must have be in the thousands.

4. The sustained attack was estimated to be less than 20MB/sec however an accurate measurement is not available.

Observations:

1. Apache based access control lists proved useless. Apache simply ran out of processes within the first wave and stopped before it could even begin to reject connections. Turning keep alive off and other tuning tricks might have helped if the attack was significantly less but provided no relief as apache was simply swamped in the first few milliseconds.

2. When traffic was moved to a more powerful host it might have been possible to use the firewall by using a script to build the IPtables, however the number of entries in the IP table are limited and the unique IPs exceeded three hundred at a time. That fact plus the overhead of the script and the constant updating of the tables would have brought the server to its knees and the excess IP would still have flooded Apache.

Solutions such as running http as server type inetd (a significant performance hit in itself) with a massive deny list or a very restrictive allow list in the hosts.allow file might have given us back control of the server but would have done little to bring customers back since the store sells country wide and if you arbitrarily block massive ranges of IPs you block customers too. We could have spent days trying to identify safe ranges and never succeeded.

3. There is an additional complication in that the traffic looks like normal traffic with proper handshake and all. Scripts that flag IPs based on the number of connections would only be partially effective since the first few connection would be allowed until the max was reached. The techs at the mitigation service revealed that they relied on pattern matching and signatures which means to be totally effective scripts would need to be constantly updated by someone or some other service similar to virus and spam protection schemes.

4. This level of attack would probably be sustainable by a server with a reasonable firewall implementation in place, although some performance degradation would likely be evident.

5. Finally we were told by knowledgeable sources that there were multiple attacks of this kind against other websites that are in the same business as our customer. According to our sources this is not uncommon. The attacks are not random mischief but are paid for by someone to whittle down the completion. Also this was really a modestly sever attack. I’m told that attacks of hundreds of time more severity than we saw happen regularly.

Take Aways:

Attacks are not likely the result of anything a website owner may have done. You cannot avoid them simply by not offending anyone. If you have any standing in the search engines you will get targeted when someone decides they want the traffic your industry is serving.

You can not wait until an attack occurs to plan for it. Moving to a better hosted server and adding protocols to mitigate attacks will help with smaller attacks and may give you early warning of a larger attack. This attack started sporadically with reports of the server being slow several days before. We do not know if that was testing or if there is just some normal ramp up to an attack like this.

Smaller server setups simply do not have the resources to fend off even a moderate attack. If you can’t justify putting each ecommerce site on its own managed private server (yeah that’s going to happen) then perhaps getting an MPS and stacking several accounts on each with separate IPs might be a solution. Hopefully only one of your accounts gets attacked at once and perhaps the MPS firewall could be made to be effective at protecting all of the sites (needs a little engineering I suspect).

Better still create your own DMZ and front end all of your accounts with a robust firewall appliance (probably not as easy as it sounds).

All in all this has been a wakeup call for us. It is without a doubt a topic that we will give great attention to from now on. I hope this post will be helpful to you all and I thank you all again for your suggestions and offers of help.

Comments

This led me to start this discussion which is most excellent because it shows how to do password security right. I’ll be honest, I’m going through right now and changing all my passwords because I was practicing several of the bad practices that Twitter’s employees were. I bet many of you are doing the same stupid things too.

While I’m on this topic, last week the hard drive in my Mac died. I lost a few days of videos and emails because I wasn’t backing up as often as I should be. Naughty me. The drive just stopped right in the middle of me working. Apple replaced the drive but that didn’t help me get back the videos and emails. Today I’m setting up my new hard drives with JungleDisk. I don’t care what you use to back up, but I know lots of you aren’t. I bought a couple of 1.5TB drives from Seagate, too. Costs $159 at Best Buy and probably cheaper online. No excuses for not backing everything up now. You haven’t done it, have you? (I know most people don’t back up).

Anyway, just a friendly reminder to pay attention to these things before you get bitten.

Comments

It’s an unexpected update, given that version 2.8.1 was released less than two weeks ago. But good to see that the community involved in WordPress development is on the case and with a quick fix.

The announcement post says this about the issue:

WordPress 2.8.2 fixes an XSS vulnerability. Comment author URLs were not fully sanitized when displayed in the admin. This could be exploited to redirect you away from the admin to another site.  Download 2.8.2 or automatically upgrade from the Tools->Upgrade page of your blog’s admin.

I’m updating and I recommend you do, too, if you run WordPress.

Comments

The hack injects php code into the settings file, and will then proceed to try to inject the malware into as many other files as possible. The key names that the hacker uses is Krisbarteo and MagicOPromotion so if your SMF system has either of those accounts, you need to head on over to the simple machines forum and read this thread. Patches are on the way, but in the mean time, there is little you can do to keep from becoming infected sort of changing file permissions and trying to clean up the mess before you get banned in Google as a malware site, that is a completely separate issue.

April Russo over on FriendFeed is hat tipped for posting the alert for everyone to see, and she also posted a quick Google hack to find out how many sites Google see’s as having the Krisbarteo user, with over 300 of them in the Google index. While not a lot, the potential for mayhem is here. It is also good to see that SMF is actively working with the community of users and being completely transparent on how this hack works. Kudos to SMF for working hard to fix the issues, and address the community.

What is interesting is that the hacker is using an older 2008 method for taking over the system, a masked file with PHP code, that has the extension JPG or GIF. This is one of the reasons why you want to make sure that systems do not execute code that is coming in from another direction. This is a classic hacker trick, and has been used successfully for years. In the mean time, follow the thread and follow the cleaning instructions on the SMF forum.

Comments

I will give credit to bandit defense for posting something interesting and new when it comes to WordPress hacking. The process is simple and elegant, which is always a good thing when it comes to hacking. The problem is that it relies on a number of security flaws that may or may not be present in the system. That is what will make this hard to accomplish unless you already know things, or the web site is poorly secured.

I made references in my previous post about the Semisecure Login WordPress plugin about how if an attacker gets a WordPress username and password for your website, it can be used to wreak havoc on the web server that’s hosting it. This post will show you exactly how to do that. It doesn’t teach how to hack WordPress installs. That would be a very interesting thing to talk about, but I’m honestly not the most knowledgeable on the subject (any comments or emails to me about it would be greatly appreciated). Rather this is what an attacker could do if they already successfully have access to an account. Source: Bandit Defense

The first thing the hacker would need to know is the admin password to the WordPress installation. Usually the WordPress password that people use is either the default password made when the account was initialized, or they use some nice dictionary word that would be easier to brute force your way into. I would not hesitate to guess that the majority of WordPress installations have one or the other. If it is the default password, then it is a combination of numbers and letters of varying lengths that will take time to brute force.

Bandit does bring up the idea of the wp_config file, that will give you the credentials to the database. If you want to go mucking about in the database there is also the connect string to the database. Harder if it is local host, easier if the database points off to something like a separate server. That would be tons easier if the goal is to control the database and do interesting things with the person’s site. WordPress gives some incredibly good examples of how to secure your default word press installation, but my belief is that few people will do this; it is always easier to just do the normal install and be on your way to using WordPress the way that it comes out of the box.

What is interesting and something that does make the approach unique is the idea of using the C99 shell, a php file with a ton of shell commands that will let you romp around the web server. What Bandit is counting on to gain access to other web sites is that each web site on the shared server is visible to each other. Not an unusual thought when it comes to low cost shared servers. It is possible to tool around other people’s web sites if the security configuration of the shared server is very poor.

Overall it is fairly unique in that the C99 shell (and he recommends you make your own) is in the uploads directory when you are done, pretty much so allowing you to run around and use PHP commands at the server.

The problem is that the person has to already have access to the WordPress installation in one form or another, and unless the installer did something really bad, like use the same name and password for the DB connection as they are using for accessing admin on the WordPress installation then this starts making sense. If they did not, it is much easier to use the wp_config file and finding out where the database is hosted to do things to someone’s WordPress installation.

Interesting way of looking at WordPress though, the problem is that this is not a “technique for everyone”, there are easier ways of accomplishing things that would be more devastating to the original WordPress installation. The cool part is that few if any have thought of dumping the C99 script in the uploads directory and using that to tool around the web site, and possibly, if the server is very poorly secured, tooling around all the web sites on the shared server.

The other really simple way of doing this, Google search “index of site:com +wp_config” will give you pretty much the same power to tool around directories.

Comments

In order to prevent SQL injections, “it’s a good practice to add a layer between a form on the front end and the database in the back end. In PHP, the PDO extension is often used to work with parameters (sometimes called placeholders or bind variables) instead of embedding user input in the statement. Another really easy technique is character escaping, where all the dangerous characters that can have a direct effect on the database structure are escaped. For instance, every occurrence of a single quote ['] in a parameter must be replaced by two single quotes ["] to form a valid SQL string literal.”

For preventing cross-site scripting (XSS), Google recommends the following measures:

• Stripping the input that can be inserted in a form (for example, see the strip tags function in PHP);
• Using data encoding to avoid direct injection of potentially malicious characters (for example, see the htmlspecialchars function in PHP);
• Creating a layer between data input and the back end to avoid direct injection of code in the application.

Tune into the post for more information!

Comments

The American Football League is looking for information on the hacker that broke into the Dolphin Stadium Web site and hacked it so that it would deliver malware to the millions of people who will be visitng the web site on Feburary 1st, 2009. The site was cleaned up and no longer poses a threat to people, but in the longer run, this is part of a far more interesting trend in malware delivery.

American Football fans looking for information on the Super Bowl in Miami may have found themselves with a nasty malware infection following a successful web attack on Jannuary 30th, 2009 . Dolphin Stadium, the venue for the game, had its website compromised and injected with exploit code, a stadium spokesman told vnunet.com. The attack was detected and removed within a few hours, and the site currently poses no danger to users. Source: Vunet

Hackers go where the people are, and when building a botnet hackers are going to take advantage of big events like the Superbowl. What is problematic is that the site was hacked, I would have thought that they would have had someone at least pentesting the site before the event. There is no indication that they did, there is also no indication that the AFL did not do this either. The draw of the event is what drew the hackers, and while it is easy to say “their security could be better” at least they discovered it quickly and fixed the issue, or at least got the site to stop delivering malware.

Good for the AFL security team that they fixed this quickly, but anyone who visited the site needs to run malware scans and hope that their AV (anti-virus) will pick this one up. The other lesson to pull away from this is that any large event is going to draw hackers. We have seen this with the elections, and other major events. If you are planning on holding a major event, then you want to monitor your site closely to make sure that if it does get hacked you can fix it quickly. The other thing to do is pen test the web site just to make sure.

Comments

Monster.com and USAJobs.gov have been hacked with the loss of login information, contact information, and in some cases, phone numbers, demographic information and userid’s and passwords. If you have an account there it is time to change your password, and do not make it the same as everywhere else. You might just want to change all your passwords if you use the same one everywhere you go.

As is the case with many companies that maintain large databases of information, Monster is the target of illegal attempts to access and extract information from its database. We recently learned our database was illegally accessed and certain contact and account data were taken, including Monster user IDs and passwords, email addresses, names, phone numbers, and some basic demographic data. The information accessed does not include resumes. Monster does not generally collect – and the accessed information does not include – sensitive data such as social security numbers or personal financial data. Source: Monster.com

What is interesting, and hence the under reporting of this break in was that Monster decided that they would just simply do a press release on their web site rather than letting people know by email. If you do not follow the security blogs, and have not been to monster lately you probably didn’t know about this issue. The good part and for this monster gets many kudo’s they have a warning message prominently displayed on their web site on the right hand side right below the login that there is a new security notice.

While they might not have sent you an e-mail, they are at the very least pushing a good warning label on their home page, and the press release is legible to just about everyone. Monster did ok here, and much better than their first data breech.

Go change your passwords, today.

Comments