In this article I want to talk about the storage of passwords in your systems. Passwords are still the most common way of being able to authenticate a user, but it is very easy to put yourself in a situation where your system is not secure and susceptible to attacks. In this article I want to discuss ways in which you shouldn’t store passwords, and talk about how you can safely store passwords and protect yourself where you have been a victim of data theft.
Storing Passwords in the Clear
Easy of Implementation : EASY
Is Good Idea : TERRIBLE IDEA
When you are developing a system that needs to authenticate a user, the biggest mistake you can make is storing passwords as clear text in your database. You may as well not bother having security as you can’t offer any kind of privacy to your users. This may seem like common sense, but there are still plenty of sites that do this. A user’s password should be secret and only known by the person who it belongs too.
To make matters even worse, some sites will also email out plain text passwords to users if they have forgotten their password. This is just a really bad idea, especially as a lot of email systems are still unencrypted.
Ease of Implementation : EASY
Is Good Idea : BETTER BUT NOT GREAT
A better approach, but still not ideal is to store a hash of the password in your database. A hash is a one way function that encrypts a password. This encryption isn’t based on a key though. A hash is a one way function that can’t be reversed mathematically. There are many different hashing algorithms out there like MD5, SHA1, and SHA256. I personally recommend using SHA256. I wrote an article on how to compute SHA256 hashes in .NET. It really isn’t that difficult .
Hashing passwords falls down in a couple of areas. First, if some people have the same password, which is entirely possible, then their hashes will be identical. If your password hash database is compromised, someone could try to group passwords together to try and guess them. This is made even easier if you store password hint question and answers in the password table, which is another really bad idea.
Plain hashes are a bad idea as they are easy to crack. Whilst a hash can’t be reversed mathematically, you can perform what is called a dictionary attack against the password. You do this by using something that are called rainbow tables. There is a really good article on rainbow tables on Wikipedia.
Here is Wikipedia’s description of a rainbow table.
A rainbow table is a precomputed table for reversing cryptographic hash functions, usually for cracking password hashes. Tables are usually used in recovering a plaintext password up to a certain length consisting of a limited set of characters. It is a practical example of a space/time trade-off, using more computer processing time at the cost of less storage when calculating a hash on every attempt, or less processing time and more storage when compared to a simple lookup table with one entry per hash. Use of a key derivation function that employs a salt makes this attack unfeasible.
Let’s look at a simple example. Let’s say for example we have the password Secret69. Now I have tried to be a good user and pick a reasonably strong password that is easy for me to remember. I have 1 capital letter, lots of lower case letters and for added super strength 2 digits. As I enter this into an online system, it uses SHA256 to hash the password and store it. In fact, why don’t you try this? Goto http://www.xorbin.com/tools/sha256-hash-calculator and enter the password as shown in the screen shot.
This produces a hash code of :
which is stored in the database. This looks pretty complex and therefore should be secure yeah? Wrong. Now go to this following website https://crackstation.net/ and enter the hash code in the box and complete the capture code.
Our reasonably strong and easy to remember password has been cracked and the password “Secret69” has been recovered. A rainbow table file is a huge dictionary of words that can run into the gigabytes. Storage is cheap so these tables can continue to grow. This method isn’t perfect, it won’t be able to recover all passwords, but if have managed to steal the password table from a site then you may easily be able to recover 30% of the passwords without breaking a sweat.
Stealing a password table is hard right? No not really, you don’t necessarily have to do it as an external attack. The data could easily be stolen by people who work in your organisation, it really isn’t hard. There is nothing stopping me right now from copying my company’s entire database onto a pen drive, tablet, ipod and just taking it home!!
Hashing Passwords with a Salt
Ease of Implementation : EASY
Is Good Idea : BETTER THAN NO SALT BUT CRACKABLE
A better option when storing passwords is to salt the password first. A rainbow table is not as effective against one-way hashes that include salts. A salt is a passphrase or random number that is appended to the password text. If we look at the example from the above then our password of Secret69 would become :
Secret69 + 562378246826458264825682642386483628.
This example gives a hash value of :
If we try to crack it with the Rainbow Table you can see that it fails.
The salt value is not a secret value and may be generated at random and stored with the password hash. I personally would store the salts in a different table to the password itself in case someone steals the password table. A large salt value prevents pre-computation attacks, including rainbow tables, by ensuring that each user’s password is hashed uniquely. This means that two users with the same password will have different password hashes (assuming different salts are used). In order to succeed, an attacker needs to pre-compute tables for each possible salt value. The salt must be large enough; otherwise an attacker can make a table for each salt value.
To generate a salt I would use a cryptographically strong random number generator. I described how to use the crypto random number generator build into .NET in this article.
Encrypting your Hashed Passwords with AES, RSA etc
Ease of Implementation : HARD
Is Good Idea : YES WITH GOOD KEY MANAGEMENT (HSM)
The concept of hashing and salting a password should be perfectly suitable for the majority of cases. You can crank this up a notch and encrypt your hashed and salted password but this can add many complexities. If you do decide to encrypt you the hashed password, then you should make sure you use an industry recognized and standard encryption algorithm. I would say the main options today are AES, RSA, or Elliptic Curve Algorithms.
The actual process of encrypting the hashed password is relatively easy. The problems occur when you need to deal with key management. If you are just storing the keys in a config file or even in the database, then you most probably shouldn’t bother encrypting the hashed password as it won’t take much effort for someone to recover the key. If your company has a Hardware Security Module (HSM) available then you can use this to safely store the keys. If you have access to a decent HSM, then this is probably worth it, but you need to think about how often you want to rotate keys and what your key revocation process is going to be. For the majority of system, this is probably a bit too much overhead to make it worthwhile. I would just stick to hashed passwords with long random number salts.
Encrypting your Passwords with BCrypt
Ease of Implementation : EASY
Is Good Idea : GOOD IDEA
There is another alternative to using hashes and salts to encrypt your password, and that is an encryption protocol called BCrypt. BCrypt is an algorithm that is based on the Blowfish encryption algorithm. You can supply an adjustable work factor to the algorithm that determines how expensive and secure the algorithm is. This enables the algorithm to scale to keep up with moores law, so as machines get faster and more powerful for cracking passwords, you can just make BCrypt work hader.
BCrypt is very slow in its operation but that is a small price to pay for an added level of security. There is a good blog post about BCrypt on the Codahale website that explains in more detail why BCrypt is a better solution than standard hashes and salts.
Inventing Your Own Uber Cool Encryption Algorithm
Ease of Implementation : HARD
Is Good Idea : TERRIBLE IDEA
This final option should really be common sense, but I have heard developers suggesting it as a good idea before, so let’s nip it in the bud. The idea is that by designing your own algorithm and keeping it secret, you are implementing a good password security solution. This just isn’t the case. Effective cryptographic algorithm design is hard. Very Hard! I love the subject of cryptography and I have even designed some of my own algorithms before but I would never even consider using one in a commercial context.
Algorithms like SHA256, AES, RSA, and Elliptic Curve etc are all open standards. Anyone can download the specification and even create their own implementation of it. The strength of the algorithms is based wholly on the strength of the key. These open standards have been subject to extensive cryptanalysis by mathematicians with brains the size of planets. Your home-grown solution will not have been subjected to the same level of scrutiny. You should also never rely on the secrecy of the algorithm as a security measure as someone will either crack your algorithm manually or resort to other measures to learn about the algorithm, of which kidnapping is an option.
There is a really good example of where this has tripped up for a company. There is a company called MiFare who create RFID chips. There is one particular chip called the MiFare Classic which is used in many door key security systems as-well as travel payment systems like the London Oyster card. They tried to keep their proprietary algorithm secret, but some clever people managed to work it out and break the algorithm. You can read up on how the MiFare Classic protocol works here from these RSA Conference slides.
Now with not too much effort you can build a system to clone oyster cards and travel for free.
So as you can see, if you want to hash a password or even encrypt it, then make sure you use standard and secure algorithms that have already been subjected to extensive cryptanalysis.
Side Effects of Password Data Theft
We have spent a lot of time talking about the most effective ways of storing passwords in your database, but we should talk about the side effects of not having a decent password storage mechanism. First, if someone steals your password database and can decode the passwords unauthorized people will have access to your systems and services. The impact of this can range in severity depending on what your system is. If you are a Software as a Service provider then people will have access to paid services that belong to someone else. As far as your customers are concerned this is theft.
There are also privacy concerns. Someone having unauthorized access to a system means they have access to lots of personally identifiable information. This is a major invasion of privacy and can lead to identity fraud.
If you are an eCommerce provider or provide financial services, then this opens you up to major fraud by people having unauthorized access to financial accounts. For an eCommerce site you are open to fraudulent purchases of goods. None of these are good situations to be in.
For a company, any of these can be disastrous. You open yourself up to major legal action for fraud, id theft, money laundering, as well as litigation around privacy concerns. This can have a huge financial impact on your company. Also, if your password database gets compromised and it gets into the mainstream press, you can suffer irreparable reputation damage which can be very hard, if not impossible to recover from.
In this article we have talked about the best measures to store passwords in your system. Storing plain text passwords is a very bad idea. Hashing passwords, whilst better, is still not a great solution as you are open to rainbow table, brute force dictionary attacks. A better solution is to store hashed passwords that are hashed with a long Salt value generated using a cryptographically strong random number generator, but this doesn’t make you totally immune to having your passwords cracked. One of the best (and most practical) solutions is to use the BCrypt encryption algorithm. This is a much more computationally expensive operation, but offers better overall security.
We also discussed about using standard encryption of your passwords, but unless you have an effective key management policy, using a Hardware Security Module, the overhead of doing this may not be worth it to you.