The history of /etc/passwd and /etc/shadow files
So, I learned something important today. When you don't have an Internet access but have still got work to do, may be with a heavy heart, but you eventually end up in a library. As it turns out, reading a howto from a book is fun too. I was looking to do a crazy thing on my system(lets leave that story for some other day ) and while I was looking for something related, I came across a wonderful write up on /etc/passwd and /etc/shadow files. I don't know how I missed this stuff while I was learning the basics.
All I had known about these files until today was that /etc/passwd file, inspite of having “passwd” term in its name, is used to store all kind of user related information other than password. And /etc/shadow is used to save the passwords for all the users listed in its companion file, /etc/passwd.
Lets go a little deeper and take a look at the content of /etc/passwd.
[shredder12]$ less /etc/passwd
It contains all kind of user-related information. Each line represents a different user and the various fields are separated by colons, e.g. Username, UID, Description, login shell etc. You may have noticed that the second field is ‘x’ for everyone. This second field is actually the password field which is not used anymore. Yep! its not a coincidence, /etc/passwd was built to save passwords.
Before /etc/shadow was enthroned, /etc/passwd was the source of user’s password. Of course, it didn’t store them in clear text. The passwords were encrypted using a one-way hash algorithm and the final string was saved in the file. “One way” means its easy to obtains hash of a string but there is not reverse path to derive the actual string from the hash. For more information on the algorithm, check the manpage of "crypt".
User authentication is confirmed by calculating the hash of the user supplied string and then matching it with the one in this file. This way the password was not directly accessible by anyone.
An important thing to note about /etc/passwd is that its readable by everyone.
[shredder12]$ ls -l /etc/passwd
-rw-r--r-- 1 root root 2147 2011-02-08 01:52 /etc/passwd
Hence, everyone could see the hashed string. The whole setup was still considered safe because there was no way to get the password from the hash. Even if an attacker decided to copy the hashed string and launch a program that checks every string’s hash to find the password(aka brute-force attack), it would have taken him weeks if not months to do so. But this was the case in 80’s when not everyone had access to a powerful personal computer.
With the advent of more powerful and cheaper computers in later years, it was easier to launch a brute-force attack to break such passwords(especially small ones). So, a better scheme was required to protect the passwords from malicious users. The developers finally decided to keep it in a separate file, which was only allowed access from the root user. This is how /etc/shadow was born.
[shredder12]$ ls -l /etc/shadow
-rw-r----- 1 root shadow 1388 2011-02-08 01:52 /etc/shadow
Lets take a look at its contents.
[shredder12]$ less /etc/shadow
/etc/shadow: permission denied
[root]# less /etc/shadow
The fields here are - username, encrypted password and other optional password aging information. If you notice the users with a “*” in place of an encrypted password, it means that they are not allowed to login and run interactive shells. Their sole purpose is to run processes and maintain file ownership across the system. If you check the shell field of such users in /etc/passwd file, they all are set to /sbin/nologin i.e. no shell for them.