The advantage of the passport is that it does not have address. It also does not have SSN (or a key that is easily cross-referenced to SSN). You can have a driver's license and a passport (I don't believe you're really allowed to have the license and the non-license ID). In addition, many people look down on the the non-driver's ID's believing that the holder is unable to get a license (or it has been revoked for some reason) rather than a reasoned choice.
I'll use my passport when I know I'm going to need specific ID (new job, getting a mortgage, etc.).
- David