Even if they have a passphrase it has to be strong enough to withstand brute force for months. If you're not smart enough to keep the private key separate from the crap you're trying to protect, why should I think you're smart enough to ensure your passphrase is good enough?