If you want to send someone a message, you have to use their public key. If a message is encrypted with your public key, it can only be decrypted by your own private key.
Yes, he meant if he gives his email out to a website (amazon.com, say) he wants to give them his PGP public key as well so that all correspondence to him will be encrypted through PGP.
I would like this, too. Internet, please get on that.
Edit: ok your edit makes more sense now :)