And then somebody loses their private key material and they look to you to fix their problem.

Client-side encryption with end-user key management is not yet practical for the average end user. Until it is (I'm hopeful we'll get there), the average service will require some sort of administrative back door that is controlled by process and people.