Do you then avoid syncing any passwords to your phone to avoid having your two factors in the same place? (And similarly, avoid syncing SMS to any devices where you do have passwords.)