Well, isn't that git or any other sane source control system do that for you? You are maintaining your own repositories of these external dependencies anyway. So why split pieces of your project on two different repos each with their own versioning, network protocol, authentication etc. What exactly are you getting in return for the added complexity?
git doesn't necessarily handle large binaries well (less true in recent versions). Also, when using Maven (etc) you can upgrade libraries just by bumping up the version # rather than hunting down the new jar manually. You can also use the same cached jar in multiple apps/modules rather that committing it in all of them - good if you've got a modularised application.
There are also other benefits like automatically downloading and linking sources and documentation (more relevant if using an IDE).