"They do but the larger governments tend to do worse."
Larger compared to what? I think places with the smallest most ineffectual governments are all hell holes. Every decent society has a large central government. The "economic freedom index" is a bit of a misnomer. They define freedom as a government large enough to protect rights. So right out the gate freedom is redefined as a large government that uses force to impose civil society.
Let me pick one example. Your freedom index lists Switzerland as "Free". Switzerland has government imposed universal healthcare. That sounds like a lot of government to me, how about you!
One last thing, none of the countries listed as "FREE" are anything close to a libertarian society based on the NAP.