A post on Marginal Revolution is so good, I have to quote in its entirety before commenting.
From my time in both the military and healthcare I can say that the biggest problem are the compliance costs.
For example, I have a phone app that allows me to send texts. We pay very good money to have said app. It does nothing that my phone cannot innately do – except be HIPAA compliant. EMR software is clunky, an active time suck, and adds little or no value … but we are required by law to use it. In each case there are scads of less specific programs out there which are insanely cheaper and more functional, but those programs cannot justify the costs of becoming compliant for a small niche of their business.
In the military we had similar difficulties. If you want systems to be secure, you need to pay extra as the marketplace does not do real security for consumer goods. Likewise, if you worry about logistical tails, building in assured access drastically increases costs.
And I fully suspect that prices will continue to diverge. As ever more of the internet ends up in a giant interconnected mess there will be fewer people able to code in a secure fashion. There will be fewer parts of the ecosystem that can be used by security conscious actors.
Then we get to actual procurement itself. People worry that arcane institutions will somehow make off with lots of money and spend it either poorly or nefariously. Absent easily observed price and cost data in both sectors we began developing rules. These rules drive firms out of the market (e.g. we needed some light interior remodeling to comply with a regulation that specified inches between things, the contractor who has been most affordable and highest quality refused to bid because the hassle on his side was too great). Eventually the rules become too complicated and you start needing specialists to interpret them. Costs skyrocket and firms abuse rules to pad profits. Then the lawyers get involved and things get more expensive. Again, medical and military consumers become a captive market facing greater monopoly as fewer firms can navigate the thicket of rules to even try to make money.
Then we have the problem that people look at these sectors and say that it is public money. All public money should help with goal X (e.g. going “green”, affirmative action, boycotting South Africa/Israel, patriotism, “America first”) and then we become even more overly constrained. Find vendors who meet one hurdle is hard, finding ones that meet 30 is nigh unto impossible unless the vendor is engineering the firm to market solely to this niche – and charging monopoly rates as his reward.
Any single thing would not be too bad for prices, but the marketplace in general is diverging from military and healthcare. Even education is diverging with mandates in FERPA and political business constraints. We have pretty effectively restricted supply, why exactly would we not expect an increase in cost? This story seems much broader than just healthcare and military procurement. The story also clarifies a bit why it's going to be hard to fix. The thicket of regulations often have a purpose -- security, to protect patent privacy, or more importantly, for military applications. But we do not often ask properly what the cost of extra regulations is. Even well done cost benefit analyses tend to take the supplier network as given, and ask what it will cost them to add just one more step. That the network will shrink and the number of potential entrants shrink more -- the best protection against monopoly power -- is really not part of any cost benefit analysis. The note also points slightly to the public choice problem. The few companies who become specialists at meeting regulations become advocates for the regulations, which puts them in fine position with the army of bureaucrats who promulgate and enforce regulations. Yes, military text messages probably need high security. Does every doctor's text message to a patient need the same?
It doesn't take long to see in this post a reading of many contemporary economic ills. The perception of increasing monopoly power fits well. The decrease of small business formation and increasing size of businesses fits. And we can think of a number of industries that have the same problem. Banking is obvious.
General aviation is a tiny, but clear example. Go to your local airport, and contrast the ramp (where planes park) to the parking lot. The ramp is typically an excellent example of a Cuban used car lot. Lovingly maintained aircraft either from the 1950s or designed in the 1950s predominate. Beautiful, yes, to nostalgic eyes, but not exactly practical. Small aircraft engines are much less reliable than automobile engines. Why? Well, they all must be FAA certified, and it's not worth the cost to certify, say, a new model of spark plug. The parking lot is full of Teslas. Well, in Palo Alto. BMWs elsewhere. But stuffed with the latest technology. Planes are not inherently more durable than cars. They're just regulated differently.
The HIPAA regulations, making electronic medical records every doctor's nightmare, and adding billions to costs, are actually an improvement. We can all remember the not too distant past, and sometimes still present, that doctors needed us to fax things around, because of the same regulations.
The central point of the story is the interplay of new technology and regulation. Our technology has huge fixed costs. Commercial off the shelf technology, usually "pretty darn good" is amazingly cheap and effective. Specialized technology written to constantly evolving regulation is nightmarishly expensive, and usually not very good. And leads to cronyism and monopoly. The cost of regulation is higher than you think. Make sure the benefits are appropriate.