An opportunistic politician would be wise to draft a bill around surveillance transparency and go to all of the people voting against Section 702 here for support (solid arm twist that backs them into a PR corner).
> I have the feeling that the introduction of automatic QA may raise the bar of quality for new releases of software, and maybe partially compensate for the lower quality of the code produced at high speed with the use of automatic programming.
In theory. The only difference between today and "the aughts" is that we have machines that can spit out a ton of code very quickly.
Nothing has changed about the discipline or honesty around testing (you can skip automated tests even faster now if you wish). You can and should work with AI to write tests, but you have to know the difference between a good test and a "looks good on paper" test in order for it to truly be effective and raise the quality of what you're building.
> I have the feeling that the introduction of automatic QA may raise the bar of quality for new releases of software, and maybe partially compensate for the lower quality of the code produced at high speed with the use of automatic programming.
I've been building a compiler with LLMs for a memory safe language like Rust with near zero cost abstractions (no GC), but with WAY less cognitive overhead.
I can tell you right now:
1) It's 100x more than I could have achieved with zero compiler design experience.
2) I'm HIGHLY skeptical that LLMs can build something of this complexity (in some ways it's more difficult than implementing a Rust compiler) - so the testing is quite robust - 3 different systems (unit, integration, fuzz tests) each with mutant testing, each with between ~65-90% line coverage and ~50-80% branch coverage, combined with ~99% line coverage and ~86% branch coverage.
There is ZERO chance I could get something even close to this level of "working" by myself ever - let alone with minimal effort.
The test is kind of simple - if LLM's can do this... They should be able to do just about anything... Compilers are notoriously difficult to verify they actually work, rather than just kind of work sometimes...
People can say I'm wasting my time all they want.
But, one, it's been enlightening. I'm literally in awe of what they can do and have done.
Two, I've developed a bunch of tooling / metrics necessary to get them to be able to do something at this level of complexity without falling over themselves. And I think it can work at scale pretty easily.
Nearly all of the research comes from the 80s or farther back for the complexity metrics.
You're not wasting your time; LLMs have written plenty of compilers. Compilers are easy for LLMs to work on, because their level of verifiability is very high. That is, an LLM can easily determine whether what a compiler is doing is correct or incorrect.
Automated verifiability goes down once a software project incorporates things like:
Hate to be a pedant, but that's really not what "zero cost abstractions" means. The idea behind those is that you get a cleaner interface to some gross machine functionality/OS API/etc. layer, but don't pay a performance cost vs. using the gross lower-level layer. E.g. Rust's Option, unlike C++'s std::optional.
What you're thinking of is "no runtime" or "lightweight runtime", which does often mean "no garbage collector".
Rust's zero cost abstractions mainly stem from its affine ownership model managing memory lifetimes safely and correctly with zero cost - as that is the killer feature... That's what I do.
When people think of "zero cost" they don't think about std::optional. They think about not having to manage memory lifetimes AND NOT having to pay for a Garbage Collector to do it for you. That was always the trade you made until Rust.
I add on some cost to locks to prevent deadlock, and some cost to loops to insert co-operative yields in concurrent contexts unless you turn it off.
Affine as in substructural linear types. They correspond to linear logic [0], and affine logic is named such because the way it's defined corresponds to affine functions. You don't literally need to scale your pointers though.
Based on what I've been seeing/reading online the past ~6 months or so, I think there's a self-fulfilling prophecy going on here.
Developers are concerned about jobs going away, but how often are they pushing back in their orgs about how AI works? In response to "are you using AI to move faster," how many are responding with "yes, but there are some things you should know..."?
If there's no pushback and just pure acceptance of stuff like tokenmaxxing, then what does anybody expect when the broader narrative around AI is that it can help a novice to grind out miracles (i.e., "holy crap, if this is what a novice can do, what can an expert do?!")?
Of course leadership is confused because (it seems) few are asserting expertise, saying "no," and stating a clear case as to why they're doing that.
The default excuse is "I don't want to lose my job" (which is a fair reaction to all of this, especially these days), but it's worth considering when/how that choice is actually just shooting future you in the foot later. It seems there's a broader trend toward compliance more than there is "you hired me to do this job properly, did you not?"
For most companies that I've worked for, pushback is normal and expected as long as you've built some trust/rapport with your management chain.
However with AI, it feels different. I have seen both technical and non-technical managers tell engineers something to the effect of "you aren't prompting correctly" if they aren't able to get the task done within some preferred time frame.
We are seeing the industry revive metrics like lines of code, number of tickets closed, bug's found (looking at you Mythos), and now even "tokenmaxxing". It's exhausting to push back on. These are all things that we know will be gamed. But the individual that brings this up might be viewed as "anti-ai" or something.
If you're an IC, I do think the best thing to do is just go along with it. Sooner or later we will see more shocked-pikachu-faced executives when they realize that engineers are spending tokens just for the sake of it.
Thank you for saying this and being willing to listen.
The worst one I saw is the load balancer config UX/DX. I use CF's load balancer product for clients and so have to do a lot of setup and teardown back-and-forth. Everything related to setting up load balancers is split across multiple screens and/or "wizards" that are extremely confusing.
A lot of the error messages you get are generic at best and so you waste a ton of time clicking between pages and tabs just to set up some pools and attach them to a load balancer.
There's also some inconsistency between how things are labeled, so one thing can have two names and you have to hold that in your head while you move around the UI.
I managed a large enterprise CF account from 2018-2023. Hundreds of load balancers. The UI changed out from under us 3 times, with some big problems being fixed, but introducing new ones. I gave bold feedback directly to the leaders responsible for this, with helpful suggestions on how to make it better.
I was really glad when they fixed the old one that had a big "X" that would delete your load balancer without a warning dialog. But I was not happy that the load balancers got increasingly complex, with settings hidden at multiple layers that you had to independently configure.
Load balancing IS complex, but this is their core business, and in many other places such as DNS, Cloudflare put a lot more thought into making it simple and intuitive to use.
Getting this stuff right takes lots of strong leadership and long-term decision making with determination and wisdom to provide the best experience for customers. Unfortunately I am not confident that is how the business is operating, especially with strong talent being let go or leaving due to lack of fostering of a healthy working environment.
One of the rare pieces of software that actually gets you excited with each new release. Moved to Resolve from Final Cut a few years back and I've never been happier. Looks like this release just continues the already great experience.
reply