I'm admittedly a complete LLM noob, so my question might not even make sense. Or it might exist and I haven't found it quite yet.
But have they considered pivoting some of said compute to some 'private, secure LLM in a box' solution?
I've lately been toying with the idea of training from extensive docs and code, some open, some not, for both code generation and insights.
I went down the RAG rabbit hole, and frankly, the amount of competing ideas of 'this is how you should do it', from personal blogs to PaaS companies, overwhelmed me. Vector dbs, ollama, models, langchain, and various one off tools linking to git repos.
I feel there has to be substantial market for whoever can completely simplify that flow for dummies like me, and not charge a fortune for the privilege.
The problem is currently all the "competing" ideas have a ton of tradeoffs and is rarely one size fits all. Furthermore, it's not clear if the idea you choose will be become obsolete by the underlying model architecture getting better. On top of all that, you are essentially competing with Anthropic/OpenAI/Google, where your only advantage is "privacy". Anyone who deeply cares about privacy, and is willing to pay for it, may just likely do it on their own (especially if your CTO is pouting money into "investing" in AI). Anyone who doesn't will likely not want to 6-7 months behind what you can get at OpenAI or Google.
But have they considered pivoting some of said compute to some 'private, secure LLM in a box' solution?
I've lately been toying with the idea of training from extensive docs and code, some open, some not, for both code generation and insights.
I went down the RAG rabbit hole, and frankly, the amount of competing ideas of 'this is how you should do it', from personal blogs to PaaS companies, overwhelmed me. Vector dbs, ollama, models, langchain, and various one off tools linking to git repos.
I feel there has to be substantial market for whoever can completely simplify that flow for dummies like me, and not charge a fortune for the privilege.