Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

There's nothing about LLMs that is inherently nondeterministic. Sure, if you're using some API you have no control over, anything could happen. But if you run it on your own hardware, you can make it as deterministic as any classic NLP approach.

And whether you always have to check everything is a separate question from nondeterminism. You could have a deterministic heuristic that is often wrong in a domain where mistakes are fatal, or you could have a nondeterministic model that is almost always correct for a task where errors cost next to nothing.



ChatGPT 4 is inherently non-deterministic. Even across the same exact model version with temperature set to 0.

https://ai.plainenglish.io/gpt-4-is-non-deterministic-af2028...


That's what I meant by "if you're using some API you have no control over, anything could happen." OpenAI could offer access to a fully deterministic version at a higher price point, but they choose not to and there's nothing you can do about that.

FYI, your source is medium-walled blog spam of https://152334h.github.io/blog/non-determinism-in-gpt-4/


If you'd read the source you linked you'd see that no, the system is inherently non-deterministic. They cannot sell you a deterministic version of GPT-4. There is no such thing. They could sell you a deterministic version of some other inferior model.


From the source, which I read:

As a consequence, the model is no longer deterministic at the sequence-level, but only at the batch-level

They could sell you a version where you shove in a whole batch, pay $batch_size as much, and always get the same result for the same batch.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: