My experience with them doesn't quite fit either: I've primarily used LLMs for giving me hints when I'm struggling with a leetcode problem or similar. They're surprisingly good at it, providing you regularly remind them to provide little clues only.