thats game design not something engine related. its been like this since before demons souls, i think it started out way back with the first kings field. its just the way fromsoft do things. everyone knows what to expect from them.
Just to underline your point, Sekiro's NPCs behave in a much more obvious and linear way relative to From's other games, so it's clearly a game design decision and not a technical one.
I would agree, but there are still some interactions that require doing the same thing repeatedly in order to make progress (eating rice balls, eavesdropping).
Especially in Elden Ring, it also contributes to the ludonarrative cohesion. Hyetta's quest (no spoilers!) isn't meant to be completed by everyone, her story is a foil to Melina's "default" questline that is mandatory for beating the game. Other characters (eg. Boc, Sellen, Selivus, Thops) also have quests that are easy to start but deliberately obfuscated towards the end.
The JRPG logic is annoying, but From uses it to beg important questions about the game world and the player. Fromsoft characters like Lautrec live in infamy for being so slippery and deliberately misleading.
They're victims of success. Lautrec was so infamous some people (read: me) are too scared to play the game without a guide now.
"What if I missed some NPC and I can't level up anymore?!"
That's what I had; I was going through Elden Ring thinking "I will just play the game straight / naive the first time, no guides". Got close to the city and thought "...this feels like a point of no return / the end, better check if I missed anything important!". 100 hours later (or whatever) and I found out it's more like the halfway point.
I just accept I'll miss things / get some sub-optimal outcomes for my first play, though I do try to explore a lot and sometimes player messages help rather than troll/spoil/be funny. Guides only come out for NG+ and beyond.
And even before King's Field, this is just the way games used to be like. Games couldn't elaborate too much, so your options were:
a) you lucked out and it turns out your brain works exactly like the designer's and the cryptic hint make sense to you,
b) you use a guide/ask a friend/call a hotline,
c) you like the game so much you end up brute forcing your way through all of that stuff,
d) you just ignore that sidequest. I sure hope for you that finishing the main quest doesn't need you to guess something unreasonably obtuse... *COUGH* Legend of Zelda's final dungeon's location *COUGH*
