FP16 multiply with FP32 accumulate performance is on par if not slower than Titan RTX due to half rate which is what the parent was likely referring to.
BTW - Whether it "could" be faster is indeed relevant because some of us are holding out for a Titan GPU next year with this unlocked. If you have unlimited budget or are under time constraints then by all means get the 3090, it is a beast. But if one has a 2080 TI then it's an important consideration.
I'm obviously aware of the theoretical limitations they referred to. My point that it smokes the Titan RTX in real world benchmarks stands, there's more to machine learning performance than just that one stat. Performance should be measured against real-world use cases, not whether there are lines in the whitepaper you object to.
If you can wait until next year you should always wait until next year, because there will (almost) always be something better than what is currently out. That's unrelated to whether or not the 3090 is good for doing ML research; it objectively is.
[1] https://www.nvidia.com/content/dam/en-zz/Solutions/geforce/a... (Appendix A)
BTW - Whether it "could" be faster is indeed relevant because some of us are holding out for a Titan GPU next year with this unlocked. If you have unlimited budget or are under time constraints then by all means get the 3090, it is a beast. But if one has a 2080 TI then it's an important consideration.