I used to be one of those finance guys who could almost code, but not really. CUDA was awesome. Stuff just worked and one did not need a CS degree to do it. Furthermore, like 15 years ago one could get guys at Nvidia to help you. It was easy. Microsoft used to be the same way with SQL Server. They would send you a dll and other crap and your stuff would just work. I built a huge financial modeling software thingy with a SQL Server backend and I did it with a history degree (I am good at math, or at least I used to be).
CUDA is the ultimate lock-in. AMD has made more powerful cards than Nvidia in the past, but even in gaming it would take them years to optimize the drivers to get the performance out of the cards. In some ways it was kind of crazy. You could buy a card, and if you kept your drivers up to date, that AMD card would double its performance over a couple years. The problem is, you want the performance NOW.
For business, AMD was never serious about general purpose compute for non-graphicy problems. Even with Tensorflow, I think 99% of the people using it were using some other dude's docker image with tensorflow and python running a quardro or even GeForce card.
AMD is catching up, and they finally have the money to hire the staff to fix this. Let's all hope that they do. But even then, we need easy solutions guys like me can use. That takes an ecosytem, not simply the tools.
The issue is not only if CUDA is better / faster than the alternatives, but the existing software requiring CUDA. It's not about the technology, but the agreements between corporations. At the end, that's always the bottleneck.
Why isn't there a DirectX module to abstract the underlying technology from the applications? Why won't the applications consume an interface no matter what's at the lower levels? Not because of technology limitations but politics $.
And why is Nvidia allowed to sell dummy bricks at rocket price if what they are actually selling is software in a dumb box? People are buying "hardware" that has not much more than what they bought 8 years ago. So we're almost buying software updates, ala FIFA games.
Again, why? Because of $. Why no regulations? Because who regulates gets enough $ not to really regulate.
Big still, yes. It puzzles me that AMD hasn't yet found its way around this moat. They didn't have competitive hardware for tensor ops until mi300, but now they do, and perhaps they will turn their attention to seriously addressing the software gap. CUDA isn't really important. What is important is PyTorch, and the python numerical ecosystem generally.
I used to be one of those finance guys who could almost code, but not really. CUDA was awesome. Stuff just worked and one did not need a CS degree to do it. Furthermore, like 15 years ago one could get guys at Nvidia to help you. It was easy. Microsoft used to be the same way with SQL Server. They would send you a dll and other crap and your stuff would just work. I built a huge financial modeling software thingy with a SQL Server backend and I did it with a history degree (I am good at math, or at least I used to be).
CUDA is the ultimate lock-in. AMD has made more powerful cards than Nvidia in the past, but even in gaming it would take them years to optimize the drivers to get the performance out of the cards. In some ways it was kind of crazy. You could buy a card, and if you kept your drivers up to date, that AMD card would double its performance over a couple years. The problem is, you want the performance NOW.
For business, AMD was never serious about general purpose compute for non-graphicy problems. Even with Tensorflow, I think 99% of the people using it were using some other dude's docker image with tensorflow and python running a quardro or even GeForce card.
AMD is catching up, and they finally have the money to hire the staff to fix this. Let's all hope that they do. But even then, we need easy solutions guys like me can use. That takes an ecosytem, not simply the tools.
The issue is not only if CUDA is better / faster than the alternatives, but the existing software requiring CUDA. It's not about the technology, but the agreements between corporations. At the end, that's always the bottleneck.
Why isn't there a DirectX module to abstract the underlying technology from the applications? Why won't the applications consume an interface no matter what's at the lower levels? Not because of technology limitations but politics $.
And why is Nvidia allowed to sell dummy bricks at rocket price if what they are actually selling is software in a dumb box? People are buying "hardware" that has not much more than what they bought 8 years ago. So we're almost buying software updates, ala FIFA games.
Again, why? Because of $. Why no regulations? Because who regulates gets enough $ not to really regulate.
NVDA market capital will be $5 trillion within 2 years
Big still, yes. It puzzles me that AMD hasn't yet found its way around this moat. They didn't have competitive hardware for tensor ops until mi300, but now they do, and perhaps they will turn their attention to seriously addressing the software gap. CUDA isn't really important. What is important is PyTorch, and the python numerical ecosystem generally.