I agree with you “I’m still fairly confident from a macro perspective that forcing Nvidia/CUDA scarcity will create innovation in the Chinese ecosystem”
I guess that’s true—though if I understand your position on Sharp China, it’s that the Chinese ecosystem is going to evolve their way out of Nvidia dependence regardless. Given that, we might as well restrict it to the extent we can now?
At least that’s what I took from listening, but interested if that’s misunderstood on my part.
My take is the level of restriction does make a significant difference in how fast indigenization happens. I also think it’s naive to believe the GPU monopoly will be eternal (it’s the same shortage dynamic)… but the difference can be next year or next decade depending on how hard we squeeze on the supply. Most Chinese companies aren’t that interested in helping be Huawei’s guinea pigs (whatever the government asks) but they will put that effort in, if given no other option.
the political directive has been set, doesnt matter what the us does now. from the april politburo study session: Xi Jinping emphasized that to seize the initiative and gain the advantage in the AI field, breakthroughs must be achieved in fundamental theories, methods, tools, and more. We must continuously strengthen basic research, concentrate resources to overcome challenges in core technologies such as high-end chips and foundational software, and build an independent, controllable, and collaboratively functioning AI foundational hardware and software system [集中力量攻克高端芯片、基础软件等核心技术,构建自主可控、协同运行的人工智能基础软硬件系统]. We should use AI to lead a paradigm shift in scientific research and accelerate technological innovations and breakthroughs across various fields. https://sinocism.com/p/april-politburo-study-session-on
Indeed, and I think investment and flows will go that direction… but the Chinese private sector is not monolithic. Here’s why I think it might take a while if Nvidia GPUs are plentiful: it took years (nearly 15 years now) of hammering out CUDA’s bugs and optimizing it for scientific computing. That’s what makes it so good now.
Scientists and researchers were VERY patient and collaborative with Nvidia because there were no other great choices (vs supercomputers which were a huge jump up in cost and inaccessibility).
It’s not going to take 15 years to copy/follow. But without those real world reps, data, and refinement, CANN is going to improve at a painfully slow pace. It’s AMD’a problem with ROCm still. It’s also similar to fabs—you can’t “theoretically” improve them past a certain point. You need to actually run the process, get the shitty yields, get the data, and iterate until it gets good, which is partly what drives the benefits from scale.
Chinese companies are not altruistic/cooperative/etc enough to just put in the pain if they have a great alternative (quite the opposite given the competitive dynamics). Even if they have public pressure to make a show of it. They’ve been doing “research systems” for years that top the supercomputer charts with Huawei that are never used in production systems. They have to really commit to using Huawei as their real production systems, not just “parallel” or backup.
Anyway—I don’t have a crystal ball either, but that’s the source of my viewpoint on the topic. Appreciate you engaging and helping elaborate on your view!
Lets see how hard China sticks to the rare earth bit. If I was Chinese leadership, I would at least consider engineering a popping of the US AI and tech bubble to push back US capital boom and investments as a strategy.
Agree that it is done and no amount of opening up of GPUs are going to change the path of China building its own semi and related tech. China leadership is not going to led around by the US especially as it has woken up to AI in each successive public policy announcement since earlier this year largely after Deepseek. American capitalists argue the opening up of GPU as a bias and then try to justify it. They say there is no reason to believe that closing that off will prevent them from developing it. It is the same justification of climate change deniers. They point to this study, trends, or another and say there is no definite proof that C leads to higher temperatures. Well, does C actually trap heat? If higher temperatures are a problem, does it no make sense to be extra cautious about C even if it is the earth's normal warming cycle? Unless you believe and have a reasonable explanation how access to US GPUs will slow down Chinese development of AI (and Chinese semi), then would it make sense to be super cautious with GPUs? Last point is that China, history, leadership, and culture is way more inclined to give up short term gain and suffer to reap long term benefits than the US. Inefficient they may be at times but the path is set. If the Chinese companies don't comply, their leadership will be replaced. Trump is playing at authoritarian compared to the Leadership.
Your point about Canadian oil sands as alternate supply is particularly relevant today. Suncor Energy exemplifies the exact framework you describe - they've spent the last decade optimizing their operations to lower breakevan points from $80/barrel to under $30/barrel through technology and process improvements, not new megaprojects. The scarcity of conventional oil and high prices drove innovation in extraction techniques (SAGD, mining efficiency, partial upgrading) that made these resources economically viable. Now with geopolitical tensions rising, these long-duration assets with 40+ year reserves represent the kind of alternate supply that constrains energy price spikes. The same market dynamics that brought solar online during the 1970s oil embargo are now validating oil sands as strategic infrastructure.
Great write up, James. I love the analysis and I completely agree, these restrictions are pushing us faster than ever to innovate mining technologies. We’ve survived on high grade deposits for decades, but it was always inevitable we’d have to go lower-grade, this is just adding fuel to the fire. One difference from the oil analogy is that these minerals are virtually everywhere, if you're able to go low-grade enough. Today certain countries with high-grade mineral resources can restrict their supply, and to some extent that's true about oil as well, but I think holding minerals hostage will be particularly hard for any country to do once low-grade mining becomes efficient.
I'd be very interested to hear the author's perspective on Jevons Paradox—which we're witnessing in real time as total AI compute costs *rise*, not fall, despite per-token efficiency gains.
While inference has become orders of magnitude less expensive on a per-token basis, newer reasoning models (o1, Claude with chain-of-thought, etc.) consume exponentially more tokens per query. A single "thinking" task can burn 10,000+ internal tokens to produce a 200-word answer—some cases use 600 tokens for just two words of output. Whatever efficiency gains we've achieved are being drowned by exploding token volume. Google now processes 980 trillion tokens monthly, up from under 10 trillion in early 2024 (a 130x increase in 18 months). The cost per token dropped 99%, but total spending keeps climbing with no end in sight.
Just coming back from a week of company visits in Shanghai with UCLA Anderson, I agree that Chinese firms will innovate once forced from 0 to 1. Whatever the current administration is attempting to achieve through trade dominance, it seems to only shift markets away from the US being the de facto first choice.
I actually talk about it both in my book and some prior pieces—with some of those exact stats as well!
Jevons Paradox isn’t really much of a paradox versus the normal way a lot of markets work. This shortage argument is the same supply/demand curve dynamic where high prices take certain demand segments totally offline and drive new supply to come online, flattening the curve. Low prices, on the other hand, bring more demand segments online—and if there isn’t a huge supply segment that gets knocked offline (there isn’t a huge marginal cost differential between suppliers) you’ll see total spending rise.
But yeah, the more markets are open/free to technologically dominant players, it’s hard to move from low end (extractive, low end manufacturing) to high (high tech, services). The US effectively closed itself off for the benefit of those lagging it in tech.
I agree with you “I’m still fairly confident from a macro perspective that forcing Nvidia/CUDA scarcity will create innovation in the Chinese ecosystem”
I guess that’s true—though if I understand your position on Sharp China, it’s that the Chinese ecosystem is going to evolve their way out of Nvidia dependence regardless. Given that, we might as well restrict it to the extent we can now?
At least that’s what I took from listening, but interested if that’s misunderstood on my part.
My take is the level of restriction does make a significant difference in how fast indigenization happens. I also think it’s naive to believe the GPU monopoly will be eternal (it’s the same shortage dynamic)… but the difference can be next year or next decade depending on how hard we squeeze on the supply. Most Chinese companies aren’t that interested in helping be Huawei’s guinea pigs (whatever the government asks) but they will put that effort in, if given no other option.
the political directive has been set, doesnt matter what the us does now. from the april politburo study session: Xi Jinping emphasized that to seize the initiative and gain the advantage in the AI field, breakthroughs must be achieved in fundamental theories, methods, tools, and more. We must continuously strengthen basic research, concentrate resources to overcome challenges in core technologies such as high-end chips and foundational software, and build an independent, controllable, and collaboratively functioning AI foundational hardware and software system [集中力量攻克高端芯片、基础软件等核心技术,构建自主可控、协同运行的人工智能基础软硬件系统]. We should use AI to lead a paradigm shift in scientific research and accelerate technological innovations and breakthroughs across various fields. https://sinocism.com/p/april-politburo-study-session-on
Indeed, and I think investment and flows will go that direction… but the Chinese private sector is not monolithic. Here’s why I think it might take a while if Nvidia GPUs are plentiful: it took years (nearly 15 years now) of hammering out CUDA’s bugs and optimizing it for scientific computing. That’s what makes it so good now.
Scientists and researchers were VERY patient and collaborative with Nvidia because there were no other great choices (vs supercomputers which were a huge jump up in cost and inaccessibility).
It’s not going to take 15 years to copy/follow. But without those real world reps, data, and refinement, CANN is going to improve at a painfully slow pace. It’s AMD’a problem with ROCm still. It’s also similar to fabs—you can’t “theoretically” improve them past a certain point. You need to actually run the process, get the shitty yields, get the data, and iterate until it gets good, which is partly what drives the benefits from scale.
Chinese companies are not altruistic/cooperative/etc enough to just put in the pain if they have a great alternative (quite the opposite given the competitive dynamics). Even if they have public pressure to make a show of it. They’ve been doing “research systems” for years that top the supercomputer charts with Huawei that are never used in production systems. They have to really commit to using Huawei as their real production systems, not just “parallel” or backup.
Anyway—I don’t have a crystal ball either, but that’s the source of my viewpoint on the topic. Appreciate you engaging and helping elaborate on your view!
All true, but the companies are not really the deciders when the Party sets a priority.
Lets see how hard China sticks to the rare earth bit. If I was Chinese leadership, I would at least consider engineering a popping of the US AI and tech bubble to push back US capital boom and investments as a strategy.
Agree that it is done and no amount of opening up of GPUs are going to change the path of China building its own semi and related tech. China leadership is not going to led around by the US especially as it has woken up to AI in each successive public policy announcement since earlier this year largely after Deepseek. American capitalists argue the opening up of GPU as a bias and then try to justify it. They say there is no reason to believe that closing that off will prevent them from developing it. It is the same justification of climate change deniers. They point to this study, trends, or another and say there is no definite proof that C leads to higher temperatures. Well, does C actually trap heat? If higher temperatures are a problem, does it no make sense to be extra cautious about C even if it is the earth's normal warming cycle? Unless you believe and have a reasonable explanation how access to US GPUs will slow down Chinese development of AI (and Chinese semi), then would it make sense to be super cautious with GPUs? Last point is that China, history, leadership, and culture is way more inclined to give up short term gain and suffer to reap long term benefits than the US. Inefficient they may be at times but the path is set. If the Chinese companies don't comply, their leadership will be replaced. Trump is playing at authoritarian compared to the Leadership.
Your point about Canadian oil sands as alternate supply is particularly relevant today. Suncor Energy exemplifies the exact framework you describe - they've spent the last decade optimizing their operations to lower breakevan points from $80/barrel to under $30/barrel through technology and process improvements, not new megaprojects. The scarcity of conventional oil and high prices drove innovation in extraction techniques (SAGD, mining efficiency, partial upgrading) that made these resources economically viable. Now with geopolitical tensions rising, these long-duration assets with 40+ year reserves represent the kind of alternate supply that constrains energy price spikes. The same market dynamics that brought solar online during the 1970s oil embargo are now validating oil sands as strategic infrastructure.
Great write up, James. I love the analysis and I completely agree, these restrictions are pushing us faster than ever to innovate mining technologies. We’ve survived on high grade deposits for decades, but it was always inevitable we’d have to go lower-grade, this is just adding fuel to the fire. One difference from the oil analogy is that these minerals are virtually everywhere, if you're able to go low-grade enough. Today certain countries with high-grade mineral resources can restrict their supply, and to some extent that's true about oil as well, but I think holding minerals hostage will be particularly hard for any country to do once low-grade mining becomes efficient.
I'd be very interested to hear the author's perspective on Jevons Paradox—which we're witnessing in real time as total AI compute costs *rise*, not fall, despite per-token efficiency gains.
While inference has become orders of magnitude less expensive on a per-token basis, newer reasoning models (o1, Claude with chain-of-thought, etc.) consume exponentially more tokens per query. A single "thinking" task can burn 10,000+ internal tokens to produce a 200-word answer—some cases use 600 tokens for just two words of output. Whatever efficiency gains we've achieved are being drowned by exploding token volume. Google now processes 980 trillion tokens monthly, up from under 10 trillion in early 2024 (a 130x increase in 18 months). The cost per token dropped 99%, but total spending keeps climbing with no end in sight.
Just coming back from a week of company visits in Shanghai with UCLA Anderson, I agree that Chinese firms will innovate once forced from 0 to 1. Whatever the current administration is attempting to achieve through trade dominance, it seems to only shift markets away from the US being the de facto first choice.
I actually talk about it both in my book and some prior pieces—with some of those exact stats as well!
Jevons Paradox isn’t really much of a paradox versus the normal way a lot of markets work. This shortage argument is the same supply/demand curve dynamic where high prices take certain demand segments totally offline and drive new supply to come online, flattening the curve. Low prices, on the other hand, bring more demand segments online—and if there isn’t a huge supply segment that gets knocked offline (there isn’t a huge marginal cost differential between suppliers) you’ll see total spending rise.
But yeah, the more markets are open/free to technologically dominant players, it’s hard to move from low end (extractive, low end manufacturing) to high (high tech, services). The US effectively closed itself off for the benefit of those lagging it in tech.