Pretty cool with China’s focus on efficiency in the AI stack. DeepSeek was the first eye-opener for how to re-think efficiency, but it appears to happen on all levels of the stack.
Fyi: article is paywalled, so block javascript on page with ublock…
I’m hoping this will go beyond AI stuff as well. Operating systems and a lot of general purpose software is also incredibly bloated. If Chinese companies start optimizing the software stack due to having slower chips, that would be a huge win.
What is this… “Nvidia’s flagship RTX 3090 GPU”? Are we in back in 2020? Half a decade ago? Is this a joke? Even then, it wasn’t the flagship, the 3090 Ti was.
You can argue that a 4090 is more of a ‘flagship’ model on the consumer market, but it could be just a typing error, and then you miss the point and the knowledge you could have learned:
“Their system, FlightVGM, recorded a 30 per cent performance boost and had an energy efficiency that was 4½ times greater than Nvidia’s flagship RTX 3090 GPU – all while running on the widely available V80 FPGA chip from Advanced Micro Devices (AMD), another leading US semiconductor firm.”
So they have found a way to use a ‘off-the-shelf’ FPGA and are using it for video inference, and to me it looks like it could match a 4090(?), but who cares. With this upgrade, these standard Fpga’s are cheaper(running 24/7)/better than any consumer Nvidia GPU up to at least 3090/4090.
And here from the paper:
"[problem] …sparse VGMs [video generating models] cannot fully exploit the effective throughput (i.e., TOPS) of GPUs. FPGAs are good candidates for accelerating sparse deep learning models. However, existing FPGA accelerators still face low throughput ( < 2TOPS) on VGMs due to the significant gap in peak computing performance (PCP) with GPUs ( > 21× ).
[solution] …we propose FlightVGM, the first FPGA accelerator for efficient VGM inference with activation sparsification and hybrid precision. […] Implemented on the AMD V80 FPGA, FlightVGM surpasses NVIDIA 3090 GPU by 1.30× in performance and 4.49× in energy efficiency on various sparse VGM workloads."
You’ll have to look up what that means yourself, but expect a throng of bitcrap miner cards to be converted to VLM accelerators, and maybe give new life for older/smaller/cheaper fpga’s ?
Thanks for taking the time to clarify all that.
It’s not a typo because the paper itself does mention 3090 as a benchmark.
I do tinker with FPGAs at home, for the fun of if (I’m no expert but the fact that I own few already shows that I know more about the topic than most people who don’t even know what it is, or what it’s for) so I’m quite aware of what some of the benefits (and trade of) can be. It’s an interesting research path (again, otherwise I wouldn’t even have invested my own resources to learn more about that architecture in the first place) so I’m not criticizing that either.
What I’m calling BS on… is the title and the “popularization” (and propaganda, let’s be honest here) article. Qualifying a 5 years old chip as flagship (when, again, it never was) and implying what the title does, is wrong. It’s overblown otherwise interesting work. That being said, I’m not surprised, OP share this kind of things regularly, to the point that I ended up blocking him.
Edit: not sure if I really have to say so but the 4090, in March 2025, is NOT the NVIDIA flagship, that’s 1 generation behind. I’m not arguing for the quality of NVIDIA or AMD or whatever chip here. I’m again only trying to highlight the sensationalization of the article to make the title look more impressive.
Edit2: the 5090, in March 2025 again, is NOT even the flagship in this context anyway. That’s only for gamers… but here the article, again, is talking about “energy-efficient AI systems” and for that, NVIDIA has an entire array of products, from Jetson to GB200. So… sure the 3090 isn’t a “bad” card for a benchmark but in that context, it is no flagship.
PS: taking the occasion to highlight that I do wish OP to actually go to China, work and live there. If that’s their true belief and they can do so, to not solely “admire” a political system from the outside, from the perspective of not participating to it, but rather give up on their citizenship and do move to China.
The propaganda aspect is import so I’m adding this to a reply rather than yet another edit.
This research is interesting. What the article tries to do isn’t clarifying the work rather than put a nation “first”. Other nations do that too. That’s not a good thing. We should celebrate research as a better understanding of our world, both natural and engineered. We should share what has been learned and built on top of each other.
Now when a nation, being China, or the US, or any other country, is saying they are “first” and “ahead” of anybody else, it’s to bolster nationalistic pride. It’s not to educate citizens on the topic. It’s important to be able to disentangle the two regardless of the source.
That’s WHY I’m being so finicky about facts in here. It’s not that I care about the topic particularly, rather it’s about the overall political process, not the science.
It appears you’ve missed the point here, which is that it turns out you can use older GPUs in creative ways to get a lot more out of them than people realized. Having latest chips isn’t the bottleneck people thought it was.
This article doesn’t talk about older gpus though? Its talking about using the V80 fpga from amd, which released in 2024 and costs 10k. Unless I’m misunderstanding something about the article? I do think its a good breakthrough being able to use an fpga like this though.
You’re right, the chip they leveraged isn’t actually that old. The key part is that we’re seeing a lot of optimizations happening in software space now that allows to use existing chips more efficiently.
turns out you can use older GPUs in creative ways to get a lot more out of them than people realized
If that’s the point then that’s the entire GPU used for mining then ML revolution, thanks to CUDA mostly, that already happened in 2010 so that’s even older, that’d 15 yeas ago.
What I was highlighting anyway is that it’s hard to trust an article where simple facts are wrong.
Imagine not being able to understand that new software optimization techniques are continuously being discovered. 🤦
Well, I honestly tried (cf history). You’re neither addressing my remark about the fact from the article nor the bigger picture. Waste of time, blocked.
