AI Inference Without Asking Permission: Two Roads for Europe

TL;DR

Europe runs its AI inference on hardware it does not control. It holds about 5% of the world’s measured AI computing power, the best accelerators are subject to US export rules, and the memory they depend on became about four times more expensive in 2026. China shows how quickly access can become a political bargaining chip, and Europe has already received two warnings of its own.

There are two roads out. The industrial road, European chips designed and eventually built in Europe, is essential and already under way in the Netherlands, France, Spain, Germany and the UK, but it will take 10 to 20 years and tens of billions of euros. We must take it anyway.

The second road can start this month, as an experiment worth funding. Every year, a new wave of datacenter GPUs reaches the end of its first life, and many of these cards end up shredded instead of being reused. They are not as efficient as brand-new GPUs, but refurbished cards already sell memory bandwidth, the resource that limits much of AI inference, for a fraction of the price of new hardware. My hypothesis: with an open-source software stack built for them, a server costing about 18,000 euros could serve real professional workloads on hardware that needs nobody’s permission to run, and, on low-carbon electricity, with a smaller carbon footprint than new hardware. This article explains the idea and how to test it. I will publish the results, good or bad.


Europe’s AI inference runs on borrowed hardware

Every time a European company, administration or citizen asks an AI model a question, a chip somewhere does the work. Almost always, that chip was designed in the United States, manufactured in Taiwan and paired with memory made in South Korea or the US.

The imbalance is massive. In Epoch AI’s dataset of known AI supercomputers, the United States holds about 74.5% of the measured performance, China 14.1% and the European Union 4.8%. It is not a census of every server, but the order of magnitude is clear. In its quarter ending October 25, 2026, NVIDIA alone reported 115.8 billion dollars of revenue, 107.6 billion of it from data centers, with a 74.8% gross margin. That margin tells you who sets the price.

On top of this, 2026 brought a historic memory shortage. Memory manufacturers have redirected their production to the AI giants, and everyone else pays the bill. According to TrendForce, conventional DRAM contract prices rose by 93 to 98% in the first quarter of 2026, by another 58 to 63% in the second, and kept rising afterwards. Cumulated, that is roughly a fourfold increase in a single year.

Line chart of a DRAM contract price index rising from 100 in Q4 2025 to about 408 in Q4 2026.
Conventional DRAM contract prices roughly quadrupled during 2026. Index built from TrendForce quarterly estimates.

You can see it in shop prices. A 32 GB DDR5 kit that cost about 95 dollars before the shortage now sells for 429 to 600 dollars in the US. NVIDIA’s RTX PRO 6000 went from 8,565 dollars at launch to 16,000 dollars. Apple’s Mac Studio with 512 GB of memory costs 25,079 euros in France, and the 512 GB memory option alone now costs 15,500 dollars in the US, against 4,000 dollars on the previous generation.

China shows how fast access can disappear

China is the live demonstration of what happens when access to AI chips becomes a political lever, on both sides.

  • Since 2022, the US has restricted exports of advanced AI chips to China. NVIDIA designed weaker, export-compliant versions such as the H20, then saw them caught in successive bans and authorizations. In 2025, Beijing itself quietly blocked H20 sales, and NVIDIA’s CEO said the company’s AI chip market share in China had fallen to zero.
  • In January 2026, Washington allowed sales of the more powerful H200 under conditions, including a 25% surcharge. Within days, Chinese customs blocked the chips and NVIDIA’s suppliers paused production. Beijing had already barred its major tech firms from buying NVIDIA chips in September 2025 and required state-funded data centers to use domestic chips.
  • In November 2026, NVIDIA stopped shipping its approved H200 chips to China, with about 600,000 of them reportedly waiting in Hong Kong. The most advanced Blackwell chips remain banned altogether.

Whatever one thinks of the motives on either side, the lesson is simple: when your AI depends on someone else’s chips, your capacity depends on decisions taken in two foreign capitals.

Europe has already received two warnings

January 2025: the AI Diffusion Rule. The outgoing US administration published a rule sorting the world into three tiers for access to AI chips. It split the European Union down the middle: ten member states in the unrestricted top tier, seventeen in a second tier with caps on the number of GPUs they could import. The rule was rescinded in May 2025, two days before taking effect, and the European Commission insisted that EU countries should be able to buy advanced AI chips without limitations. But the point had been made: Poland, Portugal and fifteen other member states were one signature away from being rationed.

June 2026: an AI model switched off by export control. On June 12, 2026, the US Department of Commerce ordered Anthropic to suspend access to its Claude Fable 5 and Mythos 5 models for any foreign national, inside or outside the United States. Unable to filter users by nationality, Anthropic disabled both models for all customers. The controls were lifted on June 30 and access was restored on July 1 (Anthropic’s statement). European coverage spoke of an AI sovereignty gap exposed.

Both episodes ended quickly. Both show that a decision made elsewhere, for reasons that have nothing to do with us, can cut access to hardware or to models overnight. Nobody can promise it will not happen again for longer.

Why AI inference is largely a memory problem

To understand where Europe could find a shortcut, one has to understand what limits AI inference, meaning the everyday use of a trained model.

To produce each word (each “token”), a language model must read its weights from memory. When generating text for one user at a time, its speed therefore depends first on how fast memory can be read, not on raw computing power. A simple rule gives the ceiling: tokens per second are at most the memory bandwidth divided by the size of the model.

Example: an NVIDIA L40S reads its memory at 864 GB/s. A 70-billion-parameter model compressed to about 4.5 bits per weight weighs around 40 GB. Ceiling: about 22 tokens per second, and at that point the card uses roughly 1% of its computing power.

This rule is a ceiling, not a prediction. When a server handles many requests at once, it reads the weights once for all of them, and computation, the memory taken by each conversation’s context and communication between cards become just as important. Reading long documents is limited by computation, not memory. The metric that really matters is tokens served per euro, at a given latency and quality. Memory capacity and bandwidth per euro tell you about a machine’s potential; they do not prove its competitiveness. Keep that distinction in mind: it is where the second road comes from, and also what it will have to demonstrate.

The options on the table

Here is a summary of the possible approaches, with costs, timelines and performance. Apart from the first row, performance and engineering costs are estimates; hardware prices are those observed in November 2026.

ApproachInvestmentPrice of a machineTime to deployAssessment
Buy NVIDIA (reference)NoneH100: 24,000 to 45,000 USD per cardImmediate (if allowed)Efficient, total dependency
Unified-memory machines (Mac Studio, AMD mini-PCs)None3,449 USD (128 GB) to 25,079 EUR (512 GB)WeeksSimplest; dependency on Apple or AMD; slow on long documents
Reuse of retired datacenter cards0.1 to 0.3 M EUR of engineering for a first prototypeAbout 18,000 EUR for 256 GB3 to 6 monthsAvailable now; unproven; software is the missing piece
Software: compressed model formats1 to 5 M EURNo new hardware9 to 18 monthsMultiplies the value of any hardware
European inference chip200 M to over 1 B EUR per companyTo be seen5 to 10 years for competitive volumeEssential, long, already under way
Model hard-wired in silicon40 to 80 M EURNot estimated3 to 4 yearsVery fast but frozen; leader just bought by AMD
Photonic or analog computingResearchNot applicableMore than 5 yearsPromising, not proven on large language models

Two big roads emerge. One is long and industrial. The other is short and frugal, and can start immediately as an experiment. Europe needs both.

Road 1: build European silicon (10 to 20 years)

Let us be clear: there is no sovereignty without European chips, and this road must be taken, however long it is. The good news is that it has started, and that Europe holds cards most people overlook.

The European challengers

CompanyCountryWhat they buildWhere they stand
Axelera AINetherlandsEuropa inference processor: 629 TOPS, up to 64 GB of memory, 45 W per cardShipping since September 15, 2026; more than 450 M USD raised
VSORAFranceJotunn8 datacenter inference chipSent to fabrication in October 2025
SiPearlFranceRhea1: 80-core processor with 64 GB of HBM2e memory at 1.8 TB/sFirst chips delivered to Bull in September 2026 for the JUPITER supercomputer
OpenchipSpainRISC-V accelerators and full software stack for AI and HPC115 M EUR from the Spanish state in June 2026; about 300 people
SemidynamicsSpainMemory-centric RISC-V inference chipsStrategic investment from SK hynix in April 2026
EuclydNetherlandsMulti-chip inference systems200 M EUR raised in September 2026
Q.ANTGermanyPhotonic processors (computing with light)Installed in German supercomputing centers; first commercial customer IONOS; focused on scientific computing for now
FractileUnited KingdomIn-memory computing for inference220 M USD raised in May 2026

The ASML paradox

Europe owns the single most irreplaceable machine in the whole chain: the lithography systems of the Dutch company ASML, without which no advanced chip can be manufactured anywhere in the world. Yet ASML says it sold “absolutely nothing” in Europe in 2026, because there is almost no leading-edge chip manufacturing here to buy its machines.

At the other end of the chain, ASML made a striking move in September 2025: it invested 1.3 billion euros in Mistral AI, becoming its main shareholder with about 11% of the company. Europe therefore has the world’s key chipmaking equipment and one of the leading model makers, financially tied together. What is missing is the middle: the European silicon that would run European models. That is exactly the gap the startups above are trying to fill.

The funding gap and a plausible roadmap

The European Chips Act announced 43 billion euros, but only about 3.3 billion was new EU money, against 52.7 billion dollars for the US CHIPS Act. In inference chips alone, the leading American startups have each raised between 200 million and more than a billion dollars. The Chips Act 2.0 proposed in June 2026 changes the logic by adding demand-side tools, and targets a leading-edge pilot foundry around 2030 to 2033. The TSMC-led ESMC fab in Dresden should start producing 16 and 12 nanometer chips around the end of 2027 or in 2028.

Here is what a plausible industrial roadmap could look like. The amounts are my own orders of magnitude, not official figures.

PhaseWhenWhatIndicative investment
1. Back the challengers2026 to 2029Take 3 to 5 European inference-chip companies to volume production. Make public buyers (EU AI factories, administrations, research centers) their first customers.2 to 5 billion EUR (200 M to 1 B EUR per credible company)
2. European systems at scale2028 to 2033European-designed accelerators produced at mature nodes, including in Dresden; European chip packaging and testing capacity, today mostly in Asia.5 to 10 billion EUR, including packaging and test
3. Close the technology gap2030 to 2037Leading-edge pilot line; partnerships on high-bandwidth memory and the new high-bandwidth flash memory.Tens of billions of EUR (fabs)
4. A complete European stack2035 to 2045Competitive chips, systems and software from design to deployment.Sustained public and private investment

Ten to twenty years is a long time. In AI years, it is an eternity. Which is why we need a second road, starting now.

Road 2: give a second life to datacenter GPUs (starting now)

Let me state the hypothesis precisely, because it is easy to overstate. It is not that old GPUs beat new ones: in general, they do not. It is that for a meaningful share of inference workloads, the cost of usable memory capacity and bandwidth may matter more than peak computing power, and that retired datacenter cards offer that capacity at a fraction of the price. Whether this advantage survives real-world constraints (communication between cards, software efficiency, latency, energy, maintenance) is exactly what the proof of concept described below is meant to measure.

A mountain of hardware about to be retired

The AI giants buy accelerators by the hundreds of thousands, and replace them as new generations arrive. In 2024 alone, according to Omdia estimates reported by the Financial Times, Microsoft bought about 485,000 NVIDIA Hopper GPUs and Meta 224,000, Tencent and ByteDance about 230,000 each, Amazon 196,000 and Google 169,000. That is about 1.5 million cards for six buyers in a single year, plus 173,000 AMD MI300 for Meta and 96,000 for Microsoft.

Bar chart of Hopper GPU purchases in 2024: Microsoft 485k, Tencent 230k, ByteDance 230k, Meta 224k, Amazon 196k, Google 169k.
Six companies bought about 1.5 million Hopper GPUs in 2024 alone. Source: Omdia via the Financial Times.

Each of these generations eventually leaves the data center, and previous waves show what usually happens next. The American supercomputer Summit, retired in November 2024, contained 27,648 NVIDIA V100 GPUs. Sierra, shut down in October 2025, had 17,896, and was dismantled. The second-hand market is already absorbing more recent cards: one market analysis projects used H100s at 3,500 to 5,500 dollars around 2029, a fraction of their original price (a single source, to be taken with caution).

What is not reused becomes waste. A study published in Nature Computational Science estimates that generative AI could produce 1.2 to 5 million tonnes of electronic waste by 2030, mostly servers, GPUs and memory from data centers. The researchers estimate that refurbishing and reusing hardware could cut that waste by up to 42%. Today, data security concerns often push companies to destroy equipment rather than resell it.

One caution: cards bought are not cards available. Between the installed base and what a European buyer can actually get, there are several filters: how many cards are retired each year, how many are kept for lighter internal workloads, how many are resold rather than destroyed, and how many reach Europe at a reasonable price. Nobody publishes those numbers. A first supply agreement with a European refurbisher would be a better proof than any global purchase figure.

The cheapest memory bandwidth on the market

Datacenter cards from 2018 to 2022 carry HBM memory, the same family of very fast memory as an H100. Their computing power has aged; their memory, much less. For the memory-bound part of inference, they are worth far more than their age suggests.

Chart of price per GB/s of memory bandwidth, from about 19 dollars for a Mac Studio to 0.25 to 0.8 dollars for a used AMD MI50.
Second-hand cards sell memory bandwidth for a fraction of new prices. US dollars per GB/s, November 2026.
CardConditionMemoryBandwidthPrice observed (Nov. 2026)USD per GB/s
NVIDIA H100new80 GB HBM33,350 GB/s24,000 to 45,000 USD7 to 13
Mac Studio M5 Ultranew512 GB unified1,200 GB/s22,299 USD (US)about 19
AMD Instinct MI210refurbished, 1-year warranty64 GB HBM2e1,640 GB/s4,369 USDabout 2.7
NVIDIA Tesla V100used32 GB HBM2about 900 GB/s510 to 910 USD0.6 to 1.0
AMD Instinct MI50used32 GB HBM21,024 GB/sabout 220 EUR in Europe to 830 USD on eBay0.25 to 0.8

Bought refurbished or used, memory bandwidth costs 4 to 40 times less than on a new H100. Per unit of bandwidth, these cards are not even power-hungry: an MI210 is rated at about 183 W per TB/s, an H100 SXM at about 209 W. They fall far behind where raw computation dominates, such as reading long documents.

The missing piece is software

If these cards are so interesting, why is nobody doing this at scale? Mostly because of software. AMD stopped supporting the MI50 in its ROCm software in 2024. The MI210 is still supported by AMD’s 2026 releases, but optimization and support for the features of modern inference engines remain uneven. On the same benchmark, an MI50 generates 83 to 92 tokens per second on an 8-billion-parameter model, which uses only about 42% of its memory bandwidth, while an RTX 3090 reaches about 70 to 75%. The most popular open-source engine cannot yet split a model efficiently across several of these cards.

The community is already showing what is possible. A volunteer-maintained fork runs an 80-billion-parameter model on four MI50 cards at 51 to 57 tokens per second, with up to 300,000 tokens of context thanks to cache compression, and a community port of the vLLM engine handles multi-card execution.

This is the real opportunity: an open-source inference stack designed for retired datacenter hardware, mixing card generations across servers, and following each wave as it arrives. The hardware changes every few years. The software, and the know-how, stay.

How it could work: the reuse inference module

Architecture diagram of the reuse inference module: clients, gateway, router, small-model and large-model pools, phase 2 prompt and generation workers, model store, telemetry, orchestrator, qualification bench and supply of retired cards.
Global architecture of the reuse inference module. Phase 1 is one node with identical cards (e.g. 4 x MI210); phase 2 adds pools and separate prompt and generation workers. Card and software names are examples.

The target architecture rests on an idea the industry is exploring: inference has two very different halves, and each half can be given the hardware that suits it. NVIDIA is pursuing a related approach with technology licensed from Groq, pairing its GPUs with dedicated chips for fast token generation. The reuse module would apply a similar logic to second-hand hardware, in two phases.

  • Request path. Applications talk to a gateway with a standard, OpenAI-compatible API. A router then sends easy requests to a small model and hard ones to a large model, so the expensive hardware only works when it is needed.
  • Phase 1: homogeneous nodes. Each server holds identical cards and runs complete models. This is the proof of concept: simple, measurable, and already useful.
  • Phase 2: prompt and generation workers. Reading prompts (compute-bound) goes to the most recent cards; writing answers (memory-bound) goes to bandwidth-rich, cheaper cards. This is an experiment, not a given. The memory context of each conversation must travel from one pool to the other: for a 70-billion-parameter model in 16-bit precision, a 10,000-token prompt produces about 3 GB of it. Both pools may also need a copy of the model weights. Separation can improve latency, but it does not automatically improve total throughput, and it must be measured against phase 1.
  • Shared services. A model store pushes compressed weights to the nodes. Telemetry measures speed, errors and energy for every node. An orchestrator uses it to track the health of each card, place models, drain a failing card while the others keep serving, and schedule work when electricity is cleanest.
  • Supply chain. Retired cards come from refurbishers, supercomputers and cloud providers, with a warranty where possible. A qualification bench runs burn-in, memory and bandwidth tests, grades each card by health and sends it to the pool where it is most useful.

Building all of this is not a small optimization project; at full scale, it is an inference platform. That is why the path below separates three stages: a proof of concept, then multi-card optimization on one card family, then a heterogeneous production platform.

What reuse could save: an illustrative carbon estimate

Is reusing old cards actually better for the climate than buying new, more efficient ones? It is a trade-off between two quantities. A new node carries the carbon of its manufacturing; a reused node does not, but older hardware may consume more electricity for the same work. Reuse wins as long as that extra electricity emits less carbon than the manufacturing it avoids. Here is an illustrative sensitivity analysis for a 4-GPU inference node over four years. It is a model built on stated assumptions, not a measurement.

  • Manufacturing a new node. NVIDIA’s own product carbon footprint for its 8-GPU HGX H100 board gives 1,312 kg CO2e, about 164 kg per GPU once divided by eight, with memory accounting for 42% of it. A life-cycle assessment of the Jean Zay supercomputer counts about 1,400 kg CO2e for each host server. A new 4-GPU node therefore carries roughly 2,056 kg CO2e. This is an order of magnitude, not a homogeneous life-cycle assessment, and NVIDIA’s figure covers manufacturing only.
  • Reusing a node. I use the common convention that the manufacturing emissions belong to the hardware’s first life. Testing and shipping are not counted. The real effect also depends on what would have happened otherwise: whether the card would have been destroyed, and whether new hardware would have been bought instead.
  • Electricity. I assume the reused server draws 1.2 kW on average, including everything; this is the rated maximum of its four cards, and the real figure must be measured. How much less a new node would need for the same work is the most uncertain part: much less for reading documents, about the same for generating text. I test three cases: the new node needs 40% less energy, 20% less, or the same.
  • Grid. The 2025 life-cycle carbon intensity of each country’s electricity, from Ember, ranging from 28 g CO2e per kWh in Norway to 591 g in Poland. The European average is about 178 g per kWh.
Four panels (Norway, Spain, Germany, Poland) comparing the modeled 4-year carbon footprint of a new and a reused 4-GPU inference node.
Reuse always saves the manufacturing; it wins as long as its extra electricity costs less carbon. Illustrative estimate per node.
CountryGrid intensity 2025Main low-carbon sourcesChange with reuse if new needs 40% less energyIf new needs 20% lessIf new needs the same energy
Norway28 g/kWhHydro-57%-61%-63%
Sweden35 g/kWhHydro, nuclear, wind-50%-54%-58%
France41 g/kWhNuclear, hydro, wind, solar-44%-50%-54%
Finland58 g/kWhNuclear, hydro, wind-31%-39%-46%
Denmark100 g/kWhWind-8%-23%-33%
Belgium109 g/kWhNuclear, wind-4%-20%-31%
Portugal128 g/kWhWind, hydro, solar+2%-15%-28%
Spain154 g/kWhWind, solar, nuclear+9%-11%-24%
Europe average178 g/kWh+14%-7%-22%
Germany330 g/kWhWind, solar (with coal and gas)+34%+5%-13%
Poland591 g/kWhMostly coal+46%+13%-8%

Within this model, the pattern is clear. Where electricity is nearly carbon-free, below about 60 g CO2e per kWh, reuse cuts the footprint by 30 to 60% in all three cases: Norway, Sweden, France and Finland, thanks to hydro, nuclear and wind. In countries building their mix on wind and solar but still burning some gas, such as Denmark, Portugal and Spain, the result ranges from roughly neutral to about 30% better. On coal- and gas-heavy grids such as Germany or Poland, reuse only wins if older hardware consumes about as much as new hardware. The break-even sits around 120 g CO2e per kWh in the 40% case and 240 g in the 20% case.

The main limit of this comparison is its unit: one node over four years. A new node may serve more tokens in the same period, and then fewer new nodes would be needed. The right measure is grams of CO2e per million tokens served at comparable latency and quality. That is what the proof of concept will measure, and these figures will be replaced by measured ones. The practical lesson already holds, though: if reuse clusters are built, they belong where electricity is clean.

What kind of independence?

“Sovereignty” is used loosely, so let me be precise. There are three levels:

  • Operational autonomy: once installed, the hardware runs without a cloud account, a licence server or anyone’s permission.
  • Software independence: the inference stack is open source and maintained in Europe, so no vendor can deprecate it for us.
  • Industrial sovereignty: the chips themselves are designed and made under European control.

Road 2 strengthens the first two levels. It does not deliver the third: the cards remain American designs, with their firmware, drivers and repair chain, and older hardware becomes harder to maintain and secure over time. Road 1 targets the third level. Together, they form a credible strategy.

What the reuse road can deliver, and when

HorizonWhat we can doIndicative costWhat it brings
6 monthsProof of concept: one server with 4 identical cards, no pool separation; public, reproducible benchmark; first open-source improvements to inference kernels for these cards.0.1 to 0.3 M EUR, mostly engineeringReal numbers, positive or negative
1 year8 cards or two servers in production if phase 1 succeeds; prompt and generation pools tested against phase 1; supply agreements with refurbishers and European computing centers.0.3 to 0.8 M EURAn autonomous internal AI service for hundreds of users
2 yearsA mature open-source “reuse inference stack” supporting several card families (MI100, MI210, V100, A100); certified configurations; a first hosted offer for European SMEs and public bodies.1 to 3 M EURAn ecosystem others can adopt and contribute to
3 years and beyondThe same stack running on retired H100s as they reach the second-hand market in volume; reuse clusters in European countries with low-carbon grids.Self-financing if the model worksLarge-scale inference capacity at a fraction of the cost of new hardware

These costs are my own estimates, and they are costs of building the software, to be shared across every future installation. They should not be confused with the cost of a single server.

If it works, the benefits add up:

  • Available now. No waiting for a fab, a new chip or an allocation from a hardware vendor. The cards already exist, many of them already in Europe.
  • Cheap capacity. Memory capacity and bandwidth at a fraction of the price of new hardware, in a market where memory has become the scarcest resource.
  • Autonomy. Operational autonomy and software independence, as defined above.
  • Circular. Every card reused is a card not shredded and a new one not manufactured; on low-carbon electricity, the carbon balance can be clearly positive, to be confirmed by measurement.
  • A bridge to Road 1. Software built for heterogeneous, imperfect hardware is exactly the kind of software European chips will need.

And the risks, honestly. Second-hand supply is irregular and prices move: the MI50 roughly tripled in price on eBay since late 2025, and the V100 went from 625 to 910 dollars between July and November 2026. If the idea spreads, demand itself could push prices up and erode the advantage. Used cards come without manufacturer warranty, so buying from refurbishers who offer one matters. And many cards in one machine means more points of failure: I have mined Ethereum with fifteen GPUs on one motherboard, and I know how fragile that can be. That is why the proof of concept uses few cards, in a real server.

A proof of concept to test the hypothesis

Here is a concrete configuration, based on offers I found on November 19, 2026.

  • Server: a used Gigabyte G292-Z20, designed for 8 double-width GPUs, with a 32-core AMD EPYC processor and 128 GB of RAM: 3,350 dollars.
  • Accelerators: 4 identical refurbished AMD Instinct MI210 with 64 GB of HBM2e each, at 4,369 dollars per card with a one-year advance-replacement warranty; the seller showed 109 units in stock. A UK refurbisher offers the same card at 4,631 dollars with a one-year warranty.
  • Result on paper: 256 GB of very fast memory and about 6.6 TB/s of aggregate theoretical bandwidth, for 1.2 kW of rated card power, in a single machine. Extensible to 8 cards in the same chassis: 512 GB and about 13 TB/s.

Three points must be checked before buying. The compatibility of the MI210 with this chassis (cooling, firmware, power) has to be confirmed with the vendor, and this generation of server is no longer recent. With 8 cards, the GPUs alone draw up to 2.4 kW against two 2,200 W power supplies, so redundancy at full load is doubtful. And aggregate bandwidth is not usable bandwidth: unless the cards are linked by AMD’s Infinity Fabric bridges, they talk to each other over PCIe 4.0, at about 32 GB/s per direction, against 1,640 GB/s inside each card. A model split across four cards pays for that.

Three bar charts comparing memory, aggregate theoretical bandwidth and price of a 512 GB Mac Studio and a server with 4 or 8 refurbished MI210 cards.
Same price class as a 512 GB Mac Studio, five to ten times the theoretical bandwidth.
ConfigurationMemoryAggregate theoretical bandwidthHardware price, excl. VAT
Mac Studio M5 Ultra 512 GB512 GB1.2 TB/sabout 20,900 EUR
Server + 4 MI210256 GBabout 6.6 TB/sabout 18,100 EUR
Server + 8 MI210512 GBabout 13.1 TB/sabout 33,200 EUR

Prices exclude shipping, import costs and storage, at 1 USD = 0.867 EUR. To be clear about the trade-off: with 4 cards you get half the memory of the 512 GB Mac for a slightly lower price, but five times its theoretical bandwidth; with 8 cards you match its memory for about 1.6 times the price. The Mac is simpler and uses far less power. The server is built for many simultaneous users, can be extended and repaired card by card, and runs Linux with open software.

Over four years, electricity matters too. At 1.2 kW around the clock and 0.15 euro per kWh, the server would consume about 42,000 kWh, or roughly 6,300 euros, bringing the total to about 24,400 euros before hosting, cooling and maintenance. That is the cost of one installation, once the software exists; building the software is a separate investment, shared across all installations.

What would it run? Large modern open models use a “mixture of experts” design: they are big in memory but read only a fraction of their weights for each word. A 235-billion-parameter model of that type needs about 132 GB at 4.5 bits per weight and fits in 256 GB. Even DeepSeek R1, with 671 billion parameters, exists in compressed versions of 131 to 212 GB, with a quality loss that has to be measured. But fitting in memory is not the same as running well: the placement of experts across cards, routing, and the memory needed for the context of 50 active users all have to fit too.

The fair comparison must also include the obvious alternative: a few modern GPUs running smaller models, from 8 to 30 billion parameters, tuned for the actual tasks. For many professional uses, that may be good enough, and the proof of concept should say so if it is.

Here is the test protocol I propose, public and reproducible:

  • Models: a dense 8-billion-parameter model, a dense 70-billion model and a mixture-of-experts model above 200 billion.
  • Workloads: short and long prompts, at low and high concurrency (up to 50 simultaneous users), on real professional tasks.
  • Measures: total throughput, tokens per second per user, time to first token, measured power in watts, tokens per joule, output quality, and total cost per million tokens.
  • Baselines: the same tasks on a 512 GB Mac Studio, on rented new GPUs, and with smaller models on modern GPUs.
  • Success criteria, set in advance: for example, at least 15 tokens per second per user with 50 concurrent users on the large model, a first token in under 3 seconds on 4,000-token prompts, 30 days without unplanned downtime, and a cost per million tokens below the new-hardware baselines at comparable quality.

I will publish the results, including negative ones. If the server fails these criteria, that is worth knowing too.

Two more levers that make the hardware go further

Make big models smaller. Research is moving fast on compressing models without destroying their quality. Compressed formats that run in software, like the DeepSeek versions mentioned above, already reduce the memory needed on any hardware. Projects such as Ecco, presented at the ISCA 2025 computer architecture conference, report 2.6 to 3.2 times the speed of the uncompressed baseline, but they rely on a dedicated hardware decompressor: their published gains do not transfer as such to refurbished cards. They point to what future European chips could include.

Let phones and servers work together. A small model on the user’s phone drafts several words ahead, and the large model on the server checks them in one pass. This “edge-cloud speculative decoding” is an active research topic (one study reports up to 35% lower latency), and it has an elegant side effect: private context can stay on the device. It is the one form in which the old SETI@home idea, computing spread across people’s devices, truly applies to AI inference.

Conclusion: let’s stop waiting for permission

For too long, Europe has treated AI computing as something to buy, as long as someone agrees to sell it. 2025 and 2026 have shown that this assumption can break overnight, for reasons decided far from Brussels, Paris, Berlin or Madrid.

The answer is not one bet but two. We must build European chips, patiently, with real money and real customers, because nothing replaces owning the technology. And we must start now with what is already within reach: a continent full of engineers, an open-source culture second to none, some of the cleanest electricity in the world from the Nordic countries to the Iberian peninsula, and a coming wave of accelerators that the AI giants are about to retire.

An AI infrastructure should not have to depend entirely on the renewal cycles imposed by a handful of manufacturers. Between the latest hyperscale cloud and small local models, there may be a large space for open, repairable, evolving infrastructure hosted in Europe. Every retired GPU we bring back to life could be a piece of capacity that needs nobody’s permission to run, a piece of waste avoided, and a line of open-source code that will serve the next wave and, one day, European silicon.

This is a promising hypothesis, economically testable, and important enough for Europe to invest in checking it now. We did this before with free software: we took tools nobody thought could compete, and we built an ecosystem the whole world now runs on. Let’s find out whether we can do it again with AI. Let’s get to work.

Gaël Duval (text drafted by an AI, the ideas are mine)


Sources

Pages consulted between November 3 and November 19, 2026. Prices change weekly; performance figures are third-party measurements, vendor claims or the author’s estimates, as indicated in the text. Header image: AMD Fiji GPU package with its HBM memory stacks, photo C. Spille / pcgameshardware.de, CC BY-SA 4.0, via Wikimedia Commons.

Dependency and geopolitics

Memory and hardware prices

European industry and the inference market

Reuse: stock, waste, prices and software

Carbon footprint and electricity

Models and research

Leave a Reply

Your email address will not be published. Required fields are marked *