First, what is taste? Taste is the cultivated ability to notice, to understand, and to judge. It’s a sensibility to things — both tangible and intangible. Having taste doesn’t just mean being able to tell the good from the bad, or the beautiful from the ugly. More importantly, this capacity opens us to a fuller experience of life.

One popular but shaky argument for developing taste is that AI won’t have good taste, so taste is going to stay economically valuable. For example, AI won’t develop good visual taste because it can’t stop making ugly slop. Or AI won’t develop good product taste because it can’t experience human problems in real life and thus can’t empathize. But these are merely blind bets. What seems beyond AI today could become routine a year from now. In 2021, no one expected AI to be able to code like staff engineers. Neither did anyone expect AI to make Fields medal-worthy breakthroughs. It’s futile to give career advice by predicting what AI won’t be able to do.

Despite that, cultivating taste is still important. Taste is not just about making good judgments. It’s also how you experience life, and AI can never do that for you.

To develop taste is to learn and to appreciate. It’s the process of building closer connections to an increasingly alienating world. AI is becoming ever more capable. And the more we surrender to the machines, the more alienated we are. Taste is our fight against this inevitable alienation.

Taste gives us a fuller experience of life. Fuller doesn’t mean happier. It also means more suffering. A deeper connection to the world makes us more attentive to its darkness — the injustice, the prejudice, the pain in places and lives we once knew nothing about.

Here’s a personal example. The urban expansion in China is an impressive industrial feat. But learning about its history and how ordinary people were involved connected me to the skyscrapers and mega bridges on a deeper level. In the real estate gold rush, farmlands were taken from farmers with little compensation. Prices of farm products were artificially suppressed. Farmers were forced into low-paying jobs with poor labor protection. Knowing what they have been through — and that their future is as bleak as their past — makes me suffer. I suffer from knowing that they work hard but don’t get what they deserve, that they are everywhere yet invisible, and that I benefited from their deprivations. Taste leaves me unsettled whenever I see the city skyline. Understanding and empathizing with people behind these monumental facades connected me more closely to the world. It made me less alienated.

But an alienated life is a happier life. Gaining from other people’s labor without having to worry about the complications. AI agents running your life. Every question has the right solution readily available. Science is progressing at an unprecedented pace. Life expectancy is improving. Humans are colonizing Mars. Why choose more suffering? Paul Graham said there will be thinks and think-nots, but never explained why he chose to be among the thinks. Because rationally, it’s probably better to be among the think-nots. We choose to think for the same reason we choose to develop taste — to fully experience life.

Subscribe now

I’ve seen eight doctors total, in the US and China. I use the term “doctor” loosely: MDs, physical therapists, and chiropractors. Some gave me short-term relief while the rest were useless. I went to see one of the best sports medicine doctors in China. She called in the next patient before I’d been in her office for five minutes. Massage from some PTs and chiropractors helped, but the relief never lasted more than a day, and a foam roller at home was just as good. Almost every visit came with exercise assignments. But I never got the promised results from these.

My search for help could be summed up as cycles: First my muscles would flare up. Then I would seek help from a new doctor. After hundreds of dollars spent with no results, I would stop going. After enough cycles, I accepted that I might have to live with this pain for the rest of my life.

In late 2025, when my back flared up once again, I decided to try my luck with Claude. I didn’t have high hopes — how could AI fix something experienced medical professionals couldn’t? The initial answers were disappointing. Claude gave me the same advice as the doctors: posture correction, resistance-band exercises, breathing exercises, etc.

But I didn’t stop there. From using Claude at work for code and workflows, I knew a bad first answer didn’t mean it couldn’t do the job. Though Claude gave the same useless prescriptions as the doctors, the difference was I could tell Claude exactly what I felt (and didn’t feel) during these exercises — activated muscles, limited movements, pain, trigger points, etc. I also told it more about my history: when the pain started, how it spread, what I’d tried, and how none of it worked. Claude also gave me positional tests to pin down where the problem was. Finally, when I tried a thoracic rotation combined with deep breathing, I felt something I’d never felt before: relief in my upper back, next to my spine below my right shoulder blade. It was like a ball of muscle crammed tight for years finally letting go, and it was the first sign of hope in this prolonged struggle.

Claude never argued. It never cut off the conversation because our appointment time was up. It always tried its best to solve my problem. No human medical professional I could afford would listen to me talking, asking questions, and complaining for that long while keeping the same helpful attitude. I couldn’t have discovered what worked without someone willing to listen to my full history, to guide me through each exercise, and to try different approaches until we found one that worked. Having Claude available 24/7 was also a big factor. None of the doctors gave me a phone number. And even if one had, I doubt they’d have answered at midnight.

Claude is not just a good physical therapist but also a good teacher. It helped me understand my body better, which in turn made my exercises more effective. When Claude gave me exercise instructions, it referred to muscle groups, joints, and tendons I knew little about. But I was able to learn quickly with Claude’s help. My biggest limitation was overhead movement. For years I knew I lacked the normal strength and range of motion, but I never understood why. Claude taught me how my lower traps, rotator cuff, deltoids, and lat muscles each contributed to that movement. Now I can describe what I feel and understand not just what to do but also why.

Claude is also a multidisciplinary expert. It probably knows as much as most doctors in any specialty. Apart from my shoulder and back pains, I also mentioned my chronic sinusitis to Claude. Without my asking, Claude connected these two problems, which seemed completely unrelated. Claude said that because the sinusitis blocked the airways in my nose, I had adapted to mouth breathing. Mouth breathing uses the scalene (neck) muscles, while normal breathing uses the diaphragm muscle. This overworked the muscles in the front and the side of my neck while my diaphragm went dormant, which kept my ribs from expanding properly. Ribs that couldn’t expand plus the neck strain made my shoulder blades stop gliding smoothly — the culprit behind my back and shoulder pain.

Without Claude, I would never have known that I was contributing to my back and shoulder pain just by breathing incorrectly. I would never have thought of mentioning my sinusitis history during any visit to the orthopedic doctors or physical therapists. Adjusting my breathing pattern and re-learning proper diaphragmatic breathing loosened my ribcage, freed up my upper back, and eased my pain.

My body has been in a dysfunctional state for over 5 years. Undoing the damage is a long process, but I’ve made noticeable progress since I started late last year. My daily pain has dropped, I’m more active, and I understand how my body moves — whether I’m at my desk or in the gym. Most importantly, I’m hopeful. Claude helped me see a pain-free future: being outside doing stuff without having to find somewhere to lie down every few hours, lifting weights with proper muscle activation, and simply not having to worry about my body completely breaking down in my 30s.

Subscribe now

There is no unified robotics industry but two distinct ones. One produces over $5 trillion worth of goods per year1 — ranging from electric motors to frozen pizzas. The other is still in its infancy, producing only prototypes sold to researchers and tech enthusiasts, with zero real economic output.

The first one is the industrial robot industry. These are robots designed for specific tasks and require precise, step-by-step programming — like a robotic arm that welds body panels or an autonomous mobile robot (AMR) that ferries products across the warehouse.

Humanoid robots are the newcomers. As the name suggests, these robots are designed to function like humans. They don’t need detailed instructions on the precise movement for each step. Instead, the robots only need descriptions of an end goal. They can then plan the steps and act on their own.

Because people can now make machines that look, talk and think like themselves, humans are one step closer to being God. This triggers emotional responses that drive people’s attention to humanoid robots, while industrial robots, the machines that power our lives, barely get noticed.

But humanoid robots aren’t even capable enough for any valuable task yet. And even once the robots work, finding buyers for them is no easy task. Companies also have to scale production, navigate regulations, and build public trust. Each challenge can easily burn more than five years or even kill a company. In the meantime, industrial robots will keep improving while adding trillions of dollars to the economy each year.

Industrial robots

George Devol, a self-taught engineer from Connecticut, invented the first industrial robot in 1954. But his company, Unimation, only sold a few hundred units. It was Japanese firms that adopted the technology and went on to dominate the industry. Kawasaki first licensed Unimation’s technology in 1968. Other companies such as FANUC, Mitsubishi Electric and DENSO followed suit.

By 1990, Japan owned roughly half of the world’s industrial robots and produced over 70% of the world’s industrial robots each year. European robotics firms never matched Japanese dominance but still held steady market share, especially at home. Firms such as KUKA, ABB, and Comau build robots making everything from fuselages on Airbus planes to packaging for Nestlé coffee.

China started late but caught up quickly. China only began to effectively industrialize in the 1980s. Despite relying on low-skill labor early on, it quickly transitioned to world-class automated manufacturing. In 2013, China surpassed Japan in annual robot installations while still relying almost entirely on imports. But by the late 2010s, Chinese vendors like Estun Automation and Inovance Technology started taking shares from the incumbents. By 2024, Chinese robots accounted for 57% of units installed domestically — the first time domestic robotic firms took the majority market share. In the same year, China became a net exporter in industrial robots for the first time. While China was taking over industrial robotics, the rest of the global market started to contract. Demand in every other major industrial country has been falling since 2022. The growing demand in China counteracted contraction elsewhere, and has held global annual installation steady.

After over 50 years of continuous R&D and deployment expansion, industrial robots have become indispensable to manufacturing. More than 5 million units have been installed worldwide by the end of 2025.2 Amazon has over a million automated mobile robots, moving 75%+ of its packages. Robots handle over 90% of weld jobs and 85% of paint jobs for leading car brands like BMW and Mercedes-Benz. Electronics manufacturers buy more industrial robots than any other sector. In a smartphone factory, no human touches the main board during assembly. All the chips are mounted onto circuit boards by robots. Up the supply chain in semiconductor manufacturing, TSMC’s automated wafer-handling technology has reduced staff number by 95%. From 1993 to 2007, robotics drove 16% of overall productivity growth — roughly the same contribution railroads made in the 19th century.

Yet, industrial robots barely get any publicity. Meanwhile, humanoid robot companies appear in tech headlines weekly and their demos have reached billions on social media — without shipping a single deployment-ready product.

Humanoid robots

13,000 to 16,000 humanoid robots were sold globally in 2025. None has done anything economically productive. American companies, such as Figure and Boston Dynamics, have shipped less than 500 units total.3 Chinese companies produced the rest. A rough estimate tells us that out of the 14,000 robots made in China, 9000 went to universities and research institutions. 3000 went to data collection facilities, and 2000 went to stunts, shows, and exhibitions.4 Hobbyists also contributed to total sales, but the numbers were negligible. The humanoid robots’ use cases in universities, research labs, and data collection farms are straightforward. But you might wonder why over 1,000 robots were sold just for exhibition purposes. This phenomenon warrants an essay of its own — stemming from techno-chauvinism, rivalries between local governments, and poor taste built up by decades of cultural engineering.

Subscribe now

How good are the robots?

The humanoid robot industry made more progress in the early 2020s than the previous two decades combined. Previously, researchers spent decades trying to build robots that can walk without falling. By 2023, humanoid robots could walk, run, and manipulate objects with dexterous hands, doing things that no robots had even attempted just 5 years before.

But walking, running, or even backflips don’t make humanoid robots useful. Nor do folding clothes or mixing drinks. They need to be able to do an economically valuable task continuously, for at least hours, ideally days without requiring human intervention or making mistakes. They also need to meet strict safety requirements if they were to work closely with humans. Right now, humanoid robots are far from meeting these capability criteria.

Even the most capable humanoid robots today miss 1 out of 10 tasks under bounded, well-instructed conditions, and drop to 40-60% success rates when the task instructions get rephrased or the objects get rearranged. The Physical Intelligence Pi 0.7 model — currently the long-horizon state-of-the-art — hits roughly 90% on bounded multi-hour tasks like operating an espresso machine, but that’s the best public number, and most other models fall well short.

Many people have high hopes for humanoid robots to work in construction. What success rate must humanoids reach to be reliable for construction? A moderate one-story brick house takes around 10,000 bricks. To build no more than 1 defective house in every 10, the robots need to reach a success rate over 99.999% for each brick laid. This means failure rate needs to drop 10,000x from the current level.5

Another common vision is humanoid robots as domestic workers. What success rate do humanoids need to reach in this case? Again, let’s assume the robots make a mistake every ten days on average — not a high bar at all. If a robot completes 250 tasks per day, it needs a 99.958% success rate to make fewer than 1 mistake every 10 days. That’s over 200x lower than the current failure rate.6

Elder care is becoming a pressing issue in many countries. Some believe humanoid robots can help relieve this crisis. Because many tasks in elder care are crucial for a human’s health and safety, we’ll tighten the failure allowance to once a year. What does the single-task success rate need to be in order to make fewer than 1 mistake per year? Assuming the robot completes 75 tasks per day, its per-task success rate needs to reach 99.9996%. That’s almost 30,000x lower than the current state-of-the-art.7

Humanoid robots still have a long way to go. How fast will this improvement happen? The rapid improvement of large language models made many believe humanoid robots should follow a similar trajectory. However, the nature of LLMs’ rapid improvement makes that extremely unlikely. The scaling law drove most of LLMs’ improvement, and both LLMs and robotic models benefitted from the scaling law. But scaling robotic models hits a wall much faster than scaling language models. Scaling happens along two axes: data and model size. For language models, all written records can be used as training data - books, newspapers, everything on the internet, and much more. But no comparable data pool exists for robots. Every data point has to be intentionally generated through teleoperation or simulation, making data creation slow. Without enough data, researchers can’t train larger models. Even if there are eventually enough data to train bigger, smarter models, it won’t necessarily translate to better performance on real-life tasks. These tasks sometimes require split-second responses. A larger model thinks longer before acting — so even if a robot is smart enough to act correctly, it’s still worthless if it takes too long.

These are all solvable problems, but they will take longer than a lot of people expect. My rough estimate is that it will take at least 5 years before robots can do economically valuable tasks without requiring human intervention for 10+ hours. There’s no hard evidence to back this up, but I’m quite confident this won’t happen in the next 2 years. Robotics data scale linearly, which means capabilities will likely scale linearly as well, barring a paradigm change in model architecture.

If you build it, who will come?

Being able to complete valuable tasks is a necessary but not sufficient condition for widespread adoption of humanoids. Price also matters. More precisely, it’s each robot’s expected financial return divided by its price that matters.

Manufacturing adds more value per worker-hour than any other physical work, and humanoids can’t compete with industrial robots in this space. Humanoid robots are generalists. They have vision-action models capable of navigating complex terrains and fingered hands with 10+ degrees of freedom. They are designed to handle broad tasks under diverse settings. On the other hand, industrial robots work in fixed locations or routes, and have only a few degrees of freedom. They are designed specifically for tasks on assembly lines. Every piece of the machine is optimized for production speed and quality. Nothing is wasted on irrelevant tasks such as balancing and navigation. Also, modern manufacturing has been evolving around industrial robots. Factory layouts and product design are optimized for maximum output from the industrial robots. Last but not least, industrial robots’ simpler design means they are cheaper than humanoids.

In practice, the math is messier. Founders, investors, factory owners, and politicians can distort the market for profit and power. Humanoid robot makers want to sell to factories, so they can boast their deployment numbers and raise more money. Investors want the same thing for their portfolio robotics companies. Chinese local governments’ cadres also have deep vested interest in the robotics companies in their jurisdiction because they can potentially become national champions, make key technological breakthroughs, and produce substantial economic output, all accelerants for a CCP cadre’s career advancement. The mechanism in the US is different but the results are the same. Politicians are controlled by lobby groups and political donations instead of party hierarchies. Both governments will offer tax breaks, cheap credit, procurement contracts and more to help robot makers sell.

These incentives can make it seem like humanoids are displacing industrial robots in factories. But in reality these are merely schemes to prop up the humanoid robot companies. Recently, a Chinese tech publication claimed humanoid robots made by Agibot have been working in factory production lines for months, without mentioning the specs of the bots or what kind of work is being done. However, this picture from the original report gives us a hint. The so called “humanoid robot” at work has no legs — just two arms, with no dexterous hand, and a head. They basically took two robot arms, the most ubiquitous factory automation technology with millions of units already installed worldwide, put a plastic blob with an LED screen on top of the two arms as a “head”, and called it their deployment-ready humanoid product. Everyone plays along. This puff piece is the first step. New rounds of fundraising will ensue, followed by government subsidies. This is how bubbles form — by entrepreneurs, financiers, and politicians working together.

There is a bright side, though. One use case has the smallest capability gap and also the brightest commercialization outlook: selling to militaries.

When humanoid robots are evaluated for war, the criteria are almost completely opposite from those when they are evaluated for regular use cases. Consistency matters when the robots are building something useful, not when they are destroying. Safety matters when robots serve people, not when they are killing. The cost calculus is also different from civilian use cases. More robots on the front line means fewer casualties, which justifies a huge premium.

No winners in a fast takeoff

I could be completely wrong. It’s possible that within five years, humanoid robots become so good that everyone wants one. What happens then? Techno-optimists might call this utopia — everyone gets everything they want because of unlimited productivity. But in reality, the opposite can happen. A fast takeoff will benefit no one except the few equity holders in robotics companies and bureaucrats.

The US is unlikely to win in this “best case scenario” because China can produce orders of magnitude more robots. On the other hand, while China’s rich and powerful will become even better off in this boom, flooding China’s economy with highly capable robots is only going to exacerbate its economic challenges, and regular Chinese citizens will bear the weight of these problems. Theoretically, an open trade relationship could help both countries. But what happened during the past few years has made it clear that the US won’t buy Chinese humanoids due to national security concerns and China will be reluctant to sell this strategic dual-use technology to its archenemy.

The US excels at producing innovative ideas. However, for robotics, manufacturing matters more than ideas. No one can control ideas diffusing across borders, which can happen at the speed of light. On the other hand, manufacturing requires permits, materials, labor, power, water, and machinery. And scaling these takes years. In 2025, China made over 20x more humanoid robots than the US. If both China and the US expand production scale at full speed, this gap is only going to grow wider. First, China is much faster at building factories. From permitting to construction, China is faster at every step. Second, manufacturing labor is cheaper and more abundant in China - from line workers to tooling engineers to factory managers. Tim Cook once said “In the U.S. you could have a meeting of tooling engineers, and I’m not sure we could fill the room. In China you could fill multiple football fields.” Third, China dominates the global robotics supply chain. 63% of the global humanoid supply chain is in China, and Chinese components have a huge cost advantage. As an example, Tesla Optimus Gen 2 costs ~$46,000 with Chinese suppliers, and ~$131,000 without. China has so much control over crucial components such as rare earth magnets, batteries, and lidars that it’s almost impossible to build humanoid robots without touching the Chinese supply chain.

But China’s edge in production is hardly good news for its economy. China’s current challenges make the next few years the worst possible time for a robotics takeoff. China’s economy as a whole is the second largest in the world. But household consumption is only around 38% of Chinese GDP — half of the US’s. China has been growing its economy by relying heavily on infrastructure and real estate investment, and subsidizing exporters at the expense of domestic households. Low wages and poor working conditions have been troubling the Chinese working class, especially for physical laborers. An explosion of cheap robot workers will only make things worse for them. China faces an aging population crisis and humanoid robots can potentially offset the resulting labor shortage. However, a fast takeoff during the early 2030s is 15 years too early, as serious labor shortage in China is predicted to materialize at around 2045.

Conclusion

Millions of industrial robots are working in factories and warehouses, adding trillions of dollars to the economy each year. Humanoid robots are still years away from being useful, their commercial potential is uncertain, and even the best case scenario will only benefit a small group of elites. Given how much VC money went into American humanoid companies, and how much state-backed capital went into the Chinese counterparts, the humanoid robot industry will continue to make headlines. Valuations will keep rising, new models will keep breaking benchmark records, startups and investors will keep congratulating themselves. But will there ever be tens of millions of humanoid robots roaming the earth? That future might never come.

Subscribe now

Footnotes

The bottomless money pit syndrome is easy to diagnose. The tell-tale symptom is a product that takes years of research and billions in investment but shows no promise of making money. Almost all Chinese AI companies have this illness.

When patients arrive at the ER with a painful illness - appendicitis, a highway motorcycle crash, or anything in between - one of the first treatments they receive is a shot of opioid painkillers like morphine or fentanyl. It’s effective at blocking the pain, but does not cure the illness.

Same goes for the Chinese AI companies. A shot called OpenClaw temporarily numbs their pain but won’t cure the disease. And just like humans’ relationship with opioids, if they chase the OpenClaw fad for too long, these companies risk total destruction.

Chinese AI companies build the best open weight models. But they aren’t research institutes that contribute to open source just for the love of the game. They’ve poured huge investments into AI, hinged on the hope that it will make a hefty return. Sadly for them, that prospect is nowhere in sight.

Their consumer businesses are the biggest money pits. After tens of billions of dollars spent on infrastructure, over a billion on marketing, and hundreds of millions each month serving the models - how much do the Chinese AI companies make from their chatbot apps? Zero. Zero Chinese yuan. Zero dollars. Zero monopoly money. None of the top Chinese AI apps have a paid tier because Chinese consumers don’t pay for software, and no company has yet dared to challenge that for fear of wiping out its user base. So despite having over 100 million WAU, the Chinese AI chatbots’ revenue is literally zero.

These companies are doing slightly better on the enterprise-facing side. But their revenue is still orders of magnitude less than Anthropic and OpenAI’s. Sure, the Chinese models are open source and less capable, so they are expected to make less money. But a 30x1 difference is still too much. And Anthropic’s exponential revenue takeoff only exacerbates the Chinese AI companies’ woes.

Then OpenClaw went viral. But despite the unprecedented growth, its popularity was limited to hackers and tech enthusiasts who are used to fiddling with new AI tools. Most people don’t know and don’t care about using the command line interface, setting up a development environment, managing api keys, and writing access control rules. After all, most people aren’t comfortable with the instability and risk of a new, untested open source project like this, and rightfully so. OpenClaw is not for everyone.

However, leaders in the Chinese AI companies strongly disagree. They saw their API usage spike because OpenClaw’s early users were looking for cheap tokens, and they convinced themselves that this was a godsend - more people using OpenClaw means more tokens sold, which means more revenue for the company. So they concluded that they should mobilize all marketing resources, launch an epic campaign to convince the masses to install an open source npm tool called OpenClaw.

These companies were so convinced that OpenClaw would transform the business that they started doing absurd things. First, they flooded every tech/business media outlet with OpenClaw within a few days. Soon after, it got to billboards and subway advertising screens. Then, every level of state-owned media joined. This is bizarre because these media outlets have only one canon, which is to amplify the propaganda from the party’s central committee. The reporting is always provocative and chauvinistic. The recent recurring themes are the wicked Japanese Prime Minister, the dishonorable Taiwanese separatists, China’s blooming technology sector, and the corrupt government officials. So them writing about an open-source project on Github (which is banned in China), founded by someone who just joined OpenAI, and requires technical literacy 99% of people don’t have, is something straight out of a South Park episode.

Other marketing channels were even more farcical. Because people don’t have the technical know-how to install OpenClaw, companies like Tencent went old school. They set up tents on sidewalks, where people could walk in with their computers and get OpenClaw installed for them. It’s hard to fathom what was going through the mind of whoever organized this - what kind of genius comes up with the idea of scaling AGI by lining people up on sidewalk stands?

The companies’ OpenClaw marketing fiasco mirrors how people behave in crippling pain. Rationality is gone and they frenetically grasp on to anything that might relieve the pain. But the nature of these relievers means it won’t last. After these campaigns, these companies won’t be able to keep any of these users because OpenClaw is simply not a mass market product. Setting it up for someone at a service station is like a pilot bringing someone to ten thousand feet in the air in a Cessna, telling the passenger “now I’ve set up the flight for you, good luck!” and parachuting out of the cockpit.

That’s why OpenClaw is like opioids for the Chinese AI companies. It creates a lot of buzz in the short term. People get intrigued and set up OpenClaw. They will use it for a few days, find out it’s not what they imagined, and stop. API usage will grow as the marketing campaign pulls in new users - like how people stop feeling pain when they are on morphine drips. But as soon as the drug wears off, reality kicks in. The revenue growth will be gone, and the companies end up with nothing after wasting enormous amounts of time and energy. They will be further behind on model and product development because they’ve lost focus, and eventually succumb to their diseases.

Understanding the workload

What makes the business logic in voice AI complex? A 3-stage real-time voice AI workflow includes speech recognition, output synthesis, and speech generation. The output synthesis stage is where the complex business logic resides.

In this stage, an LLM takes in pre-defined contexts plus transcribed speech from the user, and synthesizes a response. To do so, it must juggle multiple context components, making the synthesis task complex. Because real-time voice AI apps are commonly built to replace humans in specific tasks, they require the following context components:

1. character prompt: what role is the AI playing, its name, background story, personality, etc.

2. task background: the circumstances of the conversation

3. user info: the user’s name, preferences, memories from previous conversations, etc.

4. task plan: step-by-step plan for this task

For example, a (simplified) context for a job interviewer AI agent looks like this:

1. character prompt: you are a seasoned web development engineer manager; your name is Jason; you’ve been at your current company for two and a half years

2. task background: you are interviewing a candidate for the junior software engineer opening on your team; the candidate needs to have strong understanding of HTML, JavaScript, and CSS; the candidate must have experience with React JS

3. user info: the interviewee is Sam; he graduated from UCLA last year; he interned at Google and Netflix; resume screening showed him as a very strong candidate

4. task plan: step 1: brief intro, ask about Sam’s background and recent projects he has worked on; step 2: probe his technical prowess by asking about JavaScript fundamentals, async operations, etc. ; step 3: give him a coding problem to test his ability to solve real world problems.

The naive approach

The most obvious and naive approach is to jam all four components into one blob - concatenating the multiple text bodies into one. Most builders choose this approach because it’s simple. However, it only works occasionally. This inconsistency makes it unsuitable for production.

The problem lies in how the LLM processes the monolithic context. In each turn, the attention mechanism generates each token sequentially. It does so by assigning an importance score to each token in the context window, which dictates how much each token affects the output. To execute the task plan correctly, the LLM needs to assign high importance to the current step of the plan and low importance to the rest. This requires the LLM to implicitly deduce the current step.

However, the available low-latency models can’t figure out the current step consistently. And these failures have severe consequences. When the AI gets the current step wrong, the conversation goes off course and renders the product unusable.

The FSM approach

Since figuring out the current step is so vital, how can we get more accurate and consistent results? The solution is the finite state machine (FSM) - an oracle that conveys exactly what the current step is. It not only helps the conversation LLM keep track of the steps, but also enables improvements in UI coordination and observability.

Instead of having to look up the current step from the full plan, the conversation LLM gets the current step’s content directly from the FSM. It no longer has access to non-current steps in the task plan, making it impossible to generate output using the wrong steps.

How does the FSM know what the correct step is? Instead of comparing the full conversation with the task plan, it tracks the state of the conversation using the step index. At each conversation turn, the FSM only needs to decide if the current step is finished. It then updates or keeps the conversation state accordingly.

To allow the FSM to judge if a step has been completed, each step in the task plan not only includes instructions for executing this step, but also has another component called success criteria. The instructions and success criteria are co-generated during task plan creation. At the end of each conversation turn, the FSM calls an LLM to judge if the current step is finished based on the success criteria and the conversation content.

What used to be one monolithic task has been split into two parallel subtasks. The FSM focuses on deciding state transitions. The conversation LLM focuses on synthesizing the output. Because there are multiple LLMs focusing on separate concerns, each LLM’s cognitive load is significantly lower than the monolithic approach.

There are also many side-benefits from having an explicit state for the LLM session. First, the state can be used for UI coordination. With FSM, each step can contain specifications for display elements as metadata. The user’s device then fetches the display elements when the session enters a new state, always ensuring the UI is synced with the conversation.

Having explicit states also leads to better observability. When the conversation LLM gives a problematic output, it’s crucial to know what part of the prompt caused that. Without the FSM, identifying the cause is mere guesswork. With the FSM, we can know that with absolute certainty. Then, we can make precise changes and iterate efficiently.

Using the FSM

Given these benefits, where can we use the FSM? It’s applicable to any voice AI use case that requires the AI to follow a step-by-step plan. It’s not useful for pure agentic workloads where an orchestrator decides the actions dynamically and there’s no pre-determined plan. Many voice AI use cases with substantial real-life value fall into the first category: An AI tutor can use an FSM to manage its lesson plan. An AI interviewer can use an FSM to manage the interview plan. AI practice partners - for lawyers to practice depositions, for doctors to practice diagnosis, etc. - can use FSMs to simulate realistic practice subjects.

The FSM does have skeptics, though. A common argument against it is “As models get smarter, real-time voice AI apps can achieve the same performance using a monolith architecture, without FSM. So building a complex FSM will be a waste of time”. This is inaccurate for two reasons. First, the FSM has benefits beyond just allowing the app to use less capable models. It also enables faster iteration during development and other product features such as UI coordination. Second, newer, smarter models can tackle tasks that previously required the FSM using just a monolith architecture. But the goalpost moves - using the smarter models with the FSM allows us to solve even harder tasks that were previously unattainable.

So an FSM is not a patchwork to compensate for current models’ lack of intelligence. It’s a sound architecture that will consistently help build better real-time voice AI products. It passes the most important test in the age of AI: Instead of being replaced by new versions of frontier models, FSM systems get better as models get more capable.

The DeepSeek Console

In early 2025, "DeepSeek consoles" suddenly became all the rage, especially among Chinese local governments and state-owned enterprises. Following DeepSeek's rise to stardom in late 2024, every company has its eyes set on deploying DeepSeek in its workplace like girls in west village trying to cop Trader Joe's tote bags. And very much like Trader Joe's tote bags for the NYC girlies, DeepSeek doesn't bring true value to most of these companies. It's a trend, a status symbol for leaders in these organizations to have the hottest buzzword tied to their names.

DeepSeek consoles are servers (mostly made by hardware firms unrelated to the AI labs) with open source LLMs like DeepSeek (and sometimes Qwen) pre-loaded. It's meant to be deployed on-prem (hosting the servers within their own data centers) to serve LLM queries. A console itself can cost anywhere between hundreds of thousands to over 2 million RMB (1 USD ≈ 7 RMB), with significant add-on costs such as software subscriptions and engineering support.

So why don't these companies & organizations just dump their files into a hosted vector database service and stick an AI chatbot on top of the data like most of the self-proclaimed "AI-enabled" companies in the world? Why spend millions of dollars on something that requires much more brainpower and capital to operate when there are much simpler and cheaper solutions?

This comes down to the weird tradition of obsessing over on-prem compute due to distrust of cloud service providers, and the widespread misconception that on-prem compute and data storage are always more secure than the cloud-based alternatives.

Another important factor is the political calculus. In China, many decisions in technological adoption are due to top-down political mandates. The foremost consideration when evaluating such decisions isn't whether the new technology is truly useful for these organizations. Instead, it's how much political capital can the business leaders and their colluding bureaucrats accumulate.

Moreover, Chinese companies and local governments generally prefer capital-intensive routes because there are more opportunities for grifting. In the case of AI deployment, every party - C-suite leaders, procurement managers, local government officials, hardware builders, GPU traffickers, etc. - is trying to siphon money from this operation, creating an incentive to spend as much as possible.

The DeepSeek console, however, is not a viable technical option for the vast majority of use cases. It's extremely hard to scale up and down, requires technical expertise that most organizations don't have, and doesn't actually have the perceived security and privacy advantages - just to name a few of its drawbacks. That's why when the grifters' party is over, what's left will be nothing but a pile of obsolete servers that have had no real utility, will continue to be useless, and will eventually be disposed for dimes on the dollar.

The Origin Story of the DeepSeek Console

The emergence of the DeepSeek consoles is best described as a combined result of historical market preferences, strategic national mandates, and profit-seeking grifters. We'll take a deep dive into the first two factors in this piece, while the last factor warrants an article of its own.

The cloud landscape is very different in China vs. in the US. While the Chinese government collects extensive data from its citizens, it supposedly takes protecting personal data very seriously (which is absolutely not true and virtually every Chinese citizen's data is for sale on Telegram.) Thus virtually the whole public sector is very slow to adopt cloud computing and storage, as it is somehow seen as less secure. Many state-owned enterprises and government branches have servers in their office buildings to store sensitive data.

This preference persisted into the adoption of AI. Many of the decision makers - from high-level technocrats to IT managers in companies - do not have a thorough understanding of the differences between the compute needs for AI vs. traditional software. Naturally, they've kept the tendency of wanting to deploy everything on-prem for AI.

Another reason why there's an appetite for the DeepSeek consoles is the Chinese government's budgetary structure. Unlike the US, where local governments collect their own taxes along with the federal government, local governments in China submit most of their tax revenue to the central government. The higher-level governments would then repay their lower-level subsidiaries via various forms of "transfer payments" based on budgetary needs. The majority of the transfer payments are "special-purpose transfer payments," which have to be spent on the specified uses (such as refurbishing run-down schools in rural areas) and are subjects to audits by higher-level governments.

AI has been listed as a national priority during the latest Chinese National People's Congress in March 2025. This indicates that AI spending is poised to constitute a larger share of the overall government budget, most likely in the form of special-purpose transfer payments to local governments. For the bureaucrats in local governments, a larger budget is without exception always better than a smaller one. Adding AI to the government's agenda via billed-by-usage cloud services surely won't make a good argument for a juicy budget increase from the higher-ups. Instead, "we'll need hundreds of millions of RMB in transfer payments to purchase DeepSeek consoles to equip our government workers with AI" is much more likely to actually bring in more money for the local governments.

Why the DeepSeek Console is a Bad Technical Choice

As we've seen, buying and deploying DeepSeek consoles is mostly a business and political decision instead of a technical one. When technical viability is the last thing considered for deploying a new technology, it's usually a sign of disaster - and that's exactly what's going to happen in this case. For most organizations looking to deploy DeepSeek consoles, they'll likely experience poor usability and low uptime due to a lack of technical expertise, bleed money from unforeseen costs, while still facing the same security vulnerabilities that were supposed to be prevented by on-prem deployment. Only in very few use cases, for those with truly specialized needs and that possess sufficient technical know-how, would warrant on-prem deployment of AI inference servers (DeepSeek consoles). And most buyers of DeepSeek consoles aren't in that category.

Pre-deployment

For many customers of the DeepSeek console, poor technical decisions begin when buying the machines. To be fair, it's very hard, if not impossible, to make the unequivocally correct decision on buying the right hardware and software stack. Top-of-the-line and entry-level consoles have price differences of over 20x, and the software platforms running on the consoles also vary greatly in price. Sure, they can run pilot programs and test a few different setups, but real-life scenarios with 500 people using the system are guaranteed to be very different from pilot programs with 5 people in the testing group. Scaling up the pilot programs linearly is likely infeasible, due to limitations in power supply, physical infrastructure, and money. So even if a pilot program goes well, the prospective clients are almost always making difficult compromises that would very likely lead to failure in real-life deployment.

Another huge hurdle for the successful deployment of DeepSeek consoles is customization, though this limiting factor also applies to cloud-based solutions. For an AI system to add value to a workforce, it has to thoroughly integrate into existing workflows and understand the contexts of the work. This means data has to be easily searchable and retrievable by the AI in order for the outputs to be helpful.

However, IT and data governance in governments and state-owned enterprises have historically been dreadful. Many computers still run on Windows 7 and are using obsolete versions of Microsoft Office, not to mention lots of the software is pirated. Almost all files are stored locally with no clear organizing structure, making it almost impossible to create an aggregated knowledge base. Workplace communications are often mixed with personal messaging on platforms like WeChat, making it impossible for LLMs to query. This historical oversight of standardized data storage and artifact building has created a huge roadblock for deploying customized AI solutions, making most of the models deployed on-prem via DeepSeek consoles no different from the publicly available versions.

In-use

The headache will only increase once the DeepSeek consoles are deployed. Keeping the servers online and ensuring the LLM inference services are consistently available are no easy tasks. They require a level of operational expertise that currently does not exist in almost all organizations seeking to deploy DeepSeek consoles, and it would take lots of money and time to build a team with such expertise. Hiring a team of two engineers to manage the DeepSeek stack can easily cost these companies and government entities north of 1.5m RMB (~200k USD) per year. Though significantly cheaper than hiring in the US, it's still a price most of these organizations aren't willing to pay. Not to mention two engineers are probably not enough for any decently large team. See a more detailed calculation on how much it costs to maintain on-prem LLM services here.

"But the hardware providers are offering on-call support," some might argue, "and there's no need to maintain an in-house engineering team to service the machines." While this is true, the true utility of these services is questionable. Many have chosen DeepSeek consoles over cloud-based solutions for the sole reason of data sovereignty. Relying on external support means allowing external access to data, thus giving up data sovereignty and contradicting the premise for buying this solution. Moreover, on-call support provided by the hardware manufacturers will always take longer to respond than in-house support, leading to more downtime.

A more crucial issue is with scaling. If DeepSeek consoles find success after the initial deployment, which, supposedly, is what both the sellers and the buyers want, they would want to expand LLM usage to reap more benefits from it. Scaling up LLM capacities for managed cloud-based solutions can be as simple as changing a number on a website. Even in the worst case, it won't take more than a day's work for an engineer. Scaling up on-prem solutions like DeepSeek consoles, on the other hand, is astronomically harder. From planning and building the physical infrastructures - including buildings, power supply, cooling, etc. - to installing the hardware and software, it could easily take months and cost millions.

As models and chips iterate, and they do iterate very fast, deployed DeepSeek consoles will soon be obsolete. So even if there's no desire to scale up the services, just maintaining it in order to benefit from the latest models is going to be expensive.

In short, the DeepSeek console is not an approach that's prepared for operational success. It gets exponentially harder to operate and maintain as adoption increases. As AI models become more capable and have more practical use cases, the sunk cost for DeepSeek consoles is going to surge, before most of the users eventually abandon this solution and incur significant financial losses.

Is It More Secure & Private?

The biggest supposed benefit of the DeepSeek console is the superior security and privacy it provides. "Running AI workloads on-prem means you're less vulnerable to attacks and data breaches." But is that really the case?

First, is deploying AI on-prem more secure than using managed cloud services? It really comes down to the quality of the user's in-house security team. If a company pays top-of-the-line salary to attract elite security engineers, then sure, running AI workloads on-prem is probably more secure than running it on the cloud. However, the reality is most often the other way around. Companies put huge emphasis on deploying the hardware for the bragging rights that come with it, and tend to conveniently ignore the seemingly low-ROI and low-prestige tasks such as security. This makes on-prem solutions often more vulnerable than cloud-based ones.

Even if the on-prem AI workloads are separated from the public internet, all it takes is an HR person clicking a link in her email or an intern using a USB stick that had been plugged into her personal laptop for malware to get in. So saying "on-prem AI is more secure" is truly a stretch, as it only applies to a few outliers who are willing to invest heavily in their security teams.

Another advantage DeepSeek consoles are supposed to possess is data privacy. It's believed that because all the data is stored on-prem, it will be safe from any unauthorized external access. However, an important premise must hold for that to be true - the organization must have its own maintenance and operating team.

As previously mentioned, maintaining and operating AI hardware like DeepSeek consoles require specialized talents, and not every company can afford to hire such talents. A popular approach is outsourcing this work to the hardware providers, where their technical teams are responsible for operating the hardware that was sold.

For the companies and government branches choosing this approach, the data privacy advantage is de facto nonexistent. Having to rely on external teams to maintain and operate the servers means high data access privileges must be granted to those outside the organization, which makes using DeepSeek consoles not that different from using cloud-based solutions, as there would always be a theoretical possibility that sensitive data is exposed to those who aren't supposed to.

Closing thoughts

The Chinese tech ecosystem is very different from its US counterpart. The interplay between government mandates and market forces often leads to decisions that are beneficial for all parties involved in the short term but are intrinsically unsound in the long run. In the case of DeepSeek consoles, enterprises chose this more capital-intensive method to deploy AI models instead of the simpler, more common cloud-based approach because that's what's most likely going to maximize personal gains for the leadership. Fundamentally, it's the grifting, both legal and illegal, that has led to the popularization of this unviable technology stack, which is likely going to cost its adopters billions in the future.

It feels like we get one of these announcements every other week now. But does it really make a difference for our day-to-day use? When is it going to come up with the perfect excuse to explain to my manager why I'm late for the third time this week? I still remember the leap from GPT-3.5 to GPT-4. How it went from outputting repetitive nonsense half of the time to all of a sudden becoming mostly useful was truly impressive.

Though models are still allegedly getting better all the time, there hasn't been such a "wow" moment after that big jump. These AIs are like having a mediocre kid who's getting B's and C's at school, doesn't get into major trouble, and gets along generally well with others - nothing to complain about. But deep down, every so often you still hope he surprises you with some sort of hidden talent and becomes a quantum physics genius or at least plays division 1 football. Similarly, the AIs are OK at almost everything. They can get a solid 8 out of 10 for almost every task, but you still can't stop hoping for a 10/10 perfect answer (that never comes).

So, are the models actually getting better, like the nice folks at the AI labs are claiming?

The answer is yes. But definitely not in a transformative way like the leap from GPT-3.5 to GPT-4. See, AI labs use benchmarks to figure out if an AI model is getting better, which are basically standardized tests for AIs. And as former exam-takers ourselves, we know these test scores have a few limitations, to say the least:

1. Someone doing well in one subject does not mean she would do well in other subjects.

2. More importantly, getting a perfect score on the exam doesn't always translate to crushing it in the real world.

The same principles also apply to exams for AI. Scoring higher on a coding benchmark has no relevance to how well a model can draft your work email. And even if the improvement is on a relevant benchmark - let's say the "email composition" benchmark - there's still no guarantee that it will be able to actually write a better email for your specific use case.

Now we see there's a disconnect between improvements on benchmarks and improvements in real life, let's take a closer look at the mechanisms that caused this disconnect.

Why choose these "irrelevant" benchmarks?

Frontier models like Gemini 2.5 , OpenAI O3, and Claude 4 all claim to be the "most intelligent model yet" in their announcements. To back up these claims, by far the most popular metrics used are math and coding benchmark scores.

Why math and code?

The first explanation is human bias. Or more specifically, human bias from the AI researchers. Most of them have math or computer science backgrounds. To them, doing well in math and coding competitions has always been a symbol of high intelligence. Using common sense: it's a bunch of math and computer people who work on AI models, of course it's important for them to make the models good at math and coding. Jesus created mankind in his own image, so when AI researchers have a chance to create intelligence, they decided to create it in their own images as well.

Of course, it's not just due to a bunch of computer scientists and mathematicians trying to create enhanced digital versions of themselves. It's also affected by the technical designs of the models (read more about model architecture here).

A tenet in AI research is that intelligence comes from scaling. It means without radical changes in the model's architecture and algorithmic design, a model's capability (perceived intelligence) can be improved by adding more training data (read more about model training here), using more computing power, or using a larger model with more parameters. To put it simply, more is better for training AI models. Right now, efforts on scaling AI models can generally be put into two categories:

1. The Imitation Method (Supervised Learning): Find more example input-output pairs. You show the AI, "Here’s a question, and here’s the perfect answer." Over and over. It's like learning to make pasta by having a Michelin-starred chef hand you a plate of perfectly cooked cacio e pepe. You taste it, you see it, you know exactly what the end-state is supposed to be, and your job is to replicate that. A lot of the basic language understanding and world knowledge in these models comes from this process.

2. The Cranky Italian Uncle Method (Reinforcement Learning (RL)): When you can't find more input-output example pairs, it's also possible to only provide the inputs, and use a separate system (can be a set of rules or another model) to verify if the output is correct. This is like when you don't have the perfect plate of pasta. Instead, you just have a very opinionated, middle-aged Italian gentleman from Naples standing over your shoulder. He won’t tell you how to make the pasta, but he will definitely let you know when you mess up. You learn by trial and error, guided by his feedback, and eventually you'll make a perfect plate of pasta that makes him stop yelling.

For a while, the first method was king. AI labs scoured the earth for more data - more of the chef-made pasta in the first example - but data is running out. They've used almost every book ever written plus the entire internet, and new data can't be created fast enough to meet the training needs. So, what to do when you can't get a hold of the perfect plate of pasta?

You have to turn to the cranky Italian uncle instead. After leaning heavily on the first method through 2023 and 2024, AI labs are increasingly looking to RL to squeeze out more intelligence.

In the RL approach, having a reliable "critic" - like how you can always trust an Italian uncle to tell if a pasta is good - is crucial. So, what kinds of problems are the easiest to get that instant, verifiable feedback for? You guessed it: coding and math. Does the code compile and produce the desired output? Do the numbers add up in the math equations? The answers can be easily checked by the computer, requiring no human in the loop, making it very scalable to improve on these tasks.

And that, in large part, is why so much of the progress you see in AI is happening in the realms of coding and mathematics. Unfortunately, it just so happens that these are also areas where improvements might not be immediately obvious to the average person just trying to get the AI to write a less boring email.

Benchmarks ≠ Real Life

To be fair, AI labs are also working on benchmarks designed to test how well these models handle everyday conversational tasks. But, even when the scores go up on these benchmarks, it doesn’t always mean your AI assistant is about to get noticeably less weird or more helpful. Here's why.

People think that because AI researchers have PhDs and work on world-changing technologies, their psyche must be quite different from the rest of us “normal” folks. However, the vast majority of them have the same virtues and flaws in their characters as everyone else. Many AI researchers are used to being the smartest person in the room, so it’s only natural for them to possess quite large egos regarding their intelligence.

Inevitably, quantifiable metrics like benchmark scores are sometimes turned into a form of dick-measuring contest for those outliers on the right side of the intelligence spectrum. Nailing that top spot on a prominent benchmark is a badge of honor - a proof that your genius is the most genius out of all geniuses. More pragmatically, it also makes for a fantastic resume. "Led the team to achieve state-of-the-art on the GLUE benchmark" looks pretty damn good when you're looking for the next cushy job or trying to raise funds for your startup. These ulterior incentives have created an obsession with benchmark scores, distracting the researchers from improving the model's usefulness in real life.

However, beyond the issues stemming from human nature, there are some technical reasons why a great benchmark score might not mean what you think it means.

The first problem is data contamination. Imagine you’re about to take a final exam. If your professor accidentally emailed you the exact exam questions and answers a week before, your perfect score wouldn’t exactly be a testament to your deep understanding of macroeconomics. Same goes for AI models. Often, the training dataset is not made public, so there's no way to know for certain whether benchmark questions are included in the training data. However, given the scale of training datasets (estimated to include over 10 trillion words/tokens), it's quite likely that some questions in benchmarks are already included in the training data. So if the AI has already "seen the answers" during training, the test scores won't tell you much anymore.

Then there’s overfitting. Going back to the exam analogy, let's say you're taking an exam on the NBA in the 2000s, and you know Professor Rivers, who is a huge Lakers fan, would be writing the exam. So you only studied the Lakers, from how Shaq spent his first paycheck to memorizing every cheerleader's name. In the end, you might ace the test, but would you actually say you have a good understanding of the NBA in the 2000s? Probably not. You've "overfitted" your knowledge to the Lakers.

AI models can do this too. They can be tuned to perform exceptionally well on the specific criteria of a particular benchmark. They learn what the benchmark "likes" and give it more of that. The result? A fantastic score on that benchmark, but potentially worse performance on everything else.

We saw this in the recent controversy with Meta’s Llama 4 models. It was believed that to attain a high rank on the LMArena benchmark (which uses human preferences to rate model outputs), Meta submitted a "customized model optimized for human preference" specifically for that benchmark. The implication being, this specially-tuned model is likely worse at other tasks precisely because it was so aggressively optimized for that one contest. This also goes to show how far companies are willing to go to chase that ranking on the leaderboard.

What This Means

AI models are getting better, more so in some areas than others. Knowing how and why they've been improving along the current trajectory, and the limitations of these proclaimed improvements, here are a few things you can do to make sure you're using the AI that best serves your needs.

Most importantly, benchmark scores are not real life - this should probably be carved into the walls of every executive suite in the AI era. When the next AI vendor comes calling, waving a shiny report proclaiming their customer service bot is "#1 on the Grand Unified Chatbot Eloquence Test" (or whatever impressive-sounding name they’ve cooked up), your first reaction should be to see that as an irrelevant vanity metric.

The crucial signal for whether an AI model is any good for you has to come from within your company, not a vendor's slide deck or a university lab's leaderboard. No one who's selling you an AI solution knows the particular challenges and unique needs of your business the way those in your company do. So, before getting on board with any AI solution, it's critical to test it thoroughly in an environment that's as close to the real production environment as possible. Only trust your own results.

Ideally, testing and collecting data on AI performance within your company should not be one-off. Instead, it's best to have a proper test-pilot-deploy feedback loop, where you're constantly gathering data on how well the AI is actually doing. This framework allows for continuous experimentation for potential improvements while maintaining stability for the majority of the organization.

To summarize, AI performance is not static. It varies wildly depending on the specific task and context you throw at it. So, before betting the house on the latest AI to work a miracle on your business, make sure to test it in an environment that closely mirrors the real use case. Only then can you make an informed decision on whether this specific AI solution will work for you.

What's In the Deal

First, a closer look at what was included in the announced deal. Most of the items are in fields such as manufacturing, energy, and defense - this includes the entire $1.2T Qatar deal, and the majority of the Saudi and UAE deals.

Both Saudi Arabia and the UAE have made plans to build AI infrastructure (data centers) as a part of their deals, but the sizes differ drastically. Saudi Arabia took a more conservative approach, aiming for a 500 mega-watt (MW, 1000MW = 1GW) datacenter in 5 years. They expect to build it in collaboration with Nvidia and AMD, the two leading AI chip providers.

The UAE, on the other hand, is aiming for the moon. It's building a 5GW AI campus near Abu Dhabi by 2030 - the largest AI cluster buildout plan so far.

To put these numbers in perspective, the largest AI data center in operation today is XAI's colossus cluster in Memphis. With 200,000 H100 GPUs, its estimated power usage is around 250 MW. By the end of this year, AWS will potentially take the throne with its Rainier cluster, consuming around 400 MW. The industry leader, OpenAI, held the largest AI cluster buildout plan before the UAE announcement. And they were only planning for a ~4.5 GW cluster, with an investment of up to $500 billion from Softbank. For reference, NYC's average power consumption is estimated to be 5.7 GW. So while Saudi's plan looks more than achievable by 2030, the UAE's plan to build an AI cluster 20 times the size of the current biggest cluster, and almost consumes as much energy as the entirety of NYC is... ambitious. But hey, they have Bugatti cop cars. If there's anyone who can build it, maybe it's them.

What's It For, Anyway?

Now, let's assume the incredibly cracked engineers from the US and Middle East are able to work together and get this 5GW data center built - then what? How will it be put to use?

A Primer on Training and Inference Data Centers

Generally speaking, AI data centers can serve two distinct functionalities: training and inference.

1. Using a data center for training means it is used to find the right weights/parameters for an AI model. I.e. It's used for research and development and doesn't interact with users directly.

2. Inference data centers serve apps like ChatGPT. When you send a chatbot a message, it's sent to an inference data center nearby to be processed.

While these two types of data centers use more or less the same kind of chip (GPUs), they are built very differently, favoring distinct characteristics.

The largest data centers are usually reserved for training. The GPUs for a training run need to be as physically close to one another as possible. Ideally all under the same roof. Because a model's parameters are constantly being updated during the training run, every GPU needs to maintain the same, latest version of the updated weights. Thus you want to house them as close as possible to minimize the latency during these updates. Theoretically, this information can be transmitted at a rate close to the speed of light via fiberoptic cables, so even if two chips are thousands of miles apart, each update should only take a few milliseconds. However, a large training run involves hundreds of millions (a conservative estimate) of gradient updates, which makes inter-GPU communication time a significant overhead.

The good news is, training data centers don't need to interact with users. So it doesn't matter where it is built. It can be in the middle-of-nowhere, as long as its chips are close together.

On the other hand, data centers for inference are almost the polar opposite. It needs to be close to the users so they can have a low-latency, smooth user experience. But since the parameter values are already set, each GPU server only needs to maintain a copy of the parameters and doesn't need to communicate with other GPUs. So a more distributed datacenter design with a small GPU cluster close to every population hub is ideal for inference tasks.

A helpful analogy would be to compare this to a manufacturer of, let's say, smartphones. Training data centers are like the factories. You want different factory lines located as closely as possible, ideally all in the same building so you can save on shipping costs. Inference data centers are like the stores. You want to have many stores in various cities, so consumers can get to your stores and buy your phones easily.

In general, the complexity and cost of building data centers scale hyper-linearly with their size, meaning that a data center twice as big would cost more than twice as much. This is due to challenges in areas such as networking, power supply, and cooling. Because of this challenge in scaling up data centers, almost all the biggest AI clusters are dedicated to training.

So which kind of data center will the UAE 5GW be? In the Commerce Department press release, it states:

The UAE-US AI campus will include 5GW of capacity for AI data centers in Abu Dhabi, providing a regional platform from which US hyperscalers will be able to offer latency-friendly services to nearly half of the global population living within 3,200 km (2,000 miles) of the UAE. Once completed, the facility will leverage nuclear, solar, and gas power to minimize carbon emissions and will also house a science park driving advancements in AI innovation.

It's not hard to deduce from this paragraph that the plan for this data center is to use it for inference. This is utterly perplexing. Remember we said earlier that the largest clusters with the most compute power are normally dedicated to training? How is it possible that the largest AI cluster announced to date is an inference cluster? It's completely unfathomable when it actually gets built one day, and all the hyperscalers are just like "That's a cool 5GW cluster right there, let me use it to deploy my shipped model to better serve the public in the Middle East."

A Problem for the US

While this definitely isn't "we didn't kill JFK" level of dishonesty, it is confusing and weird that the Trump administration claims this will be an inference cluster. Why would they do this? My best guess is that the Trump admin has yet to find a convincing rationale for justifying training state-of-the-art AI models on a cluster owned by the UAE. Given how the Trump admin and its inner circle have been harping on about maintaining the US's AI & tech dominance, it would be a bad look for them to announce that the US would rely on the UAE to train future frontier AI models.

So if building the most advanced AI cluster in the Middle East is misaligned with Trump's campaign talking points, why did they still go through with it? A likely explanation is that the US lacks confidence in its ability to build a comparable cluster domestically.

The main bottleneck would be power supply. Since 2024, many industry leaders have voiced their concern that the US could lose the lead in AI due to an inability to supply enough power to support state-of-the-art AI data centers. According to an Epoch AI estimate, it's unlikely for the US to build a >3 GW data center by 2030. The CEO of NextEra, the largest utility company in the United States, also considered it challenging to find a site for a 5 GW data center in the US.

The announcement of the latest 5GW datacenter in the UAE corroborates this point of view. Looking forward, AI dominance is arguably the top national security concern. If the US is reasonably confident with its ability to keep building the largest AI clusters domestically - in other words, if the administration had a choice - it would never even entertain the possibility of building facilities so critical to national security outside of US soil. The Trump administration must have recognized a real possibility that the US would fail to keep scaling up key AI infrastructure due to power constraints. If they don't build it somewhere else, the alternative would be to give China the lead. The US's closest allies - e.g. NATO countries - likely won't have enough industrial capacity either, making the UAE the next best option, as it remains a close ally despite not being fully aligned ideologically. Most importantly, it's power-rich.

Stated more simply, the 5GW data center in the UAE can be viewed as a contingency for if domestic plans like OpenAI's Stargate fail to materialize - a scenario the US government treats as a realistic possibility. The risk of losing the AI race to China is so big, that the US would rather allow a country like the UAE to control the critical infrastructure for training frontier AI models.

Closing Words

There is still very little detail about this 5 GW cluster, so most of the article is conjecture based on one paragraph in a press release. Despite this scarcity of information, the US choosing to build the largest AI data center in another country is saying a lot. This move suggests that the US may not be willing to bet everything on its domestic capabilities and is hedging against the possibility of slowed AI infrastructure buildout at home. While not putting all your eggs in one basket is often prudent, it can also be interpreted as a sign of wavering self-confidence.