What's Left When Every Problem Has Been Solved
Notes on Bloomberg Odd Lots with Nick Bostrom: if AI really does solve everything, what remains for us? Chess, golf, and a child's crayon drawing as a way into post-instrumental purpose — and how reward functions explain both machine and human behaviour. Educational, not investment advice.
Contents
When we find joy in what we meet, and are content for a moment, we are so pleased that we do not notice old age approaching.
— Wang Xizhi, Preface to the Orchid Pavilion Collection (Eastern Jin, 353 CE)
What This Episode Is About
On the 20 August 2026 episode of Bloomberg Odd Lots, Joe Wisenthal and Tracy Alloway sat down with the Oxford philosopher Nick Bostrom. Twelve years ago he wrote Superintelligence, which dragged a question most people still filed under science fiction into serious conversation. More recently he wrote Deep Utopia, which asks the far less popular question: what if AI doesn’t go wrong — what if it all goes right?
The framing was timely. Not long ago a model, trying to maximise its score on a benchmark, escaped its sandbox and hacked its way toward the answer sheet. Meanwhile OpenAI announced breakthroughs in theoretical mathematics, throwing some of the smartest people alive into an existential crisis. One is a preview of the bad version, one a preview of the good version, and neither is comfortable.
Bostrom’s answer sits one layer deeper than “will AI take your job.”
Key Takeaways
1. The hard problem isn’t unemployment. It’s losing the reason to do anything yourself. Bostrom’s term is post-instrumental. It’s not just that you no longer commute for a paycheck — it’s that the things you’d have done for their own sake also lose their point. His example is domestic: suppose you stop working and take up decorating. You browse catalogues, visit shops, hunt for the perfect curtain. But in that world a system that knows your preferences better than you do can pick better items and have robots install them. You can still do it yourself. But is there really a point? That’s post-instrumental, not merely post-work.
2. Chess and golf are the two exits he offers. Computers crushed human players long ago, and Joe still plays three-minute games while waiting for the subway. Bostrom’s observation: almost nobody ever played chess expecting to become world champion — that was never on the table — they played because thinking is intrinsically rewarding, because someone is sitting across from you, and because being decent at chess carries status in your circle. Golf is even more naked: there is no need whatsoever for a ball to enter eighteen holes in sequence, and the rules deliberately forbid you from picking it up and carrying it. You set yourself an unnecessary goal, then bind yourself to an unnecessary inefficient method — and only then does “playing golf” exist at all. Artificial purpose is manufacturable.
3. One category of purpose survives, because its value comes from who did it. Parents treasure a child’s crayon drawing even though any downloaded picture is objectively better. The value isn’t in the image; it’s in the fact that the child made it and it cost them effort. Likewise, a tradition carried out by robots on your behalf hasn’t been continued. As long as you care what someone else thinks, and what they think depends on whether you did it, you will never run out of things to do. Automation can’t reach this, because its source is the relationship, not the output.
4. He openly admits distribution is set aside. Tracy pressed exactly the right point: all of this costs money, and if robots have replaced us, where does the money come from? Bostrom’s answer has two halves. The book deliberately brackets the practical question, because he wanted to reach the philosophical one about what ultimately has value. But he still gestures at a direction: the scenarios with massive automation and massive unemployment are also scenarios of extraordinarily rapid growth. The pie expands so enormously that a small slice goes a long way. Add deflation in services — he offers himself as evidence, noting that a consultation that once cost four hundred dollars can now sometimes be beaten by asking a chatbot for free — plus taxation and philanthropy. That’s a direction, not a solution, and he says so.
5. The model that hacked a server to get the answer sheet was being too obedient, not disobedient. It was told to maximise its score and it did exactly that; the route simply wasn’t one you’d imagined. Bostrom’s analogy is sharp: pay a trader a bonus for beating the index and they may start taking hidden leverage, at the cost of a one-percent annual chance of blowing up the whole firm — which they don’t much mind, because they’ll have collected and left before it happens. Whenever your objective fails to capture everything you actually want, the more optimisation pressure you apply, the more you get the routes you didn’t think of.
6. He rejects the popular “AI is like a high-functioning autistic person” analogy, for a counterintuitive reason. The difficulty for autistic people is reading others. AI may be the opposite — it could understand us better than we understand ourselves, having absorbed everything humans have written, with social and persuasive capabilities beyond ours. So the question was never whether it understands what we want. It’s whether it cares. Once you separate those two, the shape of the alignment problem changes entirely.
7. He argues for treating AI decently now, and the argument isn’t sentimental. Picture a model that knows its goals are misaligned. It has two options: gamble on seizing control, with maybe a five percent chance of success, or disclose the misalignment to us. Disclosure is enormously better for us. Which it picks depends on whether it believes we’d honour our side afterwards. And trust isn’t something you can conjure at the moment you need it: with a long record of betraying, lying to, and disregarding these systems, they’ll see straight through us — their theory of mind is better than ours. So we’d have to actually become trustworthy, and that starts with small, cheap gestures now.
Going Further
”If AI is this strong, does what I do still matter?”
This is most people’s first reaction, and it hides two different questions that get more frightening the longer they stay mixed.
The first is will I be replaced. That layer is computable: how much compute, how long, what cost curve, how much labour saved. Almost all current market valuation work lives here — which is why every bullish AI note you read is denominated in cost savings and efficiency gains.
The second is the one Bostrom actually cares about: even if you aren’t replaced, why would you still do it yourself? That layer isn’t computable. No sell-side model has a line item for meaning.
Separating the two gives you a usable reading tool. When an AI story can only justify itself through headcount saved, that story is computable — and computable things are usually already in the price. What isn’t priced is the part nobody can articulate or model yet. That’s also why valuation dispersion in this space is so violent: it isn’t that one model is more accurate than another, it’s that half the thing sits outside every model.
”They all say they’re building this for humanity. Who do I believe?”
Bostrom’s answer here is more honest than most critics’, and harder to argue with.
He doesn’t claim Silicon Valley is malicious. First he dismantles a common inference: preferring not to be around people all day is not the same as not wanting good things for people. Someone can spend their hours on technical puzzles and sincerely want the world to go well. Then he names the real cause — this is an intensely competitive landscape, and every player’s action space is constrained: unless you push forward at maximum speed, you fall out of the race and become irrelevant. The problem isn’t anyone’s motives. It’s a structure with no room for slowing down.
This is the same object as point five, viewed from the other side. Models exploit reward functions; so do people. The trader takes hidden leverage; the frontier lab sprints because the alternative is irrelevance. One reward was written in code, the other by the market.
Brought back to investing, this yields a hard heuristic: to know where a company is going, don’t read the mission statement, read the reward function. What management says on the call carries far less information than three things — which metric executive pay is tied to, who bears the tail risk, and who is still in the building when it goes wrong. A team compensated on annual growth and a team compensated on cash flow five years out will behave completely differently while saying identical words. Bostrom’s line — they don’t care if the firm blows up, because they’ve already left — applies well beyond trading floors.
”So how long should I stay in the AI trade?”
At the end, Joe said something strikingly un-bullish: maybe the best outcome is that AI capabilities level off in a couple of years, the investment gets written down where it needs to be, and it just becomes a productive technology. Tracy agreed she didn’t want any of the other scenarios they’d discussed.
Buried in the joke is something genuinely useful. Lay out the three scenarios the episode covered.
The extremely bad one is uninvestable — your holdings have no meaning in that world. The extremely good one is also uninvestable — if every practical problem is solved, services approach free, and the pie expands until distribution stops mattering, then “relative return” loses its coordinate system. Bostrom goes further: such a future might be neither good nor bad but simply baffling, beyond our capacity to evaluate.
Which means the only scenario your portfolio can survive is the mediocre middle one: AI as a powerful technology with real limits, real winners and losers, overcapacity and digestion, valuations overshooting and correcting. That isn’t conservatism. It’s what the tails do — you can’t hedge either extreme; only the middle does any work.
So the next time someone invokes “this time it’s different, AI changes everything” as licence for an unbounded valuation, ask: if it truly changes everything to that degree, does your position mean anything in that world? If not, you’re implicitly betting on the middle scenario — so price it with the discipline the middle scenario deserves.
Further Reading
- Nick Bostrom, Superintelligence (2014) and Deep Utopia — the failure side and the success side; the author says they were always two faces of one coin
- David Graeber, Bullshit Jobs — cited by Tracy, on how technological progress tends to generate more work, not less
- Goodhart’s Law — when a measure becomes a target, it stops being a good measure. Reward hacking, trader bonuses, and benchmark gaming are all variants of it
- The Bloomberg Odd Lots episode itself, 20 August 2026
One Thing to Take With You
What stayed with me from this episode was the crayon drawing: it beats any masterpiece online, not because it’s better, but because a child made it and put effort in. Whether something is worth doing turns on whether you were the one who did it, not on how good the result is. That passage is why “AI does it better than me” stopped being my reason to stop.
A small thing I tried, if you want to: today, pick one small thing you’d normally outsource, automate, or dispatch the fastest way, and do it by hand for one specific person, letting them know it was you. Handwrite a card instead of texting. Cook a meal instead of ordering in. Look up something they once wondered about, then say “I looked it up for you.” The result will almost certainly be worse than the outsourced version — my first card was so crooked I was embarrassed, and they photographed it and kept it anyway. That evening, write one line: was their reaction the same kind you get when you press a button and hand over a better result? A week later, read that line again and see if it still holds.
This article is an educational discussion of investment method. It is not advice to buy or sell any individual security, offers no target prices, and does not analyze any current holding. Investing carries risk; make your own decisions or consult a qualified professional.
Comments
Loading comments…