I was wrong about optimal stopping

mathematics — 2 minutes read — 07 Apr, 2025
🌱 Seedling

If you were tasked with a need to find the tallest mountain, and went searching in a far away land surrounded by a series of mountains, how would you finalise the tallest mountain, especially when you could still go farther, and find even more taller mountains (only if you explore even more).

There are various ways to term this, some call it the travelling salesman problem, or the “secretary problem”, or just as the “optimal stopping” problem, which attempts to come to a mathematical decision on when to actually stop in such explore versus exploit situations.

I first saw this mentioned in the book, Algorithms to Live By, and was mesmerized by the practical applications of it, in everyday life. And for years, I thought the optimal stopping problem was all about the timeline. You go 37% of the way through the time period, and you make the big decision if you see a better option. Not too soon, not too late.

But that was so wrong. It was not just “reject the first 37% of your time window, and then choose the next best”. Mathematically, it’s about the number of discrete options (aka decision counts), not time.

If we were to extend this to hiring candidates:

  • Wrong: “We’ll interview for 3 months, then decide.”

  • Right: “We expect ~20 candidates. Reject the first 7–8, hire the next best one.”

For career moves:

  • Wrong: “Switch jobs after the first 5 years.”

  • Right: “Try ~10 career directions? Explore 3–4, then commit to the next clearly better one.”

For startups:

  • Wrong: “Spend 1 year scanning ideas, then commit.”

  • Right: “If I’ll vet ~25 ideas, benchmark the first 9, then jump on the next standout.”

The only catch here is that you will not be able to know the total n in advance. How might you be able to take a guess on the total number of job offers, partners, or startup ideas that you can have? It’s based on this. But when you do have a very concrete value for (N), it becomes easy (eg. hiring rounds, apartment hunting etc)

Liked this post? Get email for new ones:

Here are some other posts you might enjoy

Thorough reference checks

I've been hiring people (and conducting more thorough reference checks) more recently now, and I've learned something important: most reference checks are useless. They're like those mandatory training videos you have to watch at big companies. Everyone goes through the motions, but nobody really learns anything. But it doesn't have to be this way. Reference checks can be incredibly valuable if you do them right. Here's how I do it....

leadership
Design education amidst social tribulations

The _Lead By Design_ program started about 5 months back in Auroville, Pondicherry for a group of 11 students from two different communities, the edayanchavadi and kottakarai village communities. It was in the initial phase difficult to get these communities together, especially the parents to allow their children to learn with each other owing to their varied social backgrounds. There were certain occasions while their used to be communal clashes. Even now, for instance, these clashes continue where one community always suppresses the other owing to the status. It was also observed that these children from the ‘_lower_’ status were labelled throughout. The labelling however was not something which happened all of a sudden. It has been happening right from when the french rule was initiated in Pondicherry....

design
Exploring "smart connections" for note taking

Not starting with a blank slate has been a great productivity boost in my writing. I wrote 50K words in 2024. And I can safely say that these 50K words have been written in a well thought manner, instead of an AI generated word salad. All this, because I've been exploring this neat little plugin called as [Smart Connections on Obsidian.](https://www.google.com/search?q=smart+connections+obsidian+plugin&sourceid=chrome&ie=UTF-8) It is a tool, and I wouldn't be naive enough to say that tools don't matter. This tool allows for two main affordances:...

knowledge
Design Manifesto

This thought was inspired by the book _Design Expertise_ (Lawson & Dorst, 2009) which includes an interview with the architect Ken Yeang where the author mentions: “I give every new member of staff the practice manual to read when they join. They can not just see past designs but study the principles upon which they’re based”. In other words, what would be the ethos behind your own unique design practice? When every designer is different in their own way, what would be one’s own **philosophy of practice**?...

design
Dear enterprises, we're tired of your subscriptions

When you build a SaaS app, how do you price it? The first option which comes to everyone's mind is a monthly/yearly subscription model While building [Clarity notes](https://claritynotes.io/), I was stuck with a usual question when it comes to building a SaaS—How should I price the app? We settled on the usual monthly/yearly pricing structure. It included a free version to allow users to try out the YouTube note-taking feature as well as a Pro Plan involving all the advanced features not available in the free version. We also tailored an Academic Plan, offering students an affordable option to take notes on YouTube....

software
Minimum Lovable Product

We might have to rethink on the definition of the 'Minimum Viable Prototype'. Especially since the bar for what's viable keeps rising up, with the likes of Gumroad, etc being built in a weekend....

product
Complex project management for product

In complex companies, with non-obvious interdependencies and hard schedule constraints, organizing the process is a huge "full-time" job, and can save weeks of delays if done right. It signifies the role of a project manager, despite the teams having a product manager in their fold. As [Ben Kuhn](https://www.benkuhn.net/) puts it, when done right, it can save weeks of delays and nip problems in their bud even before they materialise in the first place....

products
Brew your ideas lazily

Leonardo da Vinci's masterpiece, the Mona Lisa achieved through the painstaking application of countless gossamer-thin layers of oil paint over the course of many years, many months. The _sfumato_ technique which Da Vinci popularised, involved applying more than 40 layers of paint, each only 10 to 50 micrometers thick, using fingers to blend the colors and create the depth of illusion. The creation process was "perpetually unfinished" — He began the portrait around 1503, but didn't complete it till his death in 1519. And it would be foolish to dismiss him as a 'master procrastinator' — _What if the attribute of delayed procrastination in itself had some merit?_ I'm beginning to suspect that this was the hidden reason behind the genius of Mona Lisa. The art of brewing ideas lazily. In 2009, a review of three dozen studies conducted by researchers of Lancaster University concluded that setting aside a problem was helpful in improving the performance of divergent thinking tasks....

ideas
New mediums for humans to complement superintelligence

If superintelligence has already been commoditized and neatly packaged into the workforce, what would our renewed "mediums of message" look like? To understand the shape + form of medium 2.0, we might need a quick detour through the history of mediums and 'what has been.' ...

ai-coding
Writing in Future Past

We lack frequent usage of the **future past tense** in modern discourse. When I was recently drafting my new year resolutions, I noticed the use of 'I can', and 'I will', and found myself questioning the format, especially when I see that I'm good at making promises, but end up being miserable at keeping them....

writing
Idea in the shower, testing before breakfast

Imagine having an idea in the shower and testing it before breakfast? It's highly plausible now as AI lets you **prototype at the speed of thought**. Currently, I use Claude Projects and Cursor to build what I call **disposable apps** - quick prototypes that prove a point (read more in [this essay about vibe coding]([[Vibe coding]])). The magic? No sunk costs. I can write 5,000 lines of code in ten minutes, test it, and throw it away if it doesn't work. This freedom to experiment has transformed how I solve problems....

software
Can a thought be an algorithm?

When you actively hold a question in your mind, you start seeing potential answers and questions related to it pop up in your radar. The question and the answer co-evolve in a gracious dance enriching our understanding of the world and space around us. Questions act as your personal radar. > The textbook definition of a question is **a sentence used to seek information**. But it feels borderline criminal to reduce something as foundational as a question to something…transactional. We think in questions. If knowledge is a complex web inside the mind, each node is a question, and each connecting line the information you’ve picked up on the journey from node to node. The pursuit of knowledge is the accumulation of questions. They shape our identity as much by being asked as by being answered....

knowledge
Importance of Why

When it comes to task management, a fundamental principle stands tall: explain the reason before the details of what and how. By letting others know the why behind a task, it facilitates faster completion....

task-management
Publish Originally, Syndicate Elsewhere

Writing for yourself on your personal website is the purest form of self-expression on the internet. It avoids any trappings from the algorithmic maze. And there are no digital echo chambers. It's just you and your ideas in your own cozy little garden. We're witnessing the renaissance of personal websites. As social platforms become increasingly unstable, more creators are rediscovering the power of digital sovereignty....

blogging
We have been scammed by the Gaussian distribution club

Taleb insists that we've been scammed by the Gaussian distribution club. The gaussian distribution has become so ubiquitous in our daily jargons, oru day-to-day decisions even. ...

mathematics
It's computational everything

I was listening to a talk by [Debbie Mcmahon from Financial Times recently at Productcon](Notes%20from%20ProductCon%202025.md) recently and I was surprised by the fact that even in journalism we're seeing tailwind trends more and more computational skills required — genAI is now used to spot stories, and to spin up newsworthy headlines and narratives around these stories. They are now, '**computational journalists**'...who are equipped with AI, data and trendspotting skills. ...

rough-notes
AI git commits

I built a CLI tool that leverages OpenAI to automatically generate concise summaries of git commit changes. You can find the [git-commit-summarizer npm package](https://www.npmjs.com/package/git-commit-summarizer) with installation instructions and usage details. The package helps developers:...

prototypes
Opportunity Harvesting

Intended Audience — For those who are in a transitory phase in their careers, looking for their next big leap of faith. This is a guide to harvest opportunies in a systematic fashion In the past seven months since I'd shifted to London, I've been on the lookout for harvesting opportunities around me. I deliberately avoid using the term 'job hunt' here, as I sought to broaden my scope beyond traditional roles, exploring opportunities with VCs, startup incubators, part-time gigs, and more. Opportunity harvesting could even mean having genuinely interesting conversations with people you admire online. That was success, too....

careers
Meta-analysis for contradictory research findings

In the world of nutrition research, contradictory findings are as common as fad diets. One day, a study proclaims the benefits of a low-carb diet for weight loss. The next, another study champions a plant-based diet for overall health. This constant flip-flopping of dietary advice leaves most of us feeling like we're stuck in a nutritional ping-pong match....

health
Pluginisation of Modern Software

Transitioning from Adobe to Figma was a big change for me in my design journey. At that time, the whole design ecosystem was revolving around Adobe. For image manipulation, you had Photoshop, Illustrator for vector graphics, Indesign for reports, XD for website or app prototypes and so on....

design