Quick Answer
Survivorship bias is a statistical error that results from concentrating on entities that passed a selection process while overlooking those that did not. Startup marketing advice is built almost entirely from the companies that made it, so the channel that failed for most teams leaves no record. Aviation faced the same gap and built confidential, non-punitive near-miss reporting, and a small team can copy four of its habits.
You pick your next channel the way most founders do, from what worked for someone else. The case study is real, the numbers are real, and the play still stalls when you run it.
What you read was never a sample of everyone who tried. The teams that ran the same launch into silence didn't write it up, so the record in front of you holds winners and very little else.
This covers the four reporting habits aviation used to close that gap, what Abraham Wald's wartime memos actually say about returning planes, and the three conditions under which publishing your own misses does harm.
Four Reporting Habits That Put the Crashes Back Into Startup Advice
You counter survivorship bias in startup marketing by recording your own misses while they are fresh, protecting whoever records them, stripping identifying detail, and publishing the pattern on a schedule. Those four habits are the working parts of the Aviation Safety Reporting System, cut down to a size a five-person company can run.
None of them needs a tool or a budget. Each one below covers what it is, how it works, an example, how to run it, how you know it worked, and where it usually goes wrong.
1. File the Report Within Ten Days of the Miss
The first habit is a deadline. When a campaign, launch or outbound sequence underperforms, someone writes it up within ten days. That is the same window a pilot has to report to NASA to qualify for the FAA's waiver of penalty.
The deadline matters because memory edits itself. A post-mortem written a quarter later records the story the team agreed on, not the confusion it actually felt, and the confusion is the useful part.
Take a webinar series that draws registrations but no sales conversations. The ten-day note records what you expected, what you did, what happened in plain numbers, and which signal you read wrongly.
Run it by putting those four questions in a shared document and making the note the last task of every experiment. You know it worked when notes arrive without anyone chasing them. The common mistake is waiting until the result is final, which is exactly when nobody remembers why the decision looked sensible.
2. Protect the Person Who Files It
The second habit is a promise that filing a report never counts against the person who filed it. The FAA made that promise concrete: it does not use reports submitted to NASA in enforcement actions, outside of criminal offenses and accidents.
Inside a startup the equivalent is simple and easy to break. If the marketer who reports that a partner campaign produced nothing is then blamed for the partner campaign, the next report won't be filed.
Run it by writing the rule down and having the founder file the first report, about their own call. You know it worked when people report misses they could have hidden. The mistake is feeding the notes into performance reviews, which turns a learning system into a record of who to blame.
3. Strip the Names Before Anything Leaves the Team
The third habit is de-identification. NASA's staff remove identifying information and return the identification strip to the reporter as proof of filing. Then they destroy the original report, paper and electronic, according to the ASRS program briefing.
For a software company that means removing client names, deal names, employee names and any figure that would expose a customer. What survives is the pattern: the audience, the offer, the channel, the timeline, and the point at which it stopped working.
A note written this way might read: a founder-led LinkedIn series grew followers for three months while demo requests stayed flat, because the posts spoke to peers rather than buyers. You know it worked when a colleague outside the project can't tell who was involved. The mistake is stripping so much that the conditions disappear, which leaves a moral instead of a record.
4. Publish What You Learned on a Fixed Schedule
The fourth habit turns private notes into public evidence. NASA has published CALLBACK, a monthly lessons-learned newsletter built from report excerpts, since 1979, and its program briefing counts more than 551 issues and 33,765 email subscribers for 2025.
This is the part that counters survivorship bias outside your own company, and it is the most defensible form of content marketing for tech companies you can run. Almost anyone can publish an opinion about a channel. Few can publish an accurate account of the conditions under which it failed.
Run it monthly or quarterly: two or three de-identified incidents, each with the expectation, the miss, the ignored signal and the change you made. Publishing that note under the founder's own name is the kind of founder-led writing Better Marketing builds on LinkedIn, because a specific miss is more credible than another highlight reel.
You know it worked when readers reply with their own versions of the same miss. The mistake is publishing the polished retrospective instead, which is a success story with the failure edited out.
Survivorship Bias Makes Startup Marketing Advice Look Safer Than It Is
Survivorship bias makes marketing advice look safer because the advice is written almost exclusively by the teams it worked for. A case study shows one founder who grew through a newsletter. It doesn't show how many started one and stopped at issue six.
What is missing is the base rate: of all the companies that tried the play, how many got the result? Without it, a single success reads like a method, and the conditions that made it work, such as an existing audience or a category buyers already search for, stay invisible.
AI answers repeat the same skew. ChatGPT, Perplexity and Google's AI Overviews summarize what has been published, and the published record is mostly survivors. An AI SEO service can make your pages clear enough to be quoted, but an answer engine cannot quote a failure nobody wrote down.
The same question applies when you buy help. When a B2B demand generation agency proposes a channel, ask what share of its attempts on that channel produced pipeline, not only which one became the case study.
Benchmarks carry the problem in miniature. When we traced nine SaaS benchmarks to their primary sources, three had no study behind them at all.
Abraham Wald's Memos Show What the Returning Planes Hid
The classic case of survivorship bias comes from Abraham Wald. In 1943 he wrote eight memoranda for the Statistical Research Group at Columbia University on estimating aircraft vulnerability from the damage on planes that came back.
The Center for Naval Analyses reprinted them in 1980, noting the work was never published externally. They are mathematics, not anecdote. One worked example spreads 202 hits across four parts of the returning planes and estimates how likely one hit on each part is to down a plane.
| Part of the plane | Hits seen on returning planes | Chance one hit downs the plane |
|---|---|---|
| Engines | 39 | 0.39 |
| Fuselage | 78 | 0.05 |
| Fuel system | 31 | 0.15 |
| All other parts | 54 | 0.02 |
Wald labels the example hypothetical. The fuselage carried the most holes and came out least vulnerable, while the engines showed fewer holes than their size predicts, and the method reads that gap as planes hit there that never returned.
He also stated the limit plainly: from returning planes alone you cannot estimate both where planes were hit and which hits downed them. You need to know how hits are spread in the first place, and your marketing data has the same blind spot.
One Unshared Near Miss Came Before TWA Flight 514
TWA Flight 514 is the case aviation cites for why a near miss has to travel beyond the company that had it. It is survivorship bias inside one industry: the crew that survived the error kept the lesson in-house.
In October 1974, a United Airlines flight cleared for the VOR approach to runway 12 at Dulles descended to 1,800 feet about 25 nautical miles from the VOR and landed without incident. The crew had believed the approach clearance let them descend to the final approach altitude straight away.
United investigated, and its review included contact with the FAA at the Dulles tower. It concluded that it was not necessary to contact the FAA, submit recommendations or change procedures. Instead it sent its own crews a notice warning that the words “cleared for the approach” generally put the flight crew on their own.
On December 1, 1974, TWA Flight 514 flew the same approach, descended to 1,800 feet on a similar misreading, and struck Mount Weather in Virginia. All 85 passengers and seven crew were killed. The FAA's lessons-learned record says that had the information been more widely distributed, the accident might have been avoided.
The Aviation Safety Reporting System Made Candor Cheap
The Aviation Safety Reporting System took survivorship bias out of incident data by making it safe to report the event nobody else saw. NASA's program briefing puts the design in three words: voluntary, confidential, non-punitive.
The FAA started a voluntary reporting program in April 1975. It then decided the program would work better if NASA, rather than the FAA, received and processed the reports, according to Advisory Circular 00-46F. NASA's system was established in April 1976.
The waiver of penalty is narrow, which is part of why people trust it. Under the advisory circular, neither a civil penalty nor a certificate suspension is imposed when all four conditions hold:
- The violation was inadvertent and not deliberate.
- It did not involve a criminal offense, an accident, or an action showing a lack of qualification.
- No prior FAA enforcement action found a violation in the five years before the event.
- A written report reached NASA, delivered or mailed, within ten days.
NASA's briefing states the reasoning in one line: “When organizations want to learn more about the occurrence of events, the best approach is simply to ask those involved.” Through December 2025 that approach had collected 2,321,050 reports and produced 8,069 safety alert messages to organizations in a position to fix problems.
The model has traveled well beyond cockpits. NASA lists rail, medicine, security, firefighting, public utilities, maritime and structural engineering among the fields applying it.
Aviation Removed the Penalty That Startup Marketing Still Imposes
Startup marketing keeps its survivorship bias because telling the truth about a miss still costs the teller. A founder who publishes a failed channel does it under their own name, in front of investors, candidates, customers and competitors at once.
| Element | Aviation reporting | Startup marketing today |
|---|---|---|
| Who reports | Anyone involved, voluntarily | Usually nobody |
| Deadline | Ten days for the waiver | None |
| Identity | Removed before the record is kept | Attached to every post |
| Cost of reporting | Penalty waived if inadvertent | Reputational and immediate |
| Where lessons go | CALLBACK and a public database | Private threads, if anywhere |
So the public version gets safer. The damaging figures come out, the failure is reframed as a growth lesson, and the story waits until the company has recovered enough for it to read as maturity.
Nobody outside your company can build the missing half for you. What you can do is lower the cost inside your own team, with a deadline, a protection rule and a de-identified record.
Publishing Your Near Misses Backfires Under Three Conditions
Publishing near-miss notes corrects survivorship bias only when the notes are honest about their own limits. There are three situations where it does more harm than good:
- When the sample is tiny. Four notes from one company are anecdotes, and presenting them as a benchmark repeats the error you set out to fix.
- When the detail exposes a customer. A miss that can't be told without identifying a client stays internal.
- When the public version gets written first. If the external story shapes the internal note, the reputational editing has already happened.
The aviation system has limits too. Its reports are voluntary, so they are self-selected, and NASA's briefing describes incident data as complementary to other monitoring rather than a substitute for it.
Treat published misses as one input in content marketing for tech companies, not a replacement for customer interviews or pipeline numbers. The same caution applies to launch numbers, where a launch spike measures attention, not demand.
File Your First Near Miss Report Before the Next Campaign
Survivorship bias will stay in the advice you read, because the teams that failed rarely publish. You can keep it out of your own decisions by filing within ten days, protecting the filer, stripping the names and publishing the pattern.
Those notes also make outside help sharper, whether you hire a freelancer or a B2B demand generation agency to pick the next channel. They are the one page no AI SEO service can manufacture for you.
A monthly note of what went wrong is a newsletter only you can write, and it is the kind of owned audience Better Marketing builds with founders. Trust is built in specifics, never in slogans.