2017 Prediction Results

At the start of the year, I made sixty predictions about how 2017 would go. Now it's time to see just how badly I did. The predictions left unmarked were correct, those struck-through were incorrect. I'm only throwing out the Obamacare prediction, because no one can agree on what constitutes a repeal. It is marked in italics.

PERSONAL

I will still be living in Kansas: 95%
My parents will have moved into their new house: 90%
My parents will have sold our current house: 80%
I will still be annoyed about it: 70%
I will read 10 books this year: 90%
....12 books: 85%
....15 books: 75%
....20 books: 60%
I will write an average of one blog post per month: 80%
....two blog posts per month: 60%
I will not succeed in a longform fiction writing project this year: 85%
I will not get back together with my ex-girlfriend: 85%
I will not get a new girlfriend: 65%
I will not get a boyfriend: 95%
I will not experience a major political conversion this year: 80%
I will not experience a major religious conversion this year: 95%
I will be able to perform 15 push-ups with ease: 90%
....20 push-ups: 70%
....30 push-ups: 50%
I will not be hospitalized: 95%
I will not begin drinking regularly: 90%
I will not attend a solstice event: 80%
I will participate in the LessWrong community: 70%


ACADEMIC/PROFESSIONAL
Jayhawk Rocket Design will fly the hybrid engine on a rocket: 65%
Jayhawk Rocket Design begin work on a two-stage rocket: 70%
I will have had an engineering internship: 70%
I will have a job lined up post-graduation: 60%
I will still be studying aerospace engineering: 95%
....at the University of Kansas: 93%
....on track to graduate in May 2018: 90%
I will end the year with GPA 3.0 or above: 80%
Jayhawk Rocket Design will test fire our hybrid rocket engine: 80%
I will not be planning to attend grad school: 90%
I will learn Javascript this year: 80%
I will not learn Haskell this year: 80%
I will not begin learning Russian this year: 80%
My French will not improve this year: 75%

SCIENCE/SPACE

Human trials of CRISPR announced: 80%
No successful human clones announced: 65%
Self-driving cars licensed in the US or Canada: 50%
"EM Drive" will be explained by existing physics: 75%
KIC 8462852 will not be satisfactorily explained: 80%
No strong evidence of extra-terrestrial life will be found: 95%
SpaceX launches a reused rocket: 80%
Reused rocket is successful: 70%
Falcon Heavy will fly this year: 65%
No deaths in space this year: 90%
No country leaves the ISS Treaty: 85%

POLITICS/WORLD

Donald Trump will still be President at the end of 2017: 95%
Mike Pence will still be Vice President: 95%
Obamacare will not have been repealed: 60%
United States will not go to war with a nuclear power: 85%
United States will not enter a major new war (>100 US casualties): 70%
North Korean government will not be overthrown or displaced: 95%
North Korean government will not begin liberal reforms: 93%
Theresa May will still be Prime Minister of the United Kingdom: 90%
United Kingdom will not have triggered Article 50: 70%
Ukraine Conflict will not be resolved in 2017: 85%

META

I will remember to score these predictions: 95%
I will have been overconfident in these predictions: 80%

This last one sort of requires me to do the math first, so let's get to it. Spoiler: it won't be the tie-breaker question.

50% predictions: 0 right and 2 wrong, for a score of 0%
60% predictions: 2 right and 1 wrong, for a score of 67%
65% predictions: 2 right and 2 wrong, for a score of 50%
70% predictions: 4 right and 4 wrong, for a score of 50%
75% predictions: 2 right and 1 wrong, for a score of 67%
80% predictions: 10 right and 3 wrong, for a score of 77%
85% predictions: 6 right and 0 wrong, for a score of 100%
90% predictions: 8 right and 0 wrong, for a score of 100%
93% predictions: 2 right and 0 wrong, for a score of 100%
95% predictions: 10 right and 0 wrong, for a score of 100%

Graphed, my results look like this: 

What does this chart mean? Bear with me here, because this may get confusing.

The orange line represents perfect calibration i.e. 90% of the predictions made with 90% confidence are accurate. The closer to the line, the better calibrated you are as a predictor. Results above the line are underconfident, while results below the line are overconfident. My predictions made with high confidence were underconfident, while those made with somewhat lower confidence nevertheless proved to be overconfident.

As a general rule, underconfidence is preferable to overconfidence. In colloquial terms, it's better to be safe than sorry.

If we look at individual categories, it's clear that my worst performance was in Academic/Professional predictions, while my best was either Personal or Politics/World, depending on how you choose to count. Technically Meta was 100% correct, but that wasn't really so much of a prediction as an acknowledgement that January!Me had no idea what I was doing.

There's several issues I want to rectify before my 2018 predictions. Firstly, several of my sample sizes were too small. For example, there's only two predictions with 50% calibration. I got both of these wrong, but it's hard to say whether I was dramatically overconfident or just a bit unlucky. Meanwhile, most of my predictions were made with 85%, 90%, 93%, or 95% confidence. Arguably, I was essentially shooting fish in a barrel on these. It would be better to distribute my predictions more evenly.

Furthermore, it may not be a good idea to use even two significant figures for this kind of project. Throwing out the 50% predictions and lumping everything into decades (60% and 65% predictions together, 70% and 75% predictions together....) we get results looking more like this:

with the breakdown as follows:

60–69% predictions: 4 right and 3 wrong, for a score of 57%

70–79% predictions: 6 right and 5 wrong, for a score of 55%

80–89% predictions: 16 right and 2 wrong, for a score of 84%

90–100% predictions: 20 right and 0 wrong, for a score of 100%

This more seriously reflects the confidence gap between high probability outcomes (things more than 80% likely to happen) and possibilities with lower odds. Basically, I scored a lot of probably things too low, and a bunch of improbable things too high. It's an embarrassing sort of mistake to make, but figuring out where I'm under and overconfident is the entirely purpose of this exercise, so mission accomplished.

2018 predictions, hopefully reflecting these insights, coming soon.

November/December Links

A lot of people reacted poorly to a Washington Post story implying that the Trump administration banned the Center for Disease Control from using certain words in their documents. According to National Review and the New York Times, this misrepresents the actual, more complicated situation.

Speaking of healthcare, HIV/AIDS is looking to get a lot nastier in the near future. It's not really clear what we can do about it, beyond more of the same.

Something to do once I'm out of school: print my blog. Matt Shapiro argues that we really can't depend on platforms to continue providing hosting, forever, and having off-site backups is increasingly necessary if you want your works to survive.

In 1849, a Virginia slave named Henry Brown mailed himself to freedom in a wooden box. His escape was enabled by northern abolitionists, but also by confidential parcel services. To me, this underscores the importance of allowing free, private competitors to government agencies wherever and whenever possible.

Speaking of federal services: the origin of NORAD's Santa tracker.

Astronaut Bruce McCandless II dies at age 80.

AIAA reports of the progress of advanced spacecraft propulsion in 2017.

Elon Musk tweets photos of the assembled Falcon Heavy first stage.

Going Off On A Tangent

I just remembered an incident from middle school. As a middle school memory, it's unremarkable in that it shouldn't have happened in the first place, but is unusual in that some small bit of wisdom can be gleaned from it.

Back then I was what passed for a gifted child, and consequently enrolled in what passed for a gifted education class. During one of my projects—I don't remember which one—one of the teachers said something to the effect that it seemed like I was going off on a tangent. I didn't know what a tangent was, so I tried to explain myself using the entirely incorrect geometric metaphor. Something about legs of a triangle. That's what you get for teaching algebra before trig before calculus.

With the benefit of hindsight, I can explain myself better.

Whatever line of research I was trying to pursue only looked like a tangent because outside observers weren't looking at the whole picture. My plans were outrageously overambitious as a rule; teaching a preteen the concept of planning fallacy isn't necessarily easy. Nevertheless, I can't shake the feeling that observing the size of the plan is not the hard part of that task.

If we're going to make it through the 21st century, we've got to fix a lot of different forms of irrationality. Giving our "best" and "brightest" such shoddy training does not bode well. 

And Two Other Things

Further meditating on the problem of progress, I noticed a few other things which had been going right in the world and my personal life that contributed to the perception that things were moving a specified direction.

One of the most memorable things happened on the winter solstice of 2015: SpaceX, after many failures, successfully landed a rocket on the ground. It's worth noting that this technical triumph was underscored by multiple sorts of suffering in my surroundings. I'd literally fallen and hurt myself pretty badly that evening, and then been dragged to a very uninteresting Longest Night church service that ended mere moments before the landing. My attention was entirely directed elsewhere than the mundanity around me.

At this point, landing rockets has become essentially routine. I don't even watch launches anymore, though part of that is because I don't know when they're happening without Twitter. Maybe there's an email alert service for that.

Another thing which improved in the first half of 2016 began a week after the SpaceX landing. After an entire semester apart, I drove up to St. Paul to enter the new year with my then-girlfriend. It'd taken a long time to finally get together, and then we hadn't arranged to meet up again for the entire fall. But after that, we saw each other at least every three months until breaking up. This felt like emotional progress, getting to spend time with someone I loved on a regular if infrequent basis.

...there was going to be at least one other thing, but it's not coming back to me now. Hopefully this gets the point across: largely by accident, a lot of factors were lining up in what seemed like an organized pattern, only to diverge at a later date. Homo sapiens' overactive pattern-matching instincts strike again.

Try not to make that mistake in your own future. When things feel like they're going well, don't assume that they'll stay that way. Stuff can always go wrong, and—all other things being equal—probably will.

Hyperbolic Discounting Revisited

Related Reading: Picoprocrastination, Hyperbolic Discounting in Education

My, what a difference a year makes! As the months till graduation tick down, my time horizon has seemed to grow. It's not clear whether that's a product of the hard limit of my knowledge approaching, or my lived experience increasing.

Either way, my dissatisfaction with past selves has risen dramatically as 2017 progressed. At this point, I can even admit that my behavior—while not necessarily unpredictable given those initial conditions—was the primary reason I failed so dramatically during my years at Purdue. Four years doesn't seem like a particularly long time, not nearly as long as it felt five years ago, during my first semester of university.

Motivating myself with the prospect of being done with it (no, not like that) is now a viable strategy. It never worked before, and I found those kinds of pep talks somewhere between frustrating and completely incomprehensible. When I wrote about hyperbolic discounting in education, I was just barely starting to react to that line of reasoning, after a semester of struggling to make myself work through a workload that, in retrospect, was pretty good. I probably wouldn't have believed it then, but I'm actually nostalgic for Dr. D's thermodynamics class. It's unclear, however, how much of that perception stems from the broader situation.

Noticing progress can be a subtle art, but pleasurable nonetheless.

December Daily Journal 2017?

I'm considering repeating the December Daily Journal exercise from 2015 as a way of time-tracking during these last few weeks of the semester, and to lay out the things I need to do during winter break. I'm not sure if this is actually a good idea, so I'm going to sleep on it and see what I think about it tomorrow.