The 27 Club
"There are known knowns, there are things we know we know. We also know there are known unknowns, that is to say there are some things we know we do not know. But there are also unknown unknowns - there are things we do not know we don't know."(Donald Rumsfeld, Feb 2002)
The relief that we didn't actually win at Fulham was palpable (enough indeed to encourage me to write a rare blog post), and now the priority can rightly return to League One matters.
Readers of my We Are Leeds? blog will recall my concern that our season was resembling theirs rather too closely from 2009/10.
Their remarkable start to the season (they had accumulated a whopping 57 points from 24 games, compared to our own impressive 54) only went downhill rapidly after their FA Cup exploits misallocated vital focus and energy.
Thus I can only commend Chris Powell's team for skilfully balancing the need to make the 7,000 travelling Addicks proud, whilst at the same time ensuring they went out of the competition.
More evidence surely of a truly great side.
Looking forward, there are now four possible outcomes for this season:
Outcome A: automatic promotion
Outcome B: promotion via play-offs
Outcome C: play-off defeat
Outcome D: finish outside top six
As a brief aside, if I asked you the following questions, what would your answer be?:
Question 1: which of the following events is more likely? Charlton suffer Outcome C (play-off defeat), or you randomly select a diamond from a full pack of cards?
Question 2: which of the following events is more likely? Charlton suffer Outcome C (play-off defeat), or a family of four has two sons?
I instinctively sense that most people would answer both questions by stating that the latter of the two options is more likely.
If so, then I fear that you are wrong for reasons I will now explain.
Calculating the true probability of Charlton's four season outcomes is virtually impossible even with massive computing power, because of the sheer number of relevant factors and influences, plus the so-called 'unknown unknowns' as Donald Rumsfeld referred to above.
As a proxy there are however numerous bookmakers willing to lay bets on some combination of Outcomes A-D.
Of the major bookmakers, seven are willing to lay odds on Charlton being promoted (whether automatically or via play-offs), and these odds range from a not-very-generous 2/7 (Stan James) to a more reasonable 4/11 (Sky Bet).
The probability equivalent of these odds for those (like me) that prefer to think in those terms are 78% and 73%.
I usually prefer to use Betfair's odds as the best proxy for actual probabilities as they better resemble a true market-driven view, and theirs are 1.39 or 72% in the above vernacular.
Given Betfair's odds are (as usual) similar but slightly better to those available on the High Street, I am comfortable taking 72% as being an excellent estimate of Charlton's actual probability of promotion.
However this 72% encompasses both Outcome A (automatic promotion) and Outcome B (promotion via play-offs).
In order to attach probabilities to all four outcomes, I need to use this 72% as the basis to strip out the probabilities of each separately, via some reasonable assumptions.
I will begin with Outcome D because it's the easiest.
Charlton are currently 16 points clear of Stevenage in 7th place.
With 22 games left, for Charlton to finish behind Stevenage we would have to experience form at least 0.72 points per game worse than the Hertfordshire side (and more for any of the other chasing clubs behind them).
This is equivalent to over 33 points on a full '46-game equivalent' basis.
Obviously such differentials in form are more likely as the number of games remaining falls, but still this appears a near-impossibility.
However as a Charlton fan used to disappointment, and in order to be intellectually 'honest', I will assign a 1% probability to Outcome D.
Now let's consider the dreaded play-offs.
If Charlton have to participate in the play-offs, then it is bound to feel like a meaningful disappointment given how long we would have spent in the top two positions.
The negative impact on dressing room sentiment and fan morale, allied with the tautological conclusion that our form must have been relatively poor in the closing weeks, are likely to offset the mere fact that our squad will likely be the strongest of the four clubs involved.
Conversely, given the way the League table is transpiring (with a large gap below 5th), the team which finishes 6th will feel as though it has a 'bonus free option' on promotion, and will feel under no pressure whatsoever (and probably play better as a result).
Putting these two views together, I am comfortable with the conclusion that each of the four play-off clubs will have an equal 25% chance of promotion.
What this means for the probability of Outcomes A-C is as follows, where PA below is the 'Probability of Outcome A' or automatic promotion:
PA + (0.25 x (0.99 - PA)) = 0.72
0.75PA + 0.2475 = 0.72
PA = 0.63 or 63%
The above equations strip out what aspect of the 72% probability of promotion is attributable to automatic promotion or Outcome A (63%), and what probability is attributable to promotion via the play-offs (9%, ie. 72% - 63%).
They do so simply by stating that the probability of promotion is equal to the probability of automatic promotion, plus the probability of promotion via the play-offs (which requires that we actually compete in the play-offs, ie. weren't promoted automatically or positioned outside the top 6!).
Thus the probability of Outcome A is 63%, and of Outcome B is 9%.
The slightly uncomfortable remaining conclusion therefore is that the probability of Charlton losing in the play-offs (Outcome C) at the end of the season is 27%, the balancing figure:
Probability of Outcome A: 63%
Probability of Outcome B: 9%
Probability of Outcome C: 27%
Probability of Outcome D: 1%
If you agree with my analysis to a decent degree, then I'd be keen to know whether these probabilities are similar to your own 'gut feeling' about the remainder of this season.
I wrote up this analysis because I instinctively feel that a 27% likelihood of more play-off heartache seems too high, yet I feel as though I've proved that it isn't.
And now every time I walk past parents of two boys (or look at my own two for that matter), I shut my eyes and see Nicky Bailey's play-off penalty disappearing into the night sky, and say, "Oh no Lord, please not again!"
Fantastic stuff as ever.
I must admit, I had to read this twice to make sure I was following everything correctly.
Interesting post. I understand the logic of your calculations, but they do not correspond to my own "gut instinct" of our likely chance of promotion, nor to those of other Charlton fans I know.
Your figures seem too pessimistic. My feeling is that we have about an 80-85% chance of automatic promotion. I can think of a couple of reasons why my unscientific view is more optimistic that your figures.
(i) We have had a highly successful first half of the season, and simply can't see any reason why this should not continue.
(ii) We WANT success so badly, we refuse to contemplate failure. This has no effect on the odds, but does affect perception of likely success.
Geoff
As always your logic and maths are unquestionable.
Where I have reservations is that the bookies odds are based on covering bets already in play and coming on day by day.
As some of those teams in the chasing pack are all so close together there is probably not much to choose between them so the bookies would have fairly static odds for all of the top five. As the two Sheffield clubs, and Huddersfield will have seen high levels of bets on them at various times this season the bookies would not want to leave their odds for promotion too long - especially as at least one team will finish in the top two, irrespective as to where we finish.
Thus I expect the odds - what those that gamble have put their money on, rather than a true reflection of statistical probabilities - to make our chances of failing seem higher. Despite our impressive turnout yesterday, I'm guessing we have fewer fans putting money on us than the other three clubs I mentioned. In no small part due to our drastic fall during the last five years.
After my reasoning I'll go on to give you my gut feeling. I believe the actual outcome will be top two. Like you, I believe this more following our defeat yesterday (Saturday), and even more so with the Sheffield clubs and MK Dons having more FA Cup games to play.
However, I will have a much better idea (just like the bookies will) at 5pm on the 21st of January.
Read this through 3 times - and I'll take "A". (We say that a lot in Canada.
I leave the maths to you but for me itan A for your article and A for promotion from me wizard of Oz
If Charlton can continue to show the same level of quality as your all-to-rare blogs, then we'll be going up in May however which way!!! Great reading.
All I know (and as I stated in my own blog a few weeks ago) is that if the wheels fall off now, it will be the hardest blow to take of all those we've endured in recent years.
Thoughtful post as always NYA. My gut is more pessimistic than it should be. This is a big month as previously stated. Perhaps cautious optimism would be more realistic and objective. However - I recently won £220 at 22/1 when Blackburn beat United at Old Trafford. I am obviously very happy about that, but it does show that unlikely outcomes are possible.
BTW, I'd be interested to know what the probability was of that unlikely result at Old Trafford. 22/1 seemed very long to me, hence my £10 bet.
Geoff, my data do indeed 'feel' too pessimistic but the intention of the post was to try to prove that they actually weren't. If you still disagree then there is a great betting opportunity for you to back us at 4/11 to go up.
KHA, you make an interesting point - I've discussed on my blog in previous posts how bookies odds reflect (as you say) their desire to have a 'balanced book' more than the actual probabilities. However whilst the League One betting market is unlikely to be highly active, it is one which is likely efficient enough to ensure that any such arbitrage opportunity soon disappears. In other words, if there was too much 'weight of money' on the three big Yorkshire clubs, then before long the weight will have shifted to Charlton if they continue to offer odds that are materially 'wrong'. My conjecture here is of course that the odds are not wrong!
Hungry Ted, I totally agree - I was discussing with another fan last week whether the new owners had even contemplated the prospect that we might start 2012/13 in League One. The impact on attendances and general morale etc. would be horrendous because expectations have been ratcheted up so far.
Richard, like you it seems I also tend to oppose very hot favourites in one-off matches like Man Utd vs Blackburn (I usually prefer to 'lay' the favourite at say 1/5, so that I also get the benefit of the draw). It's hard to say what the 'actual probability' of Blackburn winning that game was - the way I like to conceptualise it is to ask myself, "If this fixture was played out 1,000 times (with known facts about injuries, suspensions, form, morale etc.), how many times would I expect Blackburn to win?" Odds of 22/1 sound quite wide, but it implies that Blackburn would win 43 times out of 1,000 - funnily enough, when presented like that, I actually think they are not wide enough.
Cheers NYA, another great read. I miss it when you don't post.
I am glad we have been knocked out of the cup too. I hate cup competitions and didn't even make the trip to Fulham. I have also booked a big weekend in Bournemouth the weekend we are playing them and I think the 5th round of the cup was scheduled for that day.
My gut instinct is 90% chance of promotion. The 10% chance of not going up is only so high due to the potential of a massive injury list decimating our squad.
I know the 90% could look stupid if we lose the next 2 games but I am confident of achieving at least 4 points in the next 2 games. Against top half teams we average 2.4 points per game (as opposed to 2.1 points per game v. bottom half teams).
I expect 93 points (an unusually high figure) will guarantee automatic promotion. 13 wins from 22 games.
I'm usually a pessimist but I honestly can't see any other outcome than automatic promotion for us. The only thing stopping me from saying being Champions is my normal "glass half empty" attitude.
My gut instinct is 90% chance of promotion. The 10% chance of not going up is only so high due to the potential of a massive injury list decimating our squad.
I know the 90% could look stupid if we lose the next 2 games but I am confident of achieving at least 4 points in the next 2 games.
Against top half teams we average 2.4 points per game.
I expect 93 points (an unusually high figure) will secure promotion. 13 wins from 24 games.
I'm usually a pessimist but I honestly can't see any other outcome than automatic promotion for us. The only thing stopping me from saying being Champions is my normal "glass half empty" attitude.
Anonymous, I'm glad that someone else has noted that the next two games ARE more important than other ones, despite Chris Powell's view to the contrary. This will be the subject of my next post before Saturday.
Your observation about the form versus top half compared to bottom half sides is interesting, thanks. Somewhat oddly perhaps, Stevenage are the second best in the League against top half sides. Sheffield Weds meanwhile have only lost 2 points from a possible 39 against bottom half sides.
Made my head hurt. Herek.