NHL 2013 Wrap-up

So the Stanley Cup has been awarded. Congrats to the Chicago Blackhawks!

As I previously mentioned, I've been running a model to try to predict the NHL finals for a few years, and I've been running it again this year. The model outputs looked like this over the last few weeks:

Again, as the time through the playoffs progresses (x-axis), the probability of a given team winning is displayed as the height of the team's bar (so that at any time all teams' probabilities add up to 100%). Originally I had predicted the Senators had the highest chance of winning, with the Bruins in second and the Blackhawks in third. Fortunately enough, two out of those top three actually made it somewhere...

What's important though is to check if I'm anywhere near accurate. It's convenient, I suppose, to say that my top teams all did fairly well, but how well did I actually do? Take a look at this (format stolen straight-up from Wikipedia):

	Conference Quarterfinals					Conference Semifinals					Conference Finals					Stanley Cup Finals
	1	Pittsburgh Penguins (64.2%)	4			1	Pittsburgh Penguins (22.8%)	4
	8	New York Islanders (35.8%)	2			7	Ottawa Senators (77.2%)	1
	2	Montreal Canadiens (15.9%)	1							Eastern Conference
	7	Ottawa Senators (84.1%)	4
											1	Pittsburgh Penguins (31.7%)	0
										4	Boston Bruins (68.3%)	4
	3	Washington Capitals (37.9%)	3
	6	New York Rangers (62.1%)	4
	4	Boston Bruins (76.7%)	4			4	Boston Bruins (59.8%)	4
	5	Toronto Maple Leafs (23.3%)	3			6	New York Rangers (40.2%)	1
																E4	Boston Bruins (52.4%)	2
		(Pairings are re-seeded after the first round.)
																W1	Chicago Blackhawks (47.6%)	4
	1	Chicago Blackhawks (77.0%)	4			1	Chicago Blackhawks (58.2%)	4
	8	Minnesota Wild (23.0%)	1			7	Detroit Red Wings (41.8%)	3
	2	Anaheim Ducks (40.9%)	3
	7	Detroit Red Wings (59.1%)	4
											1	Chicago Blackhawks (70.3%)	4
										5	Los Angeles Kings (29.7%)	1
	3	Vancouver Canucks (37.8%)	0
	6	San Jose Sharks (62.2%)	4								Western Conference
		St. Louis Blues (43.1%)	2			5	Los Angeles Kings (34.3%)	4
	5	Los Angeles Kings (56.9%)	4			6	San Jose Sharks (65.7%)	3

In each bracket, the percentages are the probabilities assigned by my model. A first glance analysis of this bracket shows that in 12/15 of the series, the teams I assigned the highest odds of winning to ended up winning (exceptions are Kings v. Sharks, Penguins v. Senators, and awkwardly Bruins v. Blackhawks).

If someone was flipping a coin, they would expect to predict 12/15 or better only about 1.76% of the time. This percentage is known as a p-value, and a standard convention in, say, medical experiments, is to use a value of 5% for determining if the results are significant or just arose as a matter of chance. Regardless of if this is a reasonable standard or not (this could allow 5% of the results of studies to be due to chance, and is a debate worth having), a "conventional" medical study with a p-value of 1.76% would likely be accepted, leading me to humbly suggest that my model this year was statistically significantly above chance when it comes to predicting the outcomes of series (though not necessarily the playoffs as a whole).

Another way of measuring the accuracy is by using a Brier score (similar to what I had in my previous post) to actually measure the accuracy of the probabilities assigned. It's all well and good to say the teams with the best odds often won, but how good were the odds that I assigned?

The average Brier score for all 15 series was 0.8182 out of 1, where a 50/50 guess for any given series would give a value of 0.75. My results are basically the equivalent of assigning a value of 57% at the outset to every team that actually ends up winning (as opposed to a random value of 50%) - in other words perhaps not definitively accurate and clairvoyant, but still likely significant.

One last thing to try is to plot the overall Brier score over the entirety of the playoffs, and compare that to random chance. It would look like this:

That's actually not so good. Compared to last year, there's virtually no difference between the two. At least in the parts where the two are different (mid-May, for instance), the model is higher than chance, but everywhere else is about even. The similarity between the two suggests that there weren't a lot of cases where series ended in remarkable against-the-odds comebacks, though when one did (Blackhawks v. Red Wings, for instance) it was able to be better predicted by my model than chance.

What's encouraging about all of this is that between all of the different ways of analyzing the accuracy, they converge on suggesting that the parameters that my model uses are reasonable and significant. Also, if I had bet on the first round, I would have made a killing. Too bad...

Spring Weather

I've now been doing this weather analysis of mine for 365 days, and what a year it's been. That's 365 days of exciting weather, culminating with tornado watches a couple of weeks ago and the horrible flooding around Southern Alberta going on right now.

The end of the year also marks the end of my experiment with tracking weather stations, which I previously reported for the summer, fall, and winter. The scores for spring (out of 100) were:

• Weather Network: 72.08
• Weather Channel: 69.56
• Global Weather: 68.52
• TimeandDate.com: 68.47
• Environment Canada: 65.96
• CTV Weather: 64.31

Just in case you didn't have the previous values memorized, this is how they've changed over the last season:

A couple of things about this are really quite noteworthy. First of all, the forecasters are all a lot closer together than they were in the previous two seasons. Good for them? Also, TimeandDate.com and CTV had massive improvements to their scores, with TimeandDate.com jumping two ranks higher. For the fourth season in a row, the Weather Network and the Weather Channel both hold the number 1 and number 2 spot.
As I mentioned before, one of the issues with this analysis is that CTV Weather only presents numerical values for POP up to four days in the future. If we only take these four days for all weather stations, the results change fairly drastically:

• Weather Network: 74.47
• CTV Weather: 71.09
• Global Weather: 70.63
• Weather Channel: 70.46
• TimeandDate.com: 69.84
• Environment Canada: 67.74

The overall score graphs for spring look like this:

As always, graphs like this highlight the fact that weather forecasting gets increasingly more difficult the farther into the future you attempt to predict. Any given day into the future only has a difference in score of about 10 points between any station, and all six weather forecasters have a nearly identical trend decreasing with time.

Since it's now been a year, I can also share the overall yearly scores for each forecaster. They are:

• Weather Network: 76.39
• Weather Channel: 73.92
• Global Weather: 72.00
• Environment Canada: 71.20
• TimeandDate.com: 70.02
• CTV Weather: 60.99*

*CTV Weather isn't a fair comparison as the analysis was started only in the fall, and the best season for all the other forecasters was summer. This probably messes up their average scores a wee bit...

Aside from the difference in numbers of days accounted for and length of analysis, one of the other major differences between stations is the presentation of POP values. For instance, Environment Canada has listed policies that state that if their calculated chance of precipitation is between 30 and 70% it will be listed (at increments of 10%), but 50% isn't allowed to be listed. This forces their values to be listed as either 0, 30, 40, 60, 70, or 100. CTV has a similar policy, but the other stations present values all the way between 0 and 100 at increments of 10.

The problem with this is that all forecasters could predict a 10% chance of rain, and if it does end up raining they'd receive a pretty bad score for that prediction. However, Environment Canada and CTV would receive a much worse score, because they would have published a value of 0%.

If we account for this, the new net yearly scores become:

• Weather Network: 74.61 (-1.78)
• Weather Channel: 71.94 (-1.98)
• Environment Canada: 71.20 (No change)
• Global Weather: 69.68 (-2.32)
• TimeandDate.com: 68.14 (-1.87)
• CTV: 60.99 (No change)

So yeah, changes in how values are presented definitely have an effect on the scores. This isn't quite enough to counteract the full difference between the Weather Network and everyone else, but it does tighten things up a little bit.

One last thing. When a weather station gives a POP value, what is the actual chance of any rain falling?

Ideally, the graph would be a straight line from bottom left to top right, but almost exclusively the bars on the graph are higher than what would be expected.

Part of the reason for this is that the definition of precipitation that I used was similar to how Environment Canada defines POP - the "chance that measurable precipitation (0.2 mm of rain or 0.2 cm of snow) will fall on “any random point of the forecast region” during the forecast period." 0.2 mm isn't a ton of rain, really, so what if I bump the values up to 1 mm?

Well that's quite the improvement. In fact, it substantially helps the scores for all weather stations by about 4-7 points each. What's cool is how close the bars are for low values of POP, and how they tend to diverge at higher values.

Anyway, this has been a pretty cool project for me for the last year. Hope you had fun too!

Edmonton Transit Hotspots

Last week's post about optimized traveling around Edmonton was pretty fun, right? It actually got me thinking about some different ways of looking at Edmonton's transportation systems.

Last time was taxis between bars, so I thought this time a cool idea would be the ETS, with its fun combination of buses and LRTs. Of course, now that my blog is the number 1 Google result for "Mathematically Optimized Pub Crawl" I figured I'd expand my horizons a little bit. Without any further delay, why not take a look at this:

This is a heat map of our fair city, based on transit time required to get to our biggest tourist attraction (West Edmonton Mall). The colours range from green (less than 10 minutes) to red (more than two hours). Almost every single neighborhood is within a one hour bus/train/walk (or combination thereof) from WEM apart from the very north-east of Edmonton, but there's a little island around Clareview station of reduced travel time. Cool, eh?

Next up is Century Park station:

Century Park's map is pretty cool. The south side of the city can get there really quickly (particularly close to 23rd Ave in Mill Woods), and along the LRT line in the north is pretty well defined too. Personally I really like the Moat o' Excessive Transit Time that surrounds downtown St. Albert, and the long wait times west of the river on the south side (apparently the buses there don't like the Henday...).

For something completely different, how about Rexall Place?

Perhaps not surprisingly at all, Rexall Place is very easily accessible by public transit. I'd post maps for Commonwealth Stadium, but it would be virtually identical. The LRT line extending as far south as it does really helps, as there are enough relatively direct buses both from St. Albert and West Edmonton that drop people off close to an LRT line. When it comes to great coverage for the city, though, you really can't beat...

...the University of Alberta. Man it's almost like they really wanted it to be easy for people to get to class, eh? Pretty much the only way you'd have a tough time getting into campus (tough being more than about an hour) is if you live in the no-man's-land between Edmonton and St. Albert. Also, we can see the effectiveness of Clareview LRT again, in the top right hand corner.

Speaking of St. Albert...

What's cool about the St. Albert map is that the direct bus lines to downtown, WEM, and the U of A are clearly visible, but going any further than that is pretty much a bad idea.

I like movies. I also like being treated like a very important person. You know where Edmonton's only VIP movie theater is? Windermere.

...but maybe it's better just to drive there.

Traveling Drunkenman Problem

The Traveling Salesman Problem is a classic in computing. Given a list of cities and distances between each city, what is the shortest path that lets a salesperson visit each city exactly once and return to the starting point?

It's a bit of a tough problem at high numbers of cities, due to the exponentially increasing number of paths one can take each time a new city has been added. Because of that, the problem is useful in testing optimization processes that don't rely on simply checking all possible combinations by brute force.

What's more fun is to adapt it in such a way as to optimize, say, your drinking habits. For instance, what's the shortest path to visit the 10 best pubs in Edmonton and return to where you started?

In order to be somewhat objective, I used the Yelp list of the 10 highest rated pubs. Of course, when you're on a mathematically optimized pubcrawl, it's important to not drink and drive - so distances are converted into dollar figures following the City of Edmonton bylaws for taxis.

2 pubs: $19.40 round trip

If we start with the #1 and #2 pubs on the list, it would cost just under $20 to visit them both by taxi. No computer necessary - there's really only one way to get there and back (unless the taxi driver takes you on an expensive sight-seeing tour).

3 pubs: $30.40-33.40 round trip

There are 6 different ways to do this trip once you'd added #3 on the list, but it's not at all difficult to check out all the combinations for yourself and plan it without using a computer. Normally there wouldn't be any variation in the price (as you'd just be traveling along the same three sides of a triangle), but in the case of downtown Edmonton with all its one-way streets, some ways to drive between places end up being slightly more expensive than others.

4: $35.40-39.40

Things get a wee bit more fun here - 24 combinations to deal with, with 12 different paths between pubs to take into account.

5: $39.60-43.60

120 combinations, 20 paths between pubs. Still not too much variation between the paths yet.

6: $42.60-58.80

720 combinations, 30 paths between them. At this point there are relatively significant savings to be had by trying to optimize, but 720 combinations would be an awful lot to check by hand. The major reason why there is suddenly so much variation between different paths is that the 6th pub on the Yelp list is very close to the first. Paths that take advantage of this do well, and paths that ignore it  end up costing a lot more than necessary.

7: $46.00-63.80

5,040 combinations. Things are getting really exciting now.

8: $49.40-72.60

40,320 combinations formed from 56 different possible paths between the 8 pubs.

9: $53.00-82.40

362,880 combinations. Let's just say that's an awful lot of things to check by hand (also messes with Excel a wee bit...).

10: $57.00-98.40

3,628,800 combinations. Again, quite a bit of a burden on poor Excel. But what's cool is that by optimizing your path between 10 of Edmonton's most popular pubs, you can save more than $40 in taxi fees.

This optimized pubcrawl through these pubs would look something like this:

What became readily obvious was the sheer amount of computing power necessary to check all of these. I have Excel files that are over half a gigabyte large in order to calculate the 4 million different combinations. Solutions to larger sized problems typically require other cool algorithms that I didn't feel like actually studying to learn.

Enjoy! Hope to see you out on the nerdiest tour of Edmonton's finest pubs next weekend!

Special thanks to Finbarr Timbers and Elizabeth Croteau for the idea, and Daniel Johns for helping me out of a tricky spot with VBA.

Beer Math

Over the last week I had the privilege of going both to MKT and Underground Tap and Grill in Edmonton. I had a very pleasant experience at both. They are both known for having large beer menus, so I took the opportunity to try out a couple of new beers.

Of course, one of the problems with such a large selection on tap is that it's easy to get overwhelmed by the sheer number of options. Underground, for instance, has 72 beers on tap - how are you supposed to pick a new one worth trying? A knowledgeable beer connoisseur could probably read the descriptions and ask pointed questions, but an extreme enginerd could just use math.

Yay!

Take a look at this menu. Each beer is offered at a different combination of volume, alcohol content, and cost.

Based on this, it's actually pretty straightforward to find which beer has the most alcohol per volume at the lowest cost. In fact, it would look something like this:

First of all, that's quite the variation! You could get almost three times as drunk off of the Salmon Fly Honey Rye as the Pilsner for the same cost. My goodness - I would never advocate for such a thing.

Of course, there are many factors that influence the cost of alcohol. One of them is the mandatory  minimum alcohol prices legislated by the Government of Alberta. Their rules work out to something like this:

• Spirits/Liqueurs: $2.75/oz.
• Wine (by the glass): $0.35/oz.
• Draft Beer: $0.16/oz.
• Beer/Cider/Cooler (bottle or can): $2.75 each

Assuming an average alcohol content of 5%, a can of beer selling at a minimum of $2.75 works out to $4.58 per ounce of alcohol - more expensive, in fact, than two of the beers sold at Underground! Before you think you can use this to break the law and stick it to the man ($0.12 per ounce of alcohol at a time!), it's important to point out that as these are draft beers, they are still selling for more than $0.16 per ounce of beer, and as such are still completely legal. In fact, the minimum alcohol cost for draft beer works out to $3.20 per ounce of of alcohol. Basically, if you're paying anything more than that, you're getting ripped off...

And as a quick aside before we finish, according to the Alberta rules, wine is the cheapest way to get drunk at $2.92 per ounce of alcohol. At this rate, it would only take $28.26 worth of wine to kill me by alcohol poisoning. How cost effective!