Better Know a Computer: Colley Matrix

by aorist9 on Oct 31, 2006 8:21 AM CST in Football

As the beauty pageant in the human polls begins, Texas stands to gain much, but the computers are proving a wet blanket for our title hopes. Let's take a look at how another Computer ranking system works.

A couple weeks ago I brought you a breakdown of the Billingsley Report the only computer with Texas in the top 10. Today I want to talk about the Colley Matrix, because it's incredibly simple. So simple that Wes Colley, who has a PhD in Math from Princeton, can explain it in 23 math filled pages. The size of the explanation, however, belies its simplicity. The Colley method factors in two factors: winning percentage and strength of schedule. Colley calculates the latter with what he calls the "iterative method."

Here's how it works

First the formula

Don't worry if you don't know what that Greek letter Sigma means

Colley takes every IA team and first ranks them solely by winning percentage. The winning percentage is calculated in the first half of the equation, so in the first "rankings" the second half of the equation resolves itself to 1/2, which is the average rating of every team.

In what's called the first iteration, Colley plugs in the rankings of all of each team's IA opponents (Colley doesn't consider IAA teams in his rankings). The r in the second half of the equation is where the quantified ranking is plugged in and they are all added up (that's what the sigma means). With this done, the teams are rearranged and the process begins again.

This is repeated until changes in the ratings are negligible. Thence are the final ratings (Colley took 1 page for this explanation then spent the other 22 talking about why it works and how he figured it out).

There are some obvious weaknesses of this scheme. When calculating SOS purely by wins and losses. Hawaii, BYU, and Oregon all get the same initial value as they are all 6-2. This evens out a little bit with the iterations, but it is still an undue bump for the opponents of BYU and Hawaii.

This week's Colley rankings

Michigan
Ohio State
California
Florida
Notre Dame
USC
Rutgers
Louisville
Auburn
Tennessee
Boise State
West Virginia
Texas
Texas A&M
Missouri
Oklahoma

Complete rankings can be found on Colley's website. Also on the site is a fun feature that let's you subtract and add games and see how it would change the rankings.

I ran some hypotheticals that have been tossed around the site at times.

First I asked what would happen if Texas had not played Ohio State

Without adding another opponent for Texas we jump from 13th to 8th, still behind Michigan, OSU, Cal, Florida, ND, USC, and Rutgers.
When I added in Alabama (a beatable team from a respectable conference), we moved up to 4th, behind only Michigan, OSU, and Cal.
When I added in Central Florida (an opponent from next year), we got to 10th, two spots lower than if we had played no other game at all

Second I asked what difference it would make if Oklahoma had beaten Oregon

The answer is that Oklahoma moves from 23rd to 14th and we move past WVA and Boise State to 11th.

If I can figure out any of the other computer rankings then more segments will be forthcoming.

--AR--

0 recs | Comment 14 comments

Story-email Email | Print |

Comments

Nice

Great post. Thanks for sharing.

by PB @ BON on Oct 31, 2006 8:22 AM CST 0 recs

Nerds Vs Jocks

This is the real revenge of the nerds.

Wells excluded of course.

by AdamDC on Oct 31, 2006 9:10 AM CST 0 recs

Nerd here too

... not that there's anything wrong with that.

But seriously, look at all of the things we love in life that have been ruined by computers. Bill Gates must be behind the Colley Matrix. It's not called The Matrix for nothin'! He got beat up by his high school football team, so now he's trying to ruin the sport by injecting microchips and processors and SOI architecture and bad Java.

by patienthornsfan on Oct 31, 2006 9:47 AM CST up 0 recs

My biggest problem with this system

is that, like you pointed out, it uses straight wins/losses for the first iteration. That unfairly advantages teams with a horribly weak SoS.

To maximize returns from this system, just schedule the conference winners from MaC, Conf USA, and other powerhouses for your out of conference games. Texas should steamroll just about anyone from there, and it would provide a SoS boost as well.

by Brandon 97 on Oct 31, 2006 9:21 AM CST 0 recs

Smell Test

Well done: a nice explanation of the model and its application.

A model, however, is only as good as its predictive ability. In other words, one needs to look at the predicted results to assess the model's quality -- and in this context the model is a lot less compelling. Trying to be obective, I find it difficult to believe that several teams ranked above us -- e.g., ND, Rutgers, Boise State -- merit their positions.

Ironically, the discussions and rankings of the computers are giving me new respect for the human polls. Seriously, this year the poll with which I have most agreed is the Blog Poll, in large part because I think the voters are perhaps the most interested and best informed.

by Allaha on Oct 31, 2006 9:36 AM CST 0 recs

Colley is not supposed to be predictive...

it's what computer experts would call retrodictive, considering only what's happened so far, not unlike the "resume approach."

Pretty much all the computers used for the BCS are retrodictive. They even use Sagarin's retrodictive results even though his predictive results and the mix of the two are both more accurate.

by aorist9 on Oct 31, 2006 1:23 PM CST up 0 recs

MOV is needed

Much agreed. In order to be predictive, you really need the Margin of Victory component which the duffi at the BCS decided to make illegal to use in the computer rankings.

by drycreek on Oct 31, 2006 2:40 PM CST up 0 recs

retrodictive

Many thanks, AR: you have taught me a new word.

Insofar as retrodictive ratings are reflective of a resume methodology, I see the logic. Nonetheless, one has to question the ultimate purpose of this approach: if the goal is to rank the teams for purposes of obtaining the best BCS matchups, then implicitly the rankings should in fact be predictive (i.e., each team in theory should lose to higher ranked and beat lower ranked teams) to obtain the optimal pairings. . . . Or am I missing something here?

BTW, you may be interested in the following:
http://www.nutshellsports.com/retrodictive.html

by Allaha on Oct 31, 2006 2:55 PM CST up 0 recs

As it says in the link you left...

retrodictive ratings are a lot easier to make because all the information is right there in front of you, and their formulas are easier to understand.

by aorist9 on Oct 31, 2006 3:36 PM CST up 0 recs

criterion

I now understand retrodictive ratings are (almost by definition) easier to make just as their formulas are easier to understand.

My (non-rhetorical) question, however, is whether the focus should more appropriately be on retrodictive or predictive rankings for BCS purposes.

by Allaha on Oct 31, 2006 6:57 PM CST up 0 recs

In theory it should...

but all predictive rankings that are any good involve MOV, which was outlawed after the 2000 Miami/FSU debacle.

by aorist9 on Oct 31, 2006 7:07 PM CST up 0 recs

Two Things

First their is only one Optimal Paring a year, unless your use of the plural is indicating over a period of many years. The rest of the BCS teams besides 1 vs 2 is to make the most money and to try to respect previous contracts, not to make optimal pairings. Otherwise they would have 1 vs 2, 3 vs 4 and so on.

Second, I dont think the goal is obtaining the best BCS matchup in the title game. Sometimes a situation may occur, depending of different team strengths and weaknesses, where the third place team may have a better chance at beating the first place team than the second place team does. Rankings are not only a perdiction on future results but a reward for what you have accomplished in the year, hince the use of retrodictive rankings.

by Wells on Oct 31, 2006 3:47 PM CST up 0 recs

pairings

as you note, pairingS over multiple years . . . although I would be happy if the format changed so that 3 did play 4, 5 play 6, etc. . . . I would be really happy if 1 played 4 and 2 played 3 with the winners advancing to the MNC -- but again, that is a different discussion.
Not to get (more) philosophical, but shouldn't the goal be to obtain the best BCS matchup in the title game? I think so. . . . Precisely the point I am trying to make is that if indeed the third place team has a better chance than the second place team of beating the first place team, then that is strong (although perhaps not dispositive) evidence that 2 and 3 should be flipped in the rankings.

While I now understand the concept of retrodictive ratings (as you say, a reward for what a team has accomplished), to the extent they do NOT have predictive power, I question -- in other words, I do not know -- whether it is the best criterion for selecting the MNC participants.

by Allaha on Oct 31, 2006 6:54 PM CST up 0 recs