Point, Counterpoint: Preseason Rankings
As we continue our discussion about preseason rankings, two differing philosophies underlie most people's thinking. Either (1) preseason rankings should be an attempt to project the final -actual- rankings of teams (see: Feldman, Bruce), or (2) preseason rankings should just be an attempt to list the best teams, schedule and projected results be damned.
Today we've set up a formal point, counter-point debate between yours truly and College Football Resource. Before you read on, understand that we're each taking one side and arguing for it completely. We're each playing Devil's Advocate for the other, and these arguments do not necessarily reflect what I, or CFR, think. We're simply representing a side of the argument. With those caveats in mind, let's get started.
Burnt Orange Nation: This should be fun, CFR. Now, I'm going to start this party by laying out a premise for my side, and I'll let you take it from there.
We both agree that preseason rankings are rather meaningless, so let's not worry about that today. But since everyone does it, including those in the Blog Poll, we'd might as well talk out the differing approaches.
I'm going to argue today that we should be looking at the schedules, trying to project wins and losses, and doing our best to foresee how the final season rankings are going to look. Here's the thing: at the end of the year, the BCS standings are all that matter. And those standings invariably account for wins and losses. It doesn't matter if Georgia, by the end of the year, is the third best team in my mind. If they lost their first four, they're not going to be ranked #3. Why shouldn't I be projecting the cumulative result of their season?
College Football Resource: Peter, thank you for inviting me to be a part of this dialogue.
I think the two approaches to rankings differ thusly: the first approach, which you advocate, answers the question "based on their schedule, where should team X arrive at when the season is finished?" The second approach, which I'm advocating here, answers the question "how good should team X be this year?"
The advocates of both approaches must do some projecting, but the one has a more outward look projecting from the present to the future, while the other does the inverse, working backwards from the future to the present.
I'll toss this discussion back to you by asking the following:
Are we setting ourselves up for a fall by creating preseason rankings that assess teams not by the "content of their character" but by what their schedule says they should do?
Burnt Orange Nation: For starters, I'm not sure that they're mutually exclusive. In other words, I don't know that you can't, and shouldn't, do both.
The real tricky part of this whole matter is that every team doesn't play each other. Given that, two things seem to me to be true. First, there's a hefty amount of subjective evaluation that you have to live with. But second, because you don't play everyone, you have to place - perhaps more than you or I would prefer in an ideal world - a substantial amount of weight to wins and losses.
Let's get back to my Georgia example, as a starter. They strike me as a perfect example for why trying to rank teams based on who's "best" may be problematic. It may take a full season for the quarterback situation to resolve itself. Now, by the end of the year, with Stafford comfortable at quarterback, I can see them being one of the Top 10 teams in America. And yet, let's say these growing pains include four season losses - at South Carolina, Tennessee, Florida, and Auburn.
This is all hypothetical, but if I think Georgia's going to be a Top 10 team by the time they beat Georgia Tech and win a bowl game, but don't think they're going to have a season that permits ranking them in the Top 10, what do I do?
We're talking about preseason rankings, remember. Once we get into the meat of the season - we'll have so much more to work with. But let's run with this scenario for the fun of this argument. Your thoughts?
College Football Resource: I strongly agree that there's a tremendous amount of subjective evaluation that we must live with. I welcome it, in fact. But I'd rather we use that realization to release us from the strict burdens of wins and losses.
I could drive around in a high end luxury car if I wanted to, but it wouldn't necessarily mean I was a wealthy man. Just the same a team can be rolling around with a 11-1 or 13-0 record but it doesn't necessarily make them a great team. Leaning too heavily on objective measures hinders our ability to evaluate teams. As you said, you don't play everyone and that creates great confusion about one's performance (record) and actual aptitude (ranking).
Let's take a look at your Georgia example. I think we'd have to evaluate their expected performance in those losses, looking at if they were due to a raw quarterback finding his way, if they were close or if they were significant defeats, if the team was doing things that top 10 teams would do like stopping the run and limiting turnovers, etc. There are things to look at to help us make a reasonable subjective evaluation of their true worth.
If at some point that team had caught stride, say, settling on Matt Stafford and found some offensive consistency and started looking less like a top 25 team but a top 5-10 team, I'd have no difficulty whatsoever considering them for a top 10 preseason raking.
Part of the holdup with rankings is that people often argue about what teams "deserve". That is, if you've lost three games you shouldn't deserve to be in the top 10. Nobody deserves anything other than to be fairly and thoroughly evaluated by those considering them for the rankings. However, a team that has lost three games will suffer the consequences of those losses even from the strict "best teams" proponents. Any rational person realizes that losses matter and cannot be completely explained away. That's the balance between the pure win-loss method and the best teams method.
An example that comes to mind for choosing to rank teams by aptitude is last year's Tennessee team. I remember looking at a handful of preseason polls and magazines that all had the Volunteers in the top 2-5. I was shocked. I figured the Vols might win 8-9 games and look respectable, but I didn't think they'd be any good. As such they were not in my top 15 and that confused many people.
Now, nobody knew they'd be that bad last year, but even if they hadn't my method of ranking them was more realistic even if they had managed a more satisfactory record. I saw a team with a shaky quarterback situation, an overrated backfield and distracted players (several offseason arrests). They still had the talent and necessary soft spots in the schedule to get enough wins to be in the top 10, but I wasn't convinced they would be better than 10-15 teams. Unlike other parts of my preseason rankings, that assessment rang true.
Burnt Orange Nation: I'm not sure we quite settled this matter, CFR, but it's certainly been interesting. Thank you for representing one side of this debate, and I certainly enjoyed playing devil's advocate. I think the proper approach probably takes into account a bit of each - you want to rank your teams in a way that reflects who you think will be best by year's end, but you've got to have some sort of reward/penalty for a season's cumulative results.
Ultimately, while preseason rankings don't matter too much, regular readers of both our sites know that the college football narrative matters in the BCS system. And as Auburn's 2004 season illustrated: where you start the season has a lot to do with how you finish it.
And as the 2006 season gets closer and closer, I'm inclined to believe we could be on the brink of our first Full Scale BCS Disaster. Want a sneak preview of the BCS commissioners' worst nightmare? Have a gander. What's scary/delightful is that it's not -that- farfetched. And there are easily a dozen other, less wild, scenarios that could spark riots and couch burnings for years to come. Stay tuned. Just six weeks away, boys and girls.
--PB--
0 recs |
13
comments
Comments
Rankings
Slow day at the office so I had to weigh in on this. I think the very simple question of what a ranking is supposed to mean is being muddled in the age of school specific blogging and Direct TV packages. It's fun and certainly worthwhile to pick the results of every game on a schedule and determine a national champion in July. It's also feasible, with the ability to see every team in the nation first hand if you so desire (not to mention their recruits' high school games and spring game). But when I was growing up, I'd follow my hometown Union High School Farmers (an honest to goodness albeit ludicrous perennial national powerhouse that averaged 1 forward pass per half), and the only way blowout victories were gauged in terms of importance was the local papers' state rankings of our anonymous victims. Rankings are supposed to be a guide to the uninformed as to who should beat who if they played at that time, or how relatively competetive a team should be. Just look at Jeff Sagarin's USA Today rankings, which take it a step further in calculating HOW MUCH one team would beat a lower ranked team by. This is the point of the rankings throughout the year. Projecting team progress throughout a year or year end records is no less important - it's probably moreso - but it's a different animal entirely.
by detr0 on Jul 17, 2006 2:56 PM CDT 0 recs
Can't we have both?
Well, here's my "Extra Primo Good" preseason Top 25 ranking solution based on both theories above:
- First off, rank all D-1 teams 1-113 based on who you think the best teams are flat out (forget schedules). You'd probably base that on their performance last year, returning starters, and your own internal biases. We'll call this Top 25 the constant.
- Now take your list and look at the schedules of the top 30 or 40 teams and do the following:
B) If the team's schedule is pretty easy except for maybe one or two key games (Texas, OU, USC, etc) that you believe they'll win and keep their SOS respectable, take them up a couple of notches.
C) And if the team is like West Viriginia or TCU with no real big games against fellow Top 25 competition, just leave them where you had them originally.
Now, are we all throrougly confused? Good. Try it anyway. My theory is that your original Top 25 will look a lot like your new Top 25, but at least your scale will be graduated. It's still a long way from being a perfect science, but I think this Top 25 will quiet a lot of the arguments from both camps.
BTW, yes, I did stay at Holiday Inn Select last night.
Now shoot holes in me.
by 54b on Jul 17, 2006 3:02 PM CDT 0 recs
isnt it
by BigTexBD on
Jul 17, 2006 8:47 PM CDT
up
0 recs
It is...
Trading Places, however, is a movie that comes on the USA Network every 72 hours.
Seriously, you can set your watch by it.
by the other Andrew on
Jul 17, 2006 10:37 PM CDT
up
0 recs
I meant for you to shoot holes in my theory...
But hey, at least you guys were paying attention.
by 54b on
Jul 18, 2006 9:08 AM CDT
up
0 recs
there is no right answer
Peter's problem will be having to defend his stance and will become harder and harder with each loss and easier and easier with each successive win.
As long as you can handle the heat of your rankings, do what you think as long as you can back it up with logical arguments.
Hell, even after the Rose Bowl we had one blogger in the Blog Poll who still ranked USC #1. He certainly had that right, but stood to lose what little credibility he had left.
by the other Andrew on Jul 17, 2006 3:32 PM CDT 0 recs
good call
by USCLink on
Jul 17, 2006 4:41 PM CDT
up
0 recs
My thoughts: rank the season, not the team
--"Rankings that attempt to predict the final wins and losses of the season" (schedule matters)--
This method has a certain appeal in being able to match preseason rankings with postseason rankings. It also admits up front that when Florida loses a couple games this season, that should and will hurt them in the rankings. Yet this method, if a poll-voter claims to use it, at best disguises some "real" system that they're also using: even if someone professes to rank based on wins and losses, and is attempting to factor in likely outcomes based on W's and L's, that voter still probably wouldn't rank a 12-0 UC Davis team above an 11-1 Florida team. Or alternately, if that voter did think UC Davis was the best team in the country, the flaws of that system are evident.
Once it's clear that even a stickler using this method wouldn't rank undefeated 1-AA teams above 1-loss teams from BCS conferences, it becomes even harder to defend this method for preseason rankings. Let's say you used your crystal ball and determined that Florida will end with 2 losses, Texas with 1, West Virginia with 0, etc. You're absolutely certain that will happen, based on rigorous analysis of the teams and their schedules. Now what? Some people describe this camp as trying to "predict" the rankings, so I assume what they mean is they're trying to predict what the AP / ESPN / CFN / SI / etc. polls will say if the records are as they have predicted they will be. That's the biggest problem I see with this system: you're making your poll to predict the outcome of "the polls." That's the sort of system that recycles the big name, long history programs at the top of the polls: "there's no way a Notre Dame team with only 2 losses won't be in the top 5." Well, maybe so, but if your poll is designed to just predict what the other (uninformed) voters are going to do, why have a poll? A poll using that system does nothing to advance interesting discussion about college football, and certainly doesn't add anything useful to the decision-making process of which teams go to which bowls. All it does is send the Irish to yet another bowl loss, as everyone predicted that everyone else would rank them pretty high.
--"Ranking the team based on how good it is today, regardless of schedule, previous losses, etc."--
Especially in the preseason, this approach strikes me as more intellectually honest. It's not a prediction of anyone else's predictions (polling based on what one expects other polls to say) and it doesn't reward scheduling quirks that don't have much to do with the the way the team plays on the field.
However, as the season goes on, it becomes less appealing to use this approach. There's often every reason to believe a team is better than a team it just beat, but it doesn't feel very fair to rank the losing team much higher. It's also not really the point. the other Andrew makes this point above with Texas-USC, but a less emotionally charged example makes it clearer: after Villanova beat Georgetown in one of the biggest upsets in NCAA basketball history, they deserved to be ranked #1. But any honest observor would have to admit that if they played that game 10 times, Georgetown would beat Villanova in 9 of them. Even after the loss, Georgetown was the better team. But polls aren't the same thing as Vegas odds - an attempt to pick a winner in every game. The Georgia example from above is very relevant: if the dawgs really start to come together at the end of the season, but picked up 3-4 losses along with the way, it doesn't feel right to put them in the top 5 or top 10. The opposite example is even more relevant: if Vince suffered a season-ending injury in the Big 12 title game last year, would you drop the Longhorns a few spots in the rankings? Surely on that day, with no real backup QB ready to replace VY, the team isn't nearly as good. But it's not right to drop them below a 1-loss PSU team or an Ohio State team that was almost as good as Texas in the opener but lost because of the now unavailable VY.
In a sense, these are things that voters in the polls already take into account, but I think it's important to define what they're doing - and why. I say we rank the season instead of the team. This is most obvious around bowl time: we look at a team and decide not so much how good a team is that day (saying things like "how injured are they?" or "I really think they'd win a rematch with the 3 teams they lost to early in the season") but what that team has accomplished during the season in question (saying things like "they lost three games, but two of the losses were to 10-win teams" or "sure, they racked up 10 wins and played USC close, but what good team did they actually beat?").
This approach also splits the difference in the preseason: it allows a poller to predict how impressive a season each team will have. Each voter still has to make a judgement call as to which of many different schedules with different opponents is the most impressive, but should ultimately defend his or her rankings by saying, "Team X proved it had the best season in the following ways." The reasoning could be a season full of 40-point blowouts by West Virginia, showing that even if the schedule was harder, they still would have dominated; or it could be a series of 1-touchdown victories and losses by Florida, showing that they beat some of the best teams in the country and no one could blow them out under any circumstances. What this method precludes is pointing to an early game and saying, "this is a whole different team now that it had the season to mature." It may be true, but given the limited season of college football, we can only ever fairly evaluate a team based on its actual results. (You can, however, take into account how close a win or loss was, or if the opponent played a phenomenal game that day). Losing to a couple mediocre teams early in the season means that overall, your season wasn't as good as the season of the team that played a similar SOS but won all those games.
I'll hold off on posting my actual top 10, lest that draw more fire than the theoretical bit above, which is more carefully formulated anyway. Thoughts?
by Calfan on Jul 18, 2006 11:03 AM CDT 0 recs
wow
Although it does, in a way justify, at least in one way, boi from troy's ranking 'SC #1 after the rose bowl, which, much as I hate to admit it, was completely against rationale.
by USCLink on
Jul 18, 2006 11:15 AM CDT
up
0 recs
you think?
by Calfan on
Jul 18, 2006 11:45 AM CDT
up
0 recs
I dont see the justification either
by Wells on
Jul 18, 2006 11:47 AM CDT
up
0 recs
Ah
Oh well...this has not been a great day for thinking, my apologies.
by USCLink on
Jul 18, 2006 1:17 PM CDT
up
0 recs
the fundamental flaw...
In most cases, (well hell, in every case except for Div. 1A football) the issue is settled on the field/court. So nobody really cares that Villanova couldn't beat Georgetown again, because they won when it counted. So the national champion isn't the best team, per se, just the one that won the championship.
In Div.1A college football, you don't really have a champion, you have a team (or teams...) that in some strange, subjective mix of popularity, bias and actual competition is deemed "The Best", and therefore called a champion. In rare and beautiful cases, and yes, last year certainly springs to mind, everything comes together and the "Champion" actually is "The Best" (and here I'd tell boi to bite me, except for the obvious inherent danger in that...). Mostly, though, it leads to fiascos like BYU or Onepeat or Ara "Take a knee, let's just vote on it!" Parseghian.
Anyway, back to that strange, subjective mix - aka The Rankings.
The problem with trying to select the "Best" without regard to schedule or outcome is that you've got douchebags who actually think their opinion supercedes actual real-life events, like: scoreboard, biatch!!!
The problem with the other approach is that the predictive nature of the beast becomes hardened into fact all too easily. This is particularly true in these preseason polls. For example, say I'm using that approach and trying to make my preseason list look as much like the final list as possible. So I look at a team like WVU, particularly their schedule, and I put them way up there, because they'll be much less likely to have two losses than Florida or Auburn. So I rank them on up there, above teams which are actually "better", but who'll play tougher schedules.
What I've actually done, though, is put Flor-burn in an even tougher position, because now, in addition to playing tougher teams, they need to somehow leapfrog WVU, who's busy playing teams that won't likely help the Flor-burn cause much. Now if Flor-burn is beating a series of good teams, but staying behind the Mountaineers, that's not really fair, but, of course, if the voters go ahead and leap Flor-burn ahead of WVU after a WVU win, couches will burn. You can still see the moral equivalent of this if you poke a Cal fan two years later.
The real solution, of course, is to actually settle it on the field in a playoff, involving enough teams to minimize the ranking biases. Enough money seems to be flowing into enough pockets (and power, don't forget power. The delicious taste of control is very hard to give up, say the BCS powers...) that that solution won't happen soon. But just please, dear God, make it in my lifetime!!!
Failing that, though, it seems to me the best alternative is to make the preseason (and maybe even the first couple of weeks') rankings based on who's "better" in a perfect world environment. Then, a third of the way through the season, switch to ranking the teams based solely on outcomes. In that scenario, WVU will have to leapfrog some "better" teams, rather than vice-versa. Not perfect, but better than the other way around.
by agent orange on
Aug 18, 2006 11:57 PM CDT
up
0 recs










