Saturday, February 13, 2010

Say It Ain’t So, Nate

>


-by Doug Kahn

Nate Silver wrote this on December 23rd (2009's Most Valuable Democrat Is...): “What makes a congressman valuable to his party? One fairly intuitive answer is that it’s someone who votes with his party on key pieces of legislation more often than a typical congressman from his district would.”
I have, therefore, compiled roll call votes on ten key pieces of legislation -- in my opinion, the ten most important pieces of legislation -- that came before the House of Representatives this year. What I then did was to run a logistic regression for each vote, comparing each representative's vote to his predicted vote based on his district's PVI. For example, a congressman in a district with a PVI of R+6 had a .37 likelihood (37% chance) of voting for the stimulus package. A congressman from such a district who voted for the stimulus package would be rated positively for his vote: specifically he'd receive a score of 1 less .37, or +.63. If the congressman voted against the stimulus package, on the other hand, he'd receive a score of -.37. I then added up each representative's score across all 10 votes.

 
Nate is describing a fantasy situation. The premise is wrong in so many ways that it would take a book length essay to explain why. This seems intuitive to him? Then Nate doesn’t understand what it means to be a Democrat, especially a progressive Democrat.
 
There’s no such thing as a typical congressperson, in the way there might be a typical baseball player. There’s no such thing as a typical district. A district’s PVI is not a real measure of how politically progressive a district is. ‘Key legislation’ itself is an arguable concept. And the Democratic Party exists for principled reasons, not to rack up ‘wins’ on pieces of legislation.
 
Nate is using an inside baseball analysis; how well does a particular player perform compared to the presumed performance of that player’s generic replacement. That’s useful in baseball, which is a business based upon judging and then hiring people who will help your business succeed, that is, help your team win, therefore fill seats, make your ‘brand’ more valuable.
 
Nate Silver is, arguably, the person most responsible for reforming the analysis of baseball prospects. You could say he helped make the business of baseball more scientific, using statistics and statistical analysis in innovative ways in order to better judge the relative ‘value’ of young baseball players.
 
His methods have changed the game of baseball. (That is, the way it’s played.) Winning the World Series is an accomplishment built upon discrete steps: to win games you have to score runs and prevent the scoring of runs. Baseball is a fantastically complex team sport; but what if you could properly judge each player’s individual contribution to a run scored or prevented, and then to a game won or lost? Then you’ve created a model for young players to emulate, and fundamentally changed the game. Nate did that.
 
Nate’s intelligence and success make him an influence on the opinions of many people. That could make his analytical tool worse than useless, it could make it anti-progressive. (Nate is not a progressive, of course, not by his own description.)
 
If Nate continues to publish this particular analysis, it may have a pernicious effect on Democratic politics. Nate’s ‘picks’ for most valuable Democrat will be quoting him in their efforts to raise money and get reelected, and that should make him be more circumspect in tossing off a mathematically compromised analysis like this. Look at it this way: if even I can figure out how bad it is, it must lack competence.
 
Nate’s analysis is limited to one year, but more important it’s limited to only ten votes. It’s limited to votes that Nate says are the most important ones. It’s limited to final votes (except in the case of Stupak), when often it’s a vote on the rules governing House debate that is the crucial factor, similar to a cloture vote in the Senate.
 
But besides that, there are many ways to manipulate the statistics involved here, and you can come up with any number of mutually contradictory results. This analysis has been done before, and done better, by people who are themselves progressives. It’s called ProgressivePunch, and they come up with rankings which contradict Nate.
This is pretty simple, really. Note that the method does not account directly for a congressman’s party. This is deliberate. It's not proper, for instance, to compare Stephanie Herseth-Sandlin, the moderate congresswoman from South Dakota, to a typical Democrat, or even a typical Democrat in a conservative district, because if she were to retire, we can't take for granted that a Democrat would replace her. In fact, in South Dakota, she would probably be replaced by a Republican. Is Herseth-Sandlin -- even though she breaks with her party somewhat frequently -- more valuable to the Democrats than a typical congressman from South Dakota would be? That's what we're trying to get at.

Not really. What we’re trying to get at, we progressives that is, is how valuable to the cause of equality, social equity, humanity is a particular Congressperson. In baseball each win is as valuable as any other win. In politics, in governance, each issue is unique, not comparable to any other. Politics is not a game. Human lives are at stake.
 
When someone like Stephanie Herseth-Sandlin is in the pocket of corporations, and actively campaigns to drag the Democratic Party to the right, she deserves to be defeated. I think she has to go.

Nate’s most valuable Democrat is Bart Gordon of Tennessee. 12 of the 25 most valuable, according to Nate, are Blue Dogs, including Frank Kratovil (MD-1). I won’t bore you with an exposition of the essential uselessness of these two. ProgressivePunch scores Frank as 248th out of 258 Democratic House members.
Although 12 of the 25 most valuable Democrats are Blue Dogs, so are 8 of the 21 least valuable ones. It’s short-sighted to lump the Blue Dogs together; they disagree on as much as they agree, and although some of them are among the most counterproductive Democrats, others are among the most worthwhile.

So Nate would say that I’m short-sighted because I ‘lump the Blue Dogs together.’ I don’t lump the Blue Dogs together, Nate, they do that themselves. Go look at their website. They join a caucus that espouses regressive views on social issues, and posit ‘fiscal discipline’ (which is so overbroad and undefined it means exactly nothing) as an ethical and moral compass, while supporting the Bush tax cuts for the wealthiest Americans.
 
What we really need to do is to deliberately defeat some Blue Dogs in November. So long as Democrats retain a majority in the House, this makes the caucus more progressive, and more important, it helps get more progressive legislation out of committee. The Blue Dogs think they control the Democratic agenda in the House. They need to be taught a lesson, one that all their members will think about when real Democrats propose legislation.
 
You Could Look It Up
 
A crucial point: Nate is scoring people who have ‘fixed the game’. The Blue Dog Caucus developed and publicized its own strategy on how to come up with a Health Care bill, and it included voting in a bloc in Henry Waxman’s committee to prevent the inclusion of a strong public option. They won, and the bill that came out was a shadow of what it could have and should have been. So what if they then voted ‘aye’.
 
Let’s strip down Nate’s analysis to its simplest iteration, one crucial vote and one Senator. The most important vote in the Senate last year was on Health Care. Nebraska, PVI -14, is the most Republican state with a Democratic Senator. Ergo: Ben Nelson is the most valuable Democrat in the Senate.
 
Nate: He’s not.
 
In the 2006 book called Baseball: Between the Numbers, Nate wrote a chapter that mathematically analyzed this question: was Babe Ruth the most valuable baseball player ever? It’s a mind-numbing statistical analysis. Utilizing math similar to his analysis here, Nate proved that the most valuable baseball player of all time was Barry Bonds. The numbers lie.

Labels: , , , ,

10 Comments:

At 10:59 AM, Anonymous Anonymous said...

I'm going to take a shot at Nate's baseball statistics. Yes, there has been the appearance of success for the revised statistics to look at, but winning teams at the end of the day still win on ERA and Quality Innings Pitched (that might be a Nate thing).

He might have helped guys improve defenses for sinker ball pitchers.

The problem with Nate's numbers is they don't take into account unbalanced schedules due to the shift to divisional play, and he makes some assumptions about inter-league play.

He does use a lot of numbers but he makes too many assumptions. Plus the NL sucked for most of Bonds career versus Ruth who played in the better league at the time. And the mounds were higher. The fields were larger. You can't discount that.

 
At 11:28 AM, Blogger Doug Kahn said...

Great stuff. You know way more about the stats than I do, so thanks for commenting. I wish I had emphasized your point about assumptions, because of course that's how his political analysis fails so badly.

How much higher were the mounds back then? Even an inch would make a difference.

More important: over the years, do the Phillies have a philosophy of getting guys who hit a lot of doubles? And do they tailor the fielding to defend against them?

 
At 11:49 AM, Blogger Harry Levy said...

Great post. I would love to see the stats on congress when they have a lobbyist in their ear when there voting. That makes it a bit different than baseball eh

 
At 12:21 PM, Anonymous Anonymous said...

They lowered the mound by 5 inches and shrunk the strike zone in 1969. Back in the 90's (I can't think of the year), Bud Selig sent out an edict demanding that both Leagues start calling the strikes zones the way the rule was written. The pitching in the AL was better at the time, so they were not as noticeably hit by enforcing the rules. Heres where it gets fun.

The NL Commissioner decided there weren't enough strikes and homeruns, so he sent out and edict demanding stars give preferential treatment. Lower strike zones for the bigger pitchers (guys like Johnson ballooned), and tighter zones for guys like Bonds.

This caused Selig quite a bit of a headache, so he and the two umpires unions made a deal so the umpires were no longer under the control of the two League but dealt directly with him. Now this matches up with the end of the steroids era, but the numbers in the AL haven't changed that much versus the NL which has seen lower home runs and less strikeouts.

Just to give an idea of how important the mound difference is, Carl Yastremski won the triple crown with a .301 batting average.

 
At 12:53 PM, Anonymous Anonymous said...

I wrote that first comment, but I think it shows the problem with all of Nate's numbers. His assumptions are awful in too many cases. You can't argue with the numbers, but he treats current prices for healthcare like they are television, ignoring monopoly control, lack of price transparency, and that you don't need a television.

Yes, the Senate bill would be wonderful if the current costs of healthcare were the result of natural pricing points. Thats not the case. You can't walk into a doctor's office and get an answer to "how much will it cost to cut off my arm?" The response will be "what kind of insurance do you have?" If you want to know what a tv costs, walk into the store and ask or go on-line and see what deals people are getting. You can't do that with healthcare, so the lessons from TV pricing don't apply.

 
At 1:28 PM, Blogger Doug Kahn said...

Yes, people act as though economics is completely theoretical, and doesn't apply to the real world. The consequences of concentrated ownership is finally coming to a head, with incredible price-gouging (Anthem Blue Cross of CA, among many others) and denial of coverage.

Something has to give, and despite the dismal performance of Congress, I'm predicting it will be the insurance industry. The companies are acting as they are required to by competition and market forces, and it's obvious to everyone that it's unsustainable. Society has to either control them or dispense with them. I favor the latter.

'Unsustainable' isn't just a talking point, in time it becomes a reality, when we reach some (unknown) political and social breaking point. I believe that it's almost certain to be this year.

 
At 1:29 PM, Blogger Doug Kahn said...

I forgot to ask: what did you mean about the defenses and sinker ball pitchers?

 
At 2:27 PM, Anonymous Anonymous said...

A lot of sinker ball pitchers produce grounders. Now one of the things Nate champions is what kind of grounders are they. Certain guys who have a lot of errors also turn 5 game winners into 15 game winners because they can field those grounders. Guys like Jeter don't make that difference. As long as Jeter is going to be at short, the Yankees will not aggressively sign sinker-ball-type pitchers or they are going to pay. When a guy like Nomar was in his prime, he made a lot of errors, but the pitches who were producing these ground balls saw a lot of their wins go up because Nomar was covering more ground. Guys who pitch sinkers produce ground balls as hitters go "fishing."

There have been a few guys who can't hit for anything that always start when a guy like Derek Lowe is on the mound because people are paying attention to their abilities.

 
At 10:54 PM, Blogger Bruce said...

Anon. Respectfully, your comment about Jeter doesn't hold water. Chien Ming Wang, a noted sinker baller before he got hurt, won 19 games 2 years in a row, leading his league in victories over the same 2 year span. The shortstop behind him was Jeter. Silver's stats don't take into things like experience, knowledge of specific hitters and where they tend to hit a specific type of pitch, and a player's positioning on the field based on that experience and knowledge. In that sense, players who are as smart as Jeter are their own statisticians and play their game accordingly. They also play it at a depth that a wonk like Silver will never have the sensitivity or life experience to understand. The problem with all stat-nazis is that they never consider the intangibles. Jeter makes plays even by being where he wasn't expected, as exemplified by his famous "Flip Play" against Oakland that saved the Yankees and propelled them into the series. Some guys are just winners and they go the extra mile. There's no stat for that. As for Ruth, or any player, I think the true measure is how he did compared to the next best players of his own era. Baseball is three-dimensional chess. Silver is one dimensional. Anon., you are more correct than he is when you mention things like unbalanced schedules, changing mound heights, etc. Silver is out of his league or depth. The list of nuances that he is seemingly unaware of is a lengthy one.
Now, when it comes to politicians, I'd rather see stats that correlate their publicly stated positions, their votes (not necessarily the same thing), and the amount of money they take in from K Street bribers. I'd also like to see a statistical measure of a politician's positions vs. their party's platform. For instance, the Democratic Party's platform that states it is in favor of Universal Health Care. Coulda fooled me! But, then, I don't generally believe politicians, which brings me to another statistical category: How about a stat that shows us the percentage of pathological liars that go into politics vs. other fields.
I'd also like to see stats for personality traits such as equivocation, spinelessness, and arrogance in politicians. The stat a superficial person like Silver should perhaps look at first, though, is the stat that shows how a politician can disillusion and turn off a voter or voters so much that they will stay home on the next election day. That one just might be the most UNvaluable person to their party.

 
At 7:29 AM, Anonymous Anonymous said...

In response to Jeter, his famous flip play was a result of being out of position, walking off the field and getting lucky.

Wang would be the Yankees top starter if he stopped getting hurt. He's a super-star without injuries. Jeter doesn't cover a lot of ground. Balls hit more than a few feet from where he is standing are going to be hits whereas you replace him with almost any other shortstop they have a shot at getting to them. Jeter does get everything he touches and never misfires, but his range is severely limited, making most sinker ballers limited players. The Yankees don't aggressively go after players like Derek Lowe (not a super-star, but a proven 20 game winner) because they know they have that weakness.

Look at their starters over the years and the guys the lavished money on who maybe didn't workout:

Mussina
Clemens
Johnson
Pavano
Pettite
Sabathia
A.J. Burnett

Hell, they even brought in Flash Gordon, the only Yankee Stephen King didn't hate.

 

Post a Comment

<< Home