When to remove the starting pitcher

I've always thought managers are slow to go to the bullpen. It seems like they usually wait for the starting pitchers to get into trouble instead of trying to get them out before trouble starts, so I decided to look at the data. My strategy was to compare the last inning of starting pitching to the bullpen's average numbers, and look for significant differences. Getting bullpen numbers for each team isn't easy, and I had to copy/paste each team individually from different pages at Baseball-Reference - not wanting to do this for a bunch of different seasons, I just used the 2009 AL numbers. I would speculate that the NL managers are a bit better at pulling the starter because sometimes they are nudged to do so when he comes up to bat. As it turns out, using only 2009 gives plenty of data to see that managers do not pull the starters in a timely manner. In fact, for all 14 teams there is a significant difference between the bullpen and the last inning of their starting pitching.

Rather than using ERA to compare SPs and RPs, I used FIP. I added HBP to BB in this formula. The additive term seems to have changed from 3.20 to 3.10 since I wrote my entry on closers. The advantage of using FIP is twofold: it takes a lot of the luck out of the equation, and actually predicts future ERA better than past ERA does; and inherited runners are not important because we're just considering HR, BB, K, and IP. It's easy to estimate the standard deviation of FIP because it can be written as a function of multinomial probabilities. This is important here because I want to be able to tell if the bullpen's FIP is significantly better than the starters' FIP.

I limited the data to pitchers who started at least 15 games (64 pitchers qualify), figuring by that point the manager should have a good idea of when the pitcher is tiring. Of course I have the advantage of looking at the whole season's data to see where the differences lie - at the beginning of the season, the manager may not know how good his bullpen will be, or how his new pitchers behave in the late innings, etc. But as we'll see from the huge differences - SPs should be removed sooner rather than later!

The following table shows the team's average SP FIP for their last inning in the second column, the bullpen's FIP in the third column, and the p-value testing whether RP FIP is less than SP FIP in the final column.

team	SP FIP	RP FIP	p-value
ANA	7.278	4.274	0.000
BAL	7.522	4.557	0.001
BOS	6.334	4.154	0.001
CHA	7.209	3.927	0.000
CLE	7.851	4.686	0.000
DET	6.173	4.666	0.024
KCA	6.737	4.586	0.003
MIN	9.148	4.322	0.000
NYA	5.438	4.329	0.044
OAK	6.265	3.349	0.000
SEA	5.673	4.352	0.046
TBA	7.011	4.487	0.002
TEX	6.402	4.057	0.000
TOR	8.500	4.211	0.000

Joe Girardi of the Yankees and Don Wakamatsu of the Mariners were the best at removing their starters before they got knocked around. But they still seem to leave them in too long, and actually they might only be the best because both teams had four qualifying SP, all of whom were pretty good - notice they have the best SP numbers of any teams.

Most teams actually have slightly better RP numbers than SP numbers (even when removing the last inning pitched for all the SP); I guess this is due to being able to throw harder when you throw fewer pitches (I've also heard pitchers are worse the second time through the order - something to investigate in the future). The differences are quite small, but still, if the managers are aware of this, maybe their bullpens are already operating at their innings limit, and can't come in one inning earlier. Or at least all their good RP are operating at their innings limit. I bet some of the difference that shows up in the table is unavoidable, but it sure seems like it would help to have your best AAA pitcher on the roster, instead of another backup hitter, to eat up some of the bad SP innings.

Next time I'll look at some of the pitchers who have the biggest dropoff in FIP from the first several innings to their final inning and at the ones who are the worst in their final inning. Without needing bullpen average FIPs here, I'll be able to consider several seasons at once.


The information used here was obtained free of charge from and is copyrighted by Retrosheet. Interested parties may contact Retrosheet at "www.retrosheet.org".