PEER (Production in Equal Equipment Rating)

LewTheShoe

LewTheShoe

Seeking Skill-based Meritocracy... More HP Less DF
Joined
Apr 21, 2016
Messages
4,625
Points
593
We have endless debates and arguments about various driver's role versus the quality of his (or her) car or team in determining performance. It is often frustrating because it is impossible to know just how good each part really is. We all have our own individual gut feel, but not much else, in most cases.

I subscribe to Motorsports Analytics, a website created by David Smith that seeks to provide data-based insight into Nascar performance (www.motorsportsanalytics.com $4.25 per month). It is sort of like the sabermetrics of Nascar. The flagship metric of the site is Production in Equal Equipment Rating or PEER. Smith has a proprietary, non-disclosed model that attempts to strip out car performance and team performance, thus leaving pure driver performance in a hypothetical scenario where all drivers have equal equipment and teams. Each driver's actual results are filtered through a handicap, and the drivers having the best cars and best teams get the highest handicap.

Like WAR (wins above replacement) in baseball, all one can hope for with PEER is some degree of perspective or insight. To ask for absolute truth is too much, IMO, because all parts of Nascar performance are interconnected. Also, the model is confidential, so no way to judge whether you believe it is valid or not. Regardless of these shortcomings, I find Smith's PEER numbers interesting. Here is where PEER rankings are for 2016 after 20 races:

Serious Title Contender (PEER Above 3.00)
B Kez (for 2015 Logano led the PEER standings)
Kyle B
Harvick

Fringe Title Contender (from 2.99 to 2.00)
Kurt B
Logano
Edwards
Smoke (11 races)

Serviceable (from 1.99 to 1.00)
Matt K
JJ
Hamlin
Truex
K Larson
Dale Jr.
Chase E
Bowyer
Stenhouse
Newman
A Dillon

Competent (from 0.99 to 0.00)
M McDowell
Biffle
T Bayne
KK
The Dinger
Blaney
Jamie Mac
L Cassill
Menard
Almirola
C Whitt
DiBenedetto
C Buescher
Regan Smith
David Ragan
M Annett

Replacement Level (PEER below 0.00)
Josh Wise
Reed Sorenson
Jeffrey Earnhardt
Ty Dillon (8 races)
C Mears
B Scott
Danica
 
Wow, truex is that far down? I'm watching the replay of today's HUBcast and pretty much everyone is picking him to be in the final 4.
 
Acs said:
Wow, truex is that far down? I'm watching the replay of today's HUBcast and pretty much everyone is picking him to be in the final 4.
Click to expand...
I'm not gonna take on the role of defending the model, but I'd guess David Smith would say a "serviceable" driver with a really fast car is a formidable combination. (I also will say I have generally considered him as a journeyman driver, at least up to the point when he had that sustained hot streak last year.)
 
LewTheShoe said:
We have endless debates and arguments about various driver's role versus the quality of his (or her) car or team in determining performance. It is often frustrating because it is impossible to know just how good each part really is. We all have our own individual gut feel, but not much else, in most cases.

I subscribe to Motorsports Analytics, a website created by David Smith that seeks to provide data-based insight into Nascar performance (www.motorsportsanalytics.com $4.25 per month). It is sort of like the sabermetrics of Nascar. The flagship metric of the site is Production in Equal Equipment Rating or PEER. Smith has a proprietary, non-disclosed model that attempts to strip out car performance and team performance, thus leaving pure driver performance in a hypothetical scenario where all drivers have equal equipment and teams. Each driver's actual results are filtered through a handicap, and the drivers having the best cars and best teams get the highest handicap.

Like WAR (wins above replacement) in baseball, all one can hope for with PEER is some degree of perspective or insight. To ask for absolute truth is too much, IMO, because all parts of Nascar performance are interconnected. Also, the model is confidential, so no way to judge whether you believe it is valid or not. Regardless of these shortcomings, I find Smith's PEER numbers interesting. Here is where PEER rankings are for 2016 after 20 races:

Serious Title Contender (PEER Above 3.00)
B Kez (for 2015 Logano led the PEER standings)
Kyle B
Harvick

Fringe Title Contender (from 2.99 to 2.00)
Kurt B
Logano
Edwards
Smoke (11 races)

Serviceable (from 1.99 to 1.00)
Matt K
JJ
Hamlin
Truex
K Larson
Dale Jr.
Chase E
Bowyer
Stenhouse
Newman
A Dillon

Competent (from 0.99 to 0.00)
M McDowell
Biffle
T Bayne
KK
The Dinger
Blaney
Jamie Mac
L Cassill
Menard
Almirola
C Whitt
DiBenedetto
C Buescher
Regan Smith
David Ragan
M Annett

Replacement Level (PEER below 0.00)
Josh Wise
Reed Sorenson
Jeffrey Earnhardt
Ty Dillon (8 races)
C Mears
B Scott
Danica
Click to expand...

Interesting and good thought fir conversation topic. But I think there are just to many variables and variable topics for any program to decipher.

In addition chemistry is hard to measure. One driver might be good enough to win a championship and the biggest races in a particular season with the right team. Then there is another driver also good enough to do the same, but each one has to be paired with the team that best compliments their driving style. And it is a different team needed for both of those drivers, and if you flipped those two drivers to the other team, they would perform worse in both cases.

In each case both have the skill sets and mentality to be the absolute best. But they also are at the mercy of some chemistry as well, some of which that may be out of their comtrol.
 
Again with driver and not the team . Even as a joke the writer doesn't understand the sport.
 
ted@economy said:
Again with driver and not the team . Even as a joke the writer doesn't understand the sport.
Click to expand...
OK, Ted, I'll bite. What is it about the sport that David Smith doesn't understand?

We all know that Nascar is a team sport. Does that mean that the abilities of individual team members should not be looked at? The race teams that hire Smith for that purpose (his day job since 2007 according to his website) would seem to think not.
 
LewTheShoe said:
OK, Ted, I'll bite. What is it about the sport that David Smith doesn't understand?

We all know that Nascar is a team sport. Does that mean that the abilities of individual team members should not be looked at? The race teams that hire Smith for that purpose (his day job since 2007 according to his website) would seem to think not.
Click to expand...

IMO most contributions of race team members can be quantified without the aid of what Smith provides. When you are dealing with a sport like the NHL advanced metrics can be a valuable tool to help in evaluating players but it is nowhere near the best. Two of the best net minders in the NHL were Grant Fuhr and Billy Smith (9 total championships) and the stats they posted would get you laughed out of the league today.

As far as what Smith may or may not know about Nascar is inconsequential to me but if he can make a living providing the info he does I applaud him.
 
Skoalbandit33 said:
As far as what Smith may or may not know about Nascar is inconsequential to me but if he can make a living providing the info he does I applaud him.
Click to expand...
LOL, so true about making a living.

I don't know how one would quantify those things without digging into data in a fairly rigorous way. I suspect most decisions about drivers, crew chiefs, engineers, and other team members get made using a lot of gut feel and educated guesses. Maybe that is good enough...?

Nobody needs advanced analytical metrics to see if Jimmie Johnson is a great driver. But how about Clint Bowyer, or Jason Allgaier?
 
IMO NASCAR is exactly the kind of sport that needs these advanced metrics like PEER. In no other sport does equipment and things outside the athletes control matter so much to the end performance. Heck we debate all the time on this board whether XXXXX was really that good today or whether his car was just a rocket, and this is exactly what this stat is trying to quantify. But nah we don't need no stinkin maths.
 
A team buys in in 2007. Pays the bill.

Word leaks out ... a couple of others follow suit. One of the 3 wins a couple of races in a row. A year later, everybody is being invoiced by Mr. smith on a monthly basis.

Auto racing 101. Smart guy.
 
I don't know anything much about Smith, and it pisses me off that he won't reveal much detail about his PEER model, but I agree with @Acs about the potential to shed light on certain murky corners of conventional wisdom by analyzing facts.

On here, we endlessly debate whether Kyle Larson is a hot shoe in sub-par equipment, or an over-hyped bust. But we do that without much actual knowledge. [FWIW, Smith is firmly in the camp of hot shoe with a middling team.]
 
Acs said:
IMO NASCAR is exactly the kind of sport that needs these advanced metrics like PEER. In no other sport does equipment and things outside the athletes control matter so much to the end performance. Heck we debate all the time on this board whether XXXXX was really that good today or whether his car was just a rocket, and this is exactly what this stat is trying to quantify. But nah we don't need no stinkin maths.
Click to expand...

Advanced metrics definitely have a bearing on many players in hockey both positively and negatively. There is nothing wrong with using all the tools in your tool box but I have seen analysis paralysis ruin businesses as an natural inclination seems to skew toward over reliance.
 
There is no legitimate way to do this that I can see. I say this as a person with a background in the same sort of econometrics that are used in baseball and other sports. We can compute Wins Above Replacement in MLB from available data-we can see the individual player's contribution and compare it to the average player. In NASCAR, there is no comparable, unless he is using teammates. Even then, we can't guarantee equal equipment/crew chief skill. If he keeps his methods private, that is another red flag.
 
29-4 said:
There is no legitimate way to do this that I can see. I say this as a person with a background in the same sort of econometrics that are used in baseball and other sports. We can compute Wins Above Replacement in MLB from available data-we can see the individual player's contribution and compare it to the average player. In NASCAR, there is no comparable, unless he is using teammates. Even then, we can't guarantee equal equipment/crew chief skill. If he keeps his methods private, that is another red flag.
Click to expand...

Well said as I think there are to many variables and unknowns in Nascar to draw the correct conclusion. Advanced metrics has a place but in one case where it was pretty well ignored was with my hockey team bringing back a former player as a free agent. His numbers are not that great but he is tough mentally and physically and is a very good locker room presence and a mentor to the younger players. Those are intangibles that just can't be measured.
 
I'm not trying to promote Smith's website, but I have found various articles from it rather insightful. For example, some crew chiefs have great numbers for gaining positions in the running order on green flag pit stop cycles, while others don't. Much of it has to do with identifying the appropriate time to short pit and the best times to stay out late. It may be meaningless in one race, but over a long season it all adds up. Smith would attribute those gains or losses to the crew chief, not the driver, I suspect.

He also does detailed analysis on restarts, and which drivers gain spots most often from the preferred groove (the low groove at Indy, for example) and from the non-preferred groove. Interesting reading to me.
 
LewTheShoe said:
I'm not trying to promote Smith's website, but I have found various articles from it rather insightful. For example, some crew chiefs have great numbers for gaining positions in the running order on green flag pit stop cycles, while others don't. Much of it has to do with identifying the appropriate time to short pit and the best times to stay out late. It may be meaningless in one race, but over a long season it all adds up. Smith would attribute those gains or losses to the crew chief, not the driver, I suspect.

He also does detailed analysis on restarts, and which drivers gain spots most often from the preferred groove (the low groove at Indy, for example) and from the non-preferred groove. Interesting reading to me.
Click to expand...
Did someone say you were?

Does Smith do this for drivers who run the B Series? I would that would be more useful than analyzing the performances of people already under contract.
 
aunty dive said:
Does Smith do this for drivers who run the B Series? I would that would be more useful than analyzing the performances of people already under contract.
Click to expand...
Yes, Xfinity, trucks, K&N.
 
29-4 said:
There is no legitimate way to do this that I can see. I say this as a person with a background in the same sort of econometrics that are used in baseball and other sports. We can compute Wins Above Replacement in MLB from available data-we can see the individual player's contribution and compare it to the average player. In NASCAR, there is no comparable, unless he is using teammates. Even then, we can't guarantee equal equipment/crew chief skill. If he keeps his methods private, that is another red flag.
Click to expand...
So what you're saying is, if the data doesn't give you an elegant and absolute proof, just ignore it and look strictly at wins, top-10's, and gut feel. Like the baseball world did for a very long time. You might be right, but I think there is more to be gained. I don't know that David Smith has all the answers, but maybe he has some of them.
 
LewTheShoe said:
So what you're saying is, if the data doesn't give you an elegant and absolute proof, just ignore it and look strictly at wins, top-10's, and gut feel. Like the baseball world did for a very long time. You might be right, but I think there is more to be gained. I don't know that David Smith has all the answers, but maybe he has some of them.
Click to expand...

I don't see a way to cleanly tease out driver effects in NASCAR/other racing (it's something I thought about a lot a few years back in grad school). If Smith would tell us what is in his formula, we could evaluate it and I might have to admit I'm wrong. But if it's just a "black box," I'm not willing to put value on it.

It appears that what the list shows is drivers that significantly outperform/underperform their teammates. I could say that I have a 'proprietary formula' to do this that consists of comparing the stats of a driver to his team average. I suspect that is why the model seems to perform poorly with the 'replacement' level drivers, for which it is far harder to make comparisons (is Jimmie Johnson going to do that much better in Josh Wise's car? I suspect not).
 
Last edited:
29-4 said:
I don't see a way to cleanly tease out driver effects in NASCAR/other racing (it's something I thought about a lot a few years back in grad school). If Smith would tell us what is in his formula, we could evaluate it and I might have to admit I'm wrong. But if it's just a "black box," I'm not willing to put value on it.
Click to expand...
Yeah, good answer. The key to what he's doing is a "handicap" for team quality. I don't need an analytical procedure to tell me that JJ's 48 team is better than Clint Bowyer's 15 team. But the 15 team versus the 47 team would be useful, if it were available and if one had confidence in it. And so it goes....
 
^ Further to your "black box" comment, the "handicap" on each car is not published. It exists, it is used in the driver PEER calculations, but it is not disclosed. I suspect this is to avoid having all the debate go there, but I'm just guessing on that.
 
aunty dive said:
I'd like to see the B Main list.
Click to expand...
The B Main list is published each January, a review of Smith's 75 top prospects, meaning not yet in full time Cup competition. Unlike the Cup drivers listed in my OP, the prospects are not all covered in a spreadsheet of data. This is because they are not all competing in the same place, and a spreadsheet can't mix Xfinity data with trucks data and K&N and ARCA data. Here is a free-to-all article covering the top 15 prospects before any 2016 races had flagged:

http://www.motorsportsanalytics.com/prospects.html
 
LewTheShoe said:
OK, Ted, I'll bite. What is it about the sport that David Smith doesn't understand?

We all know that Nascar is a team sport. Does that mean that the abilities of individual team members should not be looked at? The race teams that hire Smith for that purpose (his day job since 2007 according to his website) would seem to think not.
Click to expand...
As I see it those are team ratings not driver ratings.
 
ted@economy said:
As I see it those are team ratings not driver ratings.
Click to expand...
Not correct. Team ratings are here...

http://racing-reference.info/yeardet/2016/W

What I listed in the original post is Smith's analysis of how each driver would perform over 20 races if they all had equal quality of equipment and personnel, with the only difference being the 41 separate drivers.

You can agree or disagree, but it is not valid for you to change the definition of what motorsportsanalytics.com is presenting.
 
LewTheShoe said:
Not correct. Team ratings are here...

http://racing-reference.info/yeardet/2016/W

What I listed in the original post is Smith's analysis of how each driver would perform over 20 races if they all had equal quality of equipment and personnel, with the only difference being the 41 separate drivers.

You can agree or disagree, but it is not valid for you to change the definition of what motorsportsanalytics.com is presenting.
Click to expand...

I think Ted is referring to individual race teams, e.g. the #4 car, not Stewart-Haas. In that sense, Ted is correct. We can't separate Harvick's ability from his crew chief vs. Tony Stewart's ability from his crew chief with any sort of data currently available. The only way, I suppose, would be to randomly assign drivers to cars every week.
 
^ Sorry, but Ted is *not* correct if he is saying the ratings I listed in post #1 are team ratings. Team ratings per car are in the link I just posted...

http://racing-reference.info/yeardet/2016/W

These are per car ratings with nothing separated. If you reject the notion of separating driver performance from the team, use that link. Smith believes it is valid to analyze driver and team separately, and he came up with the ratings in post #1. Again, you can disagree with what he did, but you can't take his *driver only* ratings and just re-label it as a team rating.
 
If he keeps his methods secret and therefore not subject to "peer" analysis then it's highly likely it won't stand up to scientific scrutiny. Nice try though Mr. Smith.
 
You must log in or register to reply here.
Back
Top Bottom