Pickwatch Q&A: Prediction Machine’s Paul Bessire

Welcome to the first of two interviews this week with some of the top ranked NFL experts this season. It’s rare that people get a chance to explain their approach to picks, so we wanted to ask a few of the more accurate analysts some questions that might give you an insight into their mindset.

First up is a scholar and a gentleman (literally), PredictionMachine.com’s Paul Bessire, whose consistently accurate ‘Predictalator’ has seen his site grow to be one of the most trusted picking related outlets around not only the NFL, but a range of sports. This year the Machine finished third overall in our straight up competition, so we dropped Paul a line to find out a little more about the methodology behind this success, and also find out just how the Predictalator works…

PW: Congratulations on another impressive season, third overall out of every expert we track. Given the proliferation of sites offering a ‘picking edge’, how important is it to be recognized as consistently reliable? 

PB: I think it is important because it goes hand-in-hand with our approach and our process. We take an objective, analytical stance that fully considers players, coaches, teams, officiating, weather and home field advantage. Assuming we appropriately stay on top of new rules, health, depth charts and player development, our accuracy in our predictions should match our confidence in those projections. 

Another way to say that is that our accuracy should be no less consistent than the true random turns of any given NFL week or season. When it comes to the top of the standings like these, I would guess that we would normally be in the mix with projections for any of the sports we would cover whether it be straight-up or against-the-number. 

After that, a lot of it has to do with luck and we were fortunate that the ball bounced our way in a couple critical instances to give us a top three finish (big props to the top two finishers, both of whom I have worked with at some point in my career and both of whom I respect considerably).

PW: Just to focus a little on the methodology, the machine seems pretty adept at getting results right. Can you give us a brief insight into how the system works and how/whether you make improvements during the offseason?

PB: Our mantra is that: “we play the game 50,000 times before it’s actually played.” We actually do this for every game in six different sports – NFL, college football, MLB, NBA, college basketball and NHL. For the NFL, we have the most thorough data and process.

On every simulated play, we account for the expected interactions of all 22 players on the field and layer in coaching, home field advantage (which impacts the outcome of every play and is unique to team, venue and situation), weather and other, similarly ancillary factors. By simulating the game that many times, we can assess the likelihood of just about anything happening in a game. 

This obviously includes a team’s chances of winning as well as winning by more than the spread or other items like projected stats or Super Bowl probabilities. Technology has certainly helped us over the decade that I have been doing this. It used to take me a half an hour to run one NFL game 50 times (not 50,000 times). Now it takes eight seconds to run all 267 NFL regular and postseason games 50,000 times and track as much data from those results as we can fit on the site.

PW: With that type of computer-based simulation, do upsets pretty much go out the window? Does that make it even more important to focus on the games that are harder to predict?

PB: It’s all objective to me. I don’t look at whether we have projected any upsets until after I run the games. Some weeks, we will project five or six upsets relative to the spread (though I don’t think we have ever had an NFL “upset” against a spread greater than 3.5 points) and some weeks we will not have any. 

As you may expect given my bent, I do not consider these projections as “upsets” though, since I believe our methodology is better at assessing a team’s true chances of winning than the spread (which needs to at least consider bettor bias as well). 

I do review, analyze and write about all of our NFL picks before they are published, but, honestly, my “focus” is more on the games where our projections vary most relative to the spread than simply what the consensus is for straight-up opinions. Also, the straight-up picks published on CBSsports.com are filed on Tuesday afternoon, long before I see what the consensus is on each game.

PW: has a pick ever come through and you’ve wanted to overrule it because your gut was telling you to take the other team? Quite a few games come out almost 50/50…

PB: The answer to the question is “yes,” but that has never happened in my stint with my own site When I first started doing this professionally, I did allow personal bias to make an impact (partially due to the fact that we were only simulating 50 times and randomness was a considerable driver in our official pick) in the projections. I learned pretty quickly, though, not to do that.Early on, my biases performed far worse than the technology. 

At this point, though, I have been so conditioned to the Predictalator’s approach that I rarely see a projection that surprises me and I have never changed a result in the four seasons this site has existed. 

An anecdote that relates to this stems from this year when we competed with a TV host/personality for a Man vs. Machine NFL segment each week. He got to pick three games for which he disagreed with our against-the-spread pick. He literally hand-picked them. We had no say in the matter. We went 31-13-1 ATS (started in Week 3) in those games where the human bias disagreed.

PW: in a situation like the one at the end of the season with Dallas losing Tony Romo to injury, how does Prediction Machine account for Kyle Orton at QB? Does it have a neckbeard calculator for example?

PB: No neckbeard factor, though maybe I should work that in because that was a different Kyle Orton than we expected. The answer to your question is simple. When I simulate the game, we load in our expectations for every player we anticipate playing. For Dallas, we removed Romo from the game and plugged in Orton to the simulations. 
This man is an NFL QB. 

The specific inputs for each player look at his strength-of-schedule-adjusted career stats (everything we have on him including college), age and experience to come up with our expectations for what he would do on per-play basis on an average team against average competition. 

From there, the actual team he plays for and against as well as the situations in the game itself dictate his impact on the upcoming game.

PW: Something that irked me recently was seeing a big name NFL expert quite clearly distance himself from his picks, in a way that seemed to totally ignore the fact that his reputation was partly built on his expertise in this field. Do you pay attention to any other so-called experts and their picks?

PB: I did not until I started following Pickwatch and, then, it was only to see how our projections fared relative to the others at the top with us. I don’t watch any pregame shows (aside from the aforementioned segment on which we appear), look at any polls or even try to gauge popular opinion relative to projections in any of the sports we cover. For the most part, that’s forced fluff designed to generate reactions rather than accuracy and, in that sense, I usually don’t find it entertaining (my point here is that I don’t find it a reasonable use of my time to watch ex-athletes make fun of each other for 60-90 minutes straight). 

That being said, I should note that there are a few radio and TV personalities (with John Halpin, Gregg Rosenthal and Ron Jaworski – all top five Pickwatch finishers – already at or near the top of that list) who are very good at taking more complicated topics and translating them so that the fans get the “so what” from any topic (this seems to happen more in the fantasy sports world where media has embraced analytical takes and non-athletes).

PW: Well on that topic, we’ve noticed that along with yourself and Accuscore, another ‘new’ media genre that are doing well are the aforementioned fantasy football analysts such as John, Dave Richard of CBS and a few others. Do you think there’s a case to be made that these guys and yourselves are a better source of information than someone who has played the game, but doesn’t necessarily have much of his reputation on the line each week? 

PB: While I know many of those associated with Accuscore, I can only speak for what we do. Due to health issues, my playing career ended relatively early (sophomore year in high school – I had played football every year from age 9-16). Still very much interested in staying a part of the game, analyzing football has been an interest, passion and focus of mine for 15 years since I stopped playing. 

My collegiate notebooks are full of 7-on-7 plays, all-time player rankings and new metric/ranking ideas for sports. I got my undergraduate and graduate degrees in Quantitative Analysis and every project I ever did in those classes was applied to sports, so this has been my profession for a decade now – It’s the only full-time job I have ever had. 

Though I played the game at various levels for much of my youth, I’m probably not a very good source to talk to about what it’s like to be an in NFL locker room. A similar conclusion could be drawn for a ten-year NFL vet trying to pick games… I am not saying that all former professional athletes are bad analysts. I have a respect for many of them. In many cases, they are not even expected by their employers to be good at picking games. It’s just not a necessity or even an advantage to have played at that level when it comes to predicting winners.

PW: Finally, what happens to prediction machine during the off season? I imagine its the equivalent of your holidays? I know on a personal level the season is physically and mentally draining… 

PB: Admittedly, physically and mentally draining can describe the football season, which, for me goes at full steam from a week before the college football season until Super Bowl Sunday (doing as many as 50 radio interviews across the nation each week). 

The NFL itself though, does not really have an offseason. We do a lot with the NFL Draft (including working with some teams) and I usually start working on player projections in mid-May. All the while, we will be reviewing our performance and looking into new styles and new rules to evaluate and improve the simulations for the next year (we do this for all sports we cover in their offseason). 

Plus, as a professor of Bracketology at the University of Cincinnati, March is even crazier and my real break does not really happen until the (thankfully now) four days off for the MLB All-Star break.

Thanks to Paul Bessire for speaking to Pickwatch’s Shaun Lowrie. You can find out more about the Prediction Machine and Paul at their website or on their twitter page. Yes, I’m counting the Machine as a person who owns part of their site…

Check back tomorrow for our interview with our regular season champ, FOXSPORTS.com’s John Halpin, and of course as always, keep your eye on the consensus NFL picks this week by checking our playoff picks. For more updates, follow us on twitter at @pickwatch.

About Shaun Lowrie

Shaun Lowrie is the founder and editor of Pickwatch, a site designed to give you a new perspective on experts in the NFL media by tracking their performance when picking games or predicting draft picks, team performance, or player potential. You can contact him at [email protected] or on twitter at @pickwatch

comments powered by Disqus