How Moneyball Got it Wrong!
“Putting it precisely, activating a resource and utilizing a resource are not synonymous.”
― Eliyahu M. Goldratt
“94% of problems in business are systems driven and only 6% are people driven.”― W. Edwards Deming
All organizations are complex systems. Baseball is a complex system. Making changes in a complex system requires four areas of consideration in what Dr. Edwards Deming called the System of Profound Knowledge.
Systems Thinking
Knowledge of Statistics
Theory of Knowledge
Knowledge of Psychology
Any attempt to change a complex system that doesn't consider all four lenses will most likely fail. Eliyahu Goldratt, the creator of the Theory of Constraints, calls these local optima that will rarely create global optimization.
Root of All Evil
This is not a story about money in Moneyball. It's all about transforming a complex system. The conversation over who and what changed baseball is not centered on the Oakland A's, Billy Beane, or Paul DePodesta in 2002; it's all about how the Boston Red Sox revolutionized the game two years later with John Henry, Theo Epstein, and Terry Francona.
They collectively understood that to change the system, you had to embrace complexity. The Boston Red Sox breaking the 86 year Curse of the Bambino is the real milestone that changed the game. The Curse of the Bambino started when the owner of the Red Sox sold Babe Ruth to the Yankees in 1918 to finance a Broadway play called No, No, Nanette. As Goldratt points out in his quote mentioned above, there's a difference between activating and utilizing resources. You could say that the A's activated the concept of Moneyball where the Red Sox utilized the idea. The A's owners were demanding drastic cost cuts. The Red Sox focused on utilizing a complex system to win a world series. The Red Sox Moneyball wasn't about saving "Money"; they had the second-highest payroll in 2004 behind the NY Yankees. What they did was create a system where they had a true north, "We'll do whatever it takes to win the world series." Theo Epstein was considered baseball's wonder kid when he took the VP and GM for the Red Sox. Epstein, unlike Beane, was a Yale graduate and a Juris Doctor ( Doctor of Law). Epstein later crowned the use of this system in 2016 when he again broke the actual longest baseball drought, the Chicago Cubs 108 year drought of not winning the world series.
Nothing Changed
There is no disrespect to Michael Lewis' Moneyball, but Billy Bean and his crew never got past the American League Division Series (ALDS - the first round of the playoffs). From 2000 (pre-Moneyball) to 2003 (post-Moneyball), the Oakland A's lost consecutive ALDS' each time, losing three games to 2. Furthermore, in 2001 (again pre Moneyball) the Oakland A's won 102 games with 60 losses, and then in 2002 (the Moneyball year), they won 103 games with 59 losses. Nothing changed pre-Moneyball or post-Moneyball. They won the same amount of games with or without Moneyball.
At the end of the 2002 season, John Henry, the new owner of the Boston RedSox, hired Bill James as a consultant. Henry had been a fan of Bill James long before he was a baseball owner. It was clear that the Red Sox were creating a holistic system where Henry, the owner, Epstein, the GM, and Terry Francona, the manager, were all on the same page regarding this approach that Bill James had created. Theo Epstein said of Bill James in a 2002 ESPN interview:
"What Bill offers us, more than a particular set of sophisticated statistical formulas, is a way of thinking. Bill doesn't start with an assumption and then find data to support it, like many people in baseball do. Bill starts with a question, and then he does the research objectively and doggedly, and lets the truth empirically come to him."
Quantitative vs. Qualitative
In 2002 Billy Beane and Paul DePodesta started with what would be called a deductive approach. They made challenging assumptions of how they were going to win more games with less money. Henry, Epstein, and James understood what Dr. Deming called the Theory of Knowledge (Epistemology). This was made evident in Epstein's ESPN quote about Bill James. James' approach was an inductive process of using the data to find the truth. Their discovered truth was they were going to win the world series. It's doubtful that the Oakland A's owners even knew who Bill James was in 2002, let alone understood epistemology. Epstein's ESPN quote about James describes what's called inductive reasoning.
Complexity
Ironically in 2003, just Oaklands famous Moneyball narrative, the A's again lost three games to 2 in the first round of the playoffs, but this time to the Boston Red Sox. Looking back at the 2002 A's, it's clear they didn't understand Systems Thinking. Meanwhile, John Henry acquired the Red Sox with an understanding that the whole system would have to work in concert to meet his one primary objective, break the 86 year "Curse of the Bambino." As an owner, that was his only true north. Henry's background of owning a financial trading company made him well aware of the ideas of systems thinking, statistics, probabilities, theory of knowledge, and psychology. Being new to baseball ownership, Henry brought in this as a unique philosophy. The same one that he had great success in the investment world. In the same ESPN article previously mentioned, here's what Henry said regarding this new system they were creating:
Usually, when making investments, it is implicit that investors believe they have some degree of knowledge about the future. So Wall Street has more fortune-tellers than any other industry. I feel I've had an advantage over the years because I am clear about a couple of things: 1) it is part of the nature of life itself (and markets are simply manifestations of people's expectations) to trend, and 2) I will never have a complete or full understanding of anything. Therefore, all investment decisions should be based on what can be measured rather than what might be predicted or felt.
Henry's situation with the Red Sox is not too dissimilar to Jeff Bezos, founder of Amazon. Bezos had never owned a bookstore or online store before moving out to Seattle to start his new venture. Also, like Henry, Bezos had a background in financial trading and understood a lot of the same principles that Henry understood. Leaders embrace these profound leadership qualities.
What's the Difference?
Steve Schott, the owner of the Oakland A's, was an accountant turned home builder. Schott and his partner Ken Hoffman put severe constraints on Billy Beane as it was pretty clear they knew nothing about complex systems concepts, systems thinking, statistics, knowledge, and psychology. They didn't have Henry's conviction regarding uncertainty as described in the ESPN article. On Wall Street, "Quants," quantitative analysts, don't make money because of their principles of certainty. They make money because they fundamentally believe, like Henry, in navigating through uncertainty. Schott and his partner had a very reasonable goal, which was to make money. Their approach may have been a good idea, but it wasn't an approach designed to win a world series. When baseball is only about money, baseball has lost. Even baseball's scorned previous Yankee's owner, George Steinbrenner, knew it wasn't about the money. Like Henry, when he acquired the Yankees, he proclaimed that he would be back in the world series in a few years. Steinbrenner's approach was a brute force strategy to spend as much money as possible. Steinbrenner, a shipbuilder, had way less finesse and practical analytics than Henry. Some say Stienbenner ruined the game.
The Oakland A's did increase ticket sales using Moneyball. However, at a rate not that much different from the rest of the league at the time. Boston and Oakland's revenue grew at basically the same rate during a ten-year period from 2001 to 2011. Again, then what is the difference? Boston won a world series.
Lean and Moneyball
When Toyota in the late 1970s started beating out GM, Ford, and Chrysler, they were beating them not because they were trying to save more money. Taiichi Ohno, the father of Toyota Production Systems, once said, "We don't make great cars; we make great people who make great cars." Toyota invested in all of the facets of its complex system. In contrast, the famous General Motors chairman Alfred Sloan once said, "The point is that General Motors is not in business to make cars, but to make money." Ohno's focus was a system that made great people. Sloan's focus was all about money. This was the same contrast between old school baseball and the Red Sox new school baseball. The Red Sox were Toyota, and the rest of Baseball were GM, Ford, and Chrysler.
Strongly Held Biased Culture
He tried to work within his system to Beane's credit, but he could only activate a locally optimized approach within the Oakland A's organization. He was never allowed to create a globally optimized system. There was never a top-down approach where everyone was on board. For example, Art Howe, the manager of the A's, and most of his staff never really agreed with Beane's Moneyball approach. When they acquired the extremely popular David Justice from the Yankees, they never told him about this new Moneyball approach. He resisted and didn't buy in for the first half of the season. If you remember, David Justice was hugely popular in the 90's being married to actress Halle Berry. In 2002 the Oakland A's were not all on the same pages when it came to this new way of using statistics to analyze Baseball. Art Howe and his gang were still relying on a form of baseball knowledge that had been in place for over 100 years. A tribal kind of knowledge of years of human instinct, which amounted to just a firmly held biased culture.
The Yankees Lost Too
The only thing Lewis' Moneyball got right about the Oakland A's 2002 novelty was that statistics and data analytics was a new linchpin that was changing the game of Baseball forever. However, not as a tool, but as a system. The Red Sox knew that Oakland didn't understand that it wasn't about total cost per game won; it was about "Winning the World Series." Lewis' Moneyball ordained Beane and DePodesta as the new masters of the universe in 2002. However, he and I would argue everyone else missed the point. Yes, the A's and Yankees won the same amount of wins in 2002, and the A's had a significantly lower payroll. However, Baseball isn't about comparing cost-to-win ratios; it's about flat-out winning. Counting your wins is not winning. The true tale from 2002 is that while the A's and Yankees had identical records in the regular season, neither made it past the first round in the postseason.
Lower Case "m" Moneyball
Baseball changed because of Bill James' and the system he called Sabermetrics was eventually used to obtain the ultimate prize in Baseball. The Red Sox won the world series in 2004 not because they had an optimum per-win cost. Their player payroll was the second-highest behind the Yankees at 127 million. The actual value of using a statistical system in Baseball is that you only have nine active players at one time. To win a world series, you must ensure that all of those positions are globally optimized. Lower case "m" Moneyball is about putting the best player on the field regardless of the cost. Upper case "M" Moneyball is about saving money while trying to win games. Baseball's top franchises had been in salary wars for 30 years in something called Free Agency. Again ironically, Free Agency started in 1974 when the Oakland A's Cy Young Award winner Catfish Hunter became the first free agent in Baseball. In true A's fashion, the dispute started between the A's owner Charlie Finley and Catfish Hunter. Finley refused to pay Hunter a portion of his owed money.
One of the contributing factors to changing Baseball in 2004 was that the Red Sox were the first to inject statistical systems into the free agent salary wars. In other words, the Red Sox were still paying high salaries like they were in 2002, but now they were adding statistical analysis into the equation for optimum results. They learned that you needed both to win the world series. It was the Red Sox who defined this new way of winning. The Red Sox didn't just create a system to get into the first round of the playoffs and then lose 3 years in a row. They made a system to win the World Series.
Let me be crystal clear; the Oakland A's didn't hire Bill James when he was available to be hired. No one hired Bill James until 2002 until the Red Sox did. The Oakland A's still haven't won a world series since 1989. On the other hand, the Red Sox hired Bill James and won a world series less than two years after they hired him. The Red Sox even tried to hire Billy Beane, and he turned them down, but they still won the world series in 2004 without him.
Tale as Old as Time
This is a "tale as old as time" where over the past 100 years, organizations and industries of all kinds continually miss an opportunity to use statistical systems within complex systems. They rarely delve into the realm of causal relationships to change human behavior. There's a famous saying, "correlation does not imply causation." This idea was expressed by the British statistician Karl Pearson, where sometimes data can be misleading and lead us to mistaken causation. For example, thinking that the cost per game won was somehow better than just games won.
Henry bought the Red Sox in 2002, and he understood what Bill James was doing when he hired him. He didn't have a bias against James like everyone in Baseball had. He hired James before Michale Lewis' Moneyball had even been published. As a new owner, Henry said after he hired Bill James, "I don't understand how it took so long for somebody to hire this guy." You could imagine Jeff Bezos saying something like, "I don't understand how it took so long for somebody to sell books on the internet" in 1996. Henry hired James and used him as an asset to systematically change the organization head to toe.
Ford Motors and Moneyball
This is the same story of what happened 20 years earlier with Donald Peterson, then chairman and CEO of Ford. Peterson found his Theo Epstein sort of speak when he tried to hire Dr. Deming. Deming told Peterson that he would teach Ford Motors a new way of manufacturing quality by completely changing their system with a statistical approach. This approach was something Deming had been practicing and preaching for 60 years, even while western economies continued to ignore him. It was the airing of the NBC documentary "If Japan Can Why Can't We", in 1980 that awoke the American car manufacturers and created a Bill James moment. Peterson watched Deming being interviewed in the NBC documentary and realized that he needed Dr. Deming to come in and change Ford's system. Peterson was the first of the Big 3 American car manufacturers to hire Deming. You can also hear an echo of Peterson saying, "I don't understand how it took so long for somebody to hire this guy." Deming refused Peterson's request a couple of times until Deming was sure they would accept a completely new system and not just parts of his system. Deming's approach was based on a design created in 1924 by Walter Shewhart. Shewhart being kind of the original Bill James if you will. It's not likely that Columbia Pictures will ever make a movie about Dr. Shewhart and Dr. Deming.
Sabermetrics
Bill James, another midwesterner like Shewhart and Deming, got his degree from the University of Kansas. As a VietNam veteran, James didn't find many job opportunities, so he took a job as a night-shift security guard at a pork and bean cannery. It was there he started codifying his ideas about how Baseball was stuck in an Availability Heuristic. Baseball was using a system that was based on stats, not a statistical approach. Stats is a short form of analytics and decision-making based on what everyone already knew. An example of a stats vs. a statistical system is, in Baseball before 2004, if player A has more hits than player B, player A is more valuable than player B. Just that simple. James started compiling a vast dataset of statistics in something called Sabermetrics. Sabermetrics is an empirical analysis of Baseball.
James compiled over 30 tables and a couple of hundred advanced metrics dating back to the late 1800s. As part of his empirical analysis, he invented new formulas, ones that Baseball ignored for years. Baseball, for an extended period, considered James, a pariah. A great example of James's Sabermetrics approach is where legacy baseball experts would put great emphasis on the number of hits a player got, not how many times they arrived on base. James invented a new metric called "On Base Percentage" (OPB), including walks and hits. The key here is that before James' system was in place, a player's worth was based on hits, not the number of times they arrived on base. This is an excellent example of breaking down another false causal relationship. A foundational part of Baseball is to get as many players in scoring positions as possible. Legacy baseball believed that getting a hit was worth more than a walk. What James did was break that causal relationship with his OBP metric. Getting in scoring position is the key; a systems approach doesn't care how you got there. James created a system where statistical analysis could be used as a strategy to change the human behavior of Baseball. He taught Baseball how to make more insightful decisions based on the data.
Walter Shewhart
This idea is something Walter Shewhart was famous for when he used statistics to change the human behaviors at Bell Labs in the 1920's creating telephones. Shewhart's Statistical Process Control, like James' Sabermetrics, changed manufacturing forever. Before Shewhart's new system, an acceptable way of testing the quality of telephones and telephone equipment was to allocate 10% of the manufacturing workforce to manual testing. Shewhart was able to eliminate most manual testers or move them into more meaningful jobs by using statistical analysis.
Like Shewhart, James saw things that others weren't seeing. Baseball was stuck in its tribal ways of thinking, where it didn't delve into understanding causal relationships. Before 2004, major league baseball's success was based on what Daniel Kahneman calls System 1 thinking (Thinking Fast). Kahneman's book, Thinking Fast and Slow, describes a bais called "pervasive optimistic bias." This bias generates the illusion of control. Remember Henry's ESPN quote, the second reason,
"I will never have a complete or full understanding of anything."
Baseball before 2004 was stuck in a "pervasive optimistic bias" loop. Baseball's even worse bias, however, was something called the Availability Heuristic. Basically, mental shortcuts create an illusion of validity.
Thinking Fast and Slow
Kahneman's work can be traced back to how organizations deal with the basics of causal relationships. One of my favorite stories from Thinking Fast and Slow is Julie's GPA. Kahneman asks, what is Julie's grade point average (GPA), given that she was a fluent reader since she was four years old. A typical System 1 casual response might be in the vicinity of 3.7 or 3.8. However, if asked the same question a little differently, what Big Ten university basketball scholarship did Julie get given that she was a fluent reader since she was four years old. Our System 2 response would probably kick in, and we would say that's a stupid question. The critical point is that over time, our System 1 brain stores situations as repeatable patterns. Chris Argyris, another behavioral expert, calls this the Reflective Loop in his "Ladder of Inference" concept. When assumptions are never challenged they become conclusions and ultimately beliefs. Who knows what Julie's GPA is 14 years later when she's about to graduate. The "she-reads- a-lot" metric is an insufficient element of analysis. A Bill James approach to Julie's GPA question might have been to add a new metric that takes the number of books Julie reads divided by the number of classes she attended. Sort of like an OBP for books read to attendance ratio. She might have been the most incredible reader in the world, but if she missed 30% of her classes in her 14 years, she would be unlikely to get better than a 3.0 GPA at best.
Deming's 70 years of learning was codified in his last book called New Economics, where he talked about something he called the "Theory of Psychology ." This is one of the four lenses of his System of Profound Knowledge. Dr. Deming also drew on a large body of work in Epistemology, as previously discussed. Deming postulated that to break down complex systems, you have to use multiple lenses to understand how to create change. Baseball, like all systems, is a Complex System!
Alphago
In December of 2016, Google challenged the greatest Go player in the world, Lee Sedol. The match was being compared with the historic chess match between IBM's Deep Blue and Garry Kasparov in 1997. An opening move in Chess has 20 possibilities, and in Go, there are 361 possibilities. After the first two moves of a Chess game, there are 400 possible moves. In Go, there are close to 130,000. IBM's Deep Blue was able to beat Kasparov with a brute force algorithm. For Google to beat Lee Sedol in Go, they needed to use a different kind of Artificial Intelligence (AI), something called Deep Learning. In Go, the number of possible moves is greater than all the atoms in the universe. Traditional AI algorithms can't compute that high. Deep Learning is a form of AI that learns like a human and therefore doesn't have to use a brute force computation. One of the most interesting moves in Google's "DeepMind" (Alphago) challenge match against Sedol was "Move 37." This was the 37'th move in the match. All the experts watching and commenting on the match appeared to be a wrong move by the computer program Alphago. Actually, "Move 37" turned out to be Go's Moneyball moment. Go is a game with white and black stones and whoever has the most amount of stones wins the game. The greatest Go player in the world, Lee Sedol, had what every other human had, a blindspot, that Google's Alphago program didn't have. Sedol saw the game in black and white stones, and Alphago was colorblind. The move that Alphago made was based on a non-human calculation that the game only needs to be won by a ⅕ point. Even Sedol, the greatest Go player, couldn't see or make that calculation before the move. Alphago wasn't blinded by a physical board and the colored stones that were allocated on the board. Right after "Move 37", the second highest-ranked Go player in the world in a commentary, declared that Alphago had made a mistake. He was the one who helped train the Alphago program. Sedol immediately understood the impact of the move. He was so overwhelmed by the move that he left the room for over 15 minutes to collect his thoughts. That move changed a human heuristically based strategy that had been in place for over 2500 years on how the game of Go was played in the future.
Nathan's ERA
Let's have some fun with some more casual thinking, with something I'll call "Nathan's ERA", kind of like Jule's GPA. Nathan Eovaldi is a 31-year-old pitcher for the Boston Red Sox. His career ERA is 4.26, and halfway through the 2021 season, his ERA is 4.11. That puts him about 50 on the list of the best 2021 pitchers. An ERA is a commonly accepted metric for evaluating pitchers. The Earned Run Average (ERA) is a percentage of the number of home runs over innings pitched. System 1 thinking before 2004 would have considered Nathan Eovaldi a good pitcher but not a great pitcher. However, Nathan Eovaldi in 2021 is the third highest-paid player on the Boston Red Sox, primarily because of Bill James' influence. Baseball can now use another metric in evaluating pitchers, something called Fielding Independent Pitching (FIP). FIP removes the things that are out of a pitcher's control. Things that ERA didn't consider, like fielding errors and missed plays. Nathan Eovaldi's FIP is incredibly low at 2.63 and 7th lowest out of all the 2021 pitchers. Before Bill James and the Red Sox 2004 influence, Eovaldi might not have been considered a highly paid pitcher.
Let's assume a team evaluated Eovaldi with an incredible lineup of leaders and an all-star team of Golden Glovers. Before 2004, they might have overlooked Eovaldi as a possible top franchise pitcher. They might not have considered that he might be coming from a team of terrible fielders who made a lot of errors. The FIP metric would have told them that he would have been a great fit on their team.
Most World Series Wins in the 21st Century.
Boston continually plays lower case "m" Moneyball by not trying to cut corners on salaries. For example, Nathan Eovaldi makes 17 million per year. They play lower case "m" Moneyball because they want to win the world series consistently. They have won the world series three more times since 2004, making them the most successful franchise in the first part of the 21st Century. That's Moneyball.
Love of Least Squares
Lower case "m" Moneyball comes down to understanding complex systems and causal relationships. Walter Shewhart invented the idea of understanding cause using statistics. In his Statistical Process Control (SPC), he defined two types of causes. One he called Assignable cause, and the other he called Random cause. Deming later renamed these to Special and Common cause. Shewhart's statistical process completely removes the human element from the equation, i.e., false correlation. Special cause identifies outliers, and Common cause uses statistics to understand behavioral performance. I have found no evidence of Bill James using SPC; however, he clearly understood the pitfalls of causal thinking, like Shewhart. There also is evidence that James and Deming shared love of Least Squares (a form of linear regression). Probably James' most exotic metric is the Pythagorean Expectation, a probability metric predicting how many games a team should have won.
In the end, you could say that Bill James and Theo Epstein made the most significant impact on Baseball, very much like Shewhart and Deming changed almost every manufacturing economy and some would say knowledge economies as well. Included but not limited to telephone manufacturing, USDA crops predictions, US Census sampling, Toyota and Lean, and the whole American manufacturing mindset in the 1980s. I would add DevOps and DevSecOps. Consistently overcoming bias, availability heuristic, and just plain old tribal knowledge while always giving way to scientific thinking and effective systems thinking to change human behavior. Some would say that is...
A System of Profound Knowledge.
--- Disclaimer ---
No “R” code was harmed in the making of this blog.
I am a lifelong Yankees fan, not a Red Sox Fan.
Despite the tone, Michael Lewis is one of my favorite authors.
