Turns out robot umpires have one big flaw: They’re too perfect

Late into a Pennsylvanian summer night, with the score tied at 7, the Lexington Legends' Jordan Pacheco takes a 1-2 pitch low and away without a second thought. But then, the umpire rings him up. Pacheco folds in two, grabbing his knees in disbelief. The missed call is so egregious it takes the announcer 20 seconds to realize what's happened.

Like thousands of hitters before him, Pacheco turns back to argue his case. "It was just a heat-of-the-moment thing," he tells me when I call him in July, two years after he shared the video clip of the call to Twitter. "I was like, 'You can't call that a strike!' And he was like: 'I have to. I'm sorry. You can talk to the technician who is in row 17 watching the game right now. You can yell at that guy. But don't yell at me.'"

See: Pacheco's final year as a player happened to coincide with an early test of Major League Baseball's automated ball-strike system, aka the robo-ump. The man behind the plate that night was only the messenger. The called strike three had arrived in his ear via a headset, powered by technology from a radar-based ball-tracking company called TrackMan. "We knew that it wasn't 100%, and they knew it wasn't 100%," Pacheco says. "It was definitely frustrating at the time because I was trying to work my way back to affiliate baseball and continue my dream."

In 2020, MLB abandoned TrackMan and signed a six-year deal with Hawk-Eye Innovations. Owned by Sony, the UK-based Hawk-Eye has become the go-to rule-monitoring tech provider across international sports, powering FIFA's goal-line reviews, NASCAR pit-road officiating, and much more. If the company's professional-tennis line-challenge system had been around in the 1980s, John McEnroe may never have become "Superbrat." Since 2021, under Hawk-Eye's watch, the robo-ump has continued to move through minor-league ball, inching closer to the Show.

At first glance, automating the strike zone seems like a no-brainer. In a survey of jobs Americans would most like to see replaced by robots, the umpire would surely rank near the top. Many a player and manager has been tossed for voicing objections based on an umpire's eyesight or biases; many a fan has shouted, "open your eyes, Blue, you're missing the game!" The robo-ump has been sold as a futuristic solution to an age-old problem: With a computer-powered zone, an afternoon at the park will never again be ruined by human error.

But replacing imperfect human judges slash punching bags with (near) perfect machines has proved to be a thorny challenge. The subjective has made a nasty habit of sneaking into the frame. And that's before tackling a bigger question: What do we lose by taking human umps out of play? And an even bigger one, far less asked when we talk about automation: What do we gain? As I interviewed players, coaches, a member of the MLB league office, and even an employee at Hawk-Eye, something surprising kept happening: People spoke kindly of home-plate umps. As automation knocks on the big leagues' door, some force — Nostalgia? Flaws in the system? Our own latent fear of being replaced by AI? — has the baseball world suddenly seeing Blue through rose-colored glasses.

As a young catcher, Pacheco hated umpires as much as the next guy. But he can't help feeling empathy now. "They're under a microscope, so I try not to get on them too much. I don't know who would want to be an umpire nowadays," he says. Especially now that every fan can umpire from home as they watch the strike-zone box on their TV, Pacheco adds, "They gotta be perfect."

This season, the ABS system — powered by eight Hawk-Eye cameras furtively perched like surveillance systems above the stadium's second deck — is being tested at all 30 Triple-A parks. Pacheco now serves as hitting coach for the Albuquerque Isotopes, the Colorado Rockies' Triple-A affiliate. "It's following me everywhere!" he says.

The real trouble of revolutions often comes once the fighting ends: What to do after you grab the throne? In baseball, the traditionalists sidelined the insurgent analyticists for decades, always citing "history" and "intangibility." But then the nerds found valuable edges — first Billy Beane with his preference for getting on base instead of looking good in jeans, and then with the Astros who maniacally studied tracking data to create a futuristic blueprint for player development. By the time Houston raised its first banner in 2017, it was clear the revolutionaries had won. The game had changed.

But it turned out that watching America's Pastime reduced to math was not a great way to pass time. League-wide stadium attendance dropped 14% from 2007 to 2019, and, along with every other sport save football, TV ratings are way down. Since the early 2000s, the average viewership of World Series games has halved.

As an MLB league official says on background, if someone were designing a sport from scratch, no one in their right mind would make it look like 2022 baseball. That's why the league brought in massive, progressive reforms this season — including a pitch clock, larger bases, and a limit on pickoff attempts — all with the goal of recapturing the traditional feel of the game: to speed the pace of play, minimize walks and strikeouts, and maximize athletic plays like stolen bases, diving catches, and extra-base hits. Basically, to make baseball more watchable for the TikTok generation.

So far, it's worked: The 2023 league-wide stolen-base rate has sprinted back to its 1980s level. The average game now takes only two hours and 39 minutes, down from a Ben Hurian three-hour, 11-minute 2021 peak. And the 147-year-old MLB made the Time100 Most Influential Companies this year.

Though robo-ump is sold as a more accurate future through technology, it's also another paradoxically progressive change that sets out to restore the game to the more action-packed glory of yesteryear. The MLB league official says they hope that a shrunken, more consistent zone will decrease strikeouts and walks and will lead to more balls in play. Unfortunately, so far this year, Hawk-Eye's ABS has added many more walks than the number of strikeouts it has removed. Those results are disappointing, the league official tells me, to say the least.

Founded by Paul Hawkins, who has a doctorate in artificial intelligence, in 2001 — the same year the iPod arrived — Hawk-Eye was originally conceived as an optical-tracking tool to enhance TV sports coverage. The first broadcast partner used it for cricket. But then, in 2006, Hawk-Eye became an official replay tool used by the tennis judges. Sony bought the company in 2011 for an undisclosed sum. Soon, English Premier League soccer, NASCAR, the Olympics, the Rugby World Cup, and golf's European Tour were using Hawk-Eye in broadcasts and to increase the accuracy of human refs. Hawk-Eye had 12 cameras perched in all 30 MLB stadiums by opening day 2020, marking a watershed for the American market. The NFL began using Hawk-Eye to aid with replays the next year, and the NBA just signed a multiyear deal with Hawk-Eye this March.

As I connect with Justin Goltz, Hawk-Eye North America's commercial director, he can't get his camera working on Zoom. A former college pitcher, Goltz says he's sure he'd be against ABS if he still played, because "the ambiguity of the strike zone probably works more in your favor" as a young pitcher without pinpoint control. "But as someone who now understands the nuances of the business of sports and baseball and the way they're trending with legalized gambling and betting and objectivity playing a big part in making the game as fair and as accurate as possible, I'm a firm believer that it's a step in the right direction," he says. (The MLB league official tells me that legalized gambling has not been a factor in implementing ABS.)

"The technology is there," Goltz adds, explaining the intricacies of Hawk-Eye's ball-tracking system, which uses high-performance cameras to triangulate an object's trajectory. "Our average error is about 3 millimeters, less than the width of an M&M." That level of accuracy has already leapfrogged humanity's limits. But even Goltz acknowledges there's still the issue of what zone all his company's technology is legislating.

"There are additional layers to the problem that need to be solved," he says. "How do you define the strike zone? Is it on a per-player basis? Is it a standard strike zone? Is it an oval-shaped strike zone? Is it a square-shaped strike zone?" Goltz explains that MLB is taking the lead in answering those subjective questions; Hawk-Eye just delivers the data.

I arrive at Sacramento's Sutter Health Park on August 1, the MLB trade deadline, a predictably stressful day for Triple-A ballplayers. Still, the scene around the batting cage is just as you'd expect: music blaring, bats cracking, River Cats players in sleeveless cutoffs and giant sunglasses, an unbroken hum of trash talk. Where I am in the dugout, under a sliver of shade, stands a tablet on a tripod processing data. The hitting coach rears back and fires, the minor leaguer swings, and the tablet displays the ball's launch angle, hit distance, and exit speed nearly instantaneously, all data to eventually be crunched. The days of vibes-based scouting and coaching died soon after "Moneyball" hit the bestseller list; we're now living in baseball's brave new world.

Tyler Fitzgerald, a 26-year-old shortstop, sits next to me after his round is done. The biggest difference he's noticed with robo-ump is that the high fastball, a mainstay in the modern flamethrower's arsenal, just never gets called a strike. Fitzgerald turns to a teammate who's just come back from a stint with the San Francisco Giants. "How much higher do they call it in the bigs?" he asks. The guy holds his hand 6 inches apart. "Yeah, it's a huge difference," Fitzgerald says. According to FanGraphs, the high fastball was thrown 19% of the time in 2015 and rose to 27% of the time by 2021. But, with the ABS zone, "you can completely eliminate it," Fitzgerald says, grinning. "It's been awesome." Most pitchers I talked to said they hated the new strike zone.

Rather than design ABS's parameters to match the rulebook definition of the strike zone, MLB has tried to construct a zone that feels closer to the one players and fans have come to know. This is automation imitating humanity. Frustratingly, the subjective has infiltrated what initially felt like a technological problem: What does the perfect strike zone even look like anyway? The 2D design they've landed on for this year's test is small — 17 inches wide, with the top set to 51% of the batter's measured height, closer to the waist than the shoulders when standing straight, and the bottom at 27% of the batter's height. (Robo-ump's trickle-down effects on batting stances will be fascinating; we may never again see a Bagwell crouch.) Earlier tests had a 19-inch width and used a 3D zone, but that meant curveballs that were bouncing could be called strikes if they'd technically clipped the front of the plate. On September 5, MLB changed the zone again, using Hawk-Eye's tracking data to measure the top and bottom of the zone based on where a hitter's knees and belt were on a rolling average over past plate appearances. This new zone spans from the height of their back knee to a baseball's-worth above the waist. When asked to explain the change, the MLB league official tells me pitchers and hitters preferred a physical reference for the zone. Automation imitating humanity is harder than it looks.

But most surprising of all, Triple-A hitters and pitchers alike tell me the zone varies depending on the stadium

"It's different everywhere," says Miguel Yajure, a pitcher for River Cats.

"It changes from field to field. We were just in OKC and it was a little bit lower, and then we come back home and it's a little bit higher," Pacheco, the Isotopes hitting coach, says. "It's kind of another aspect of the game that we just gotta adjust to. It's kind of like having an umpire, I guess."

When I mention these gripes to Goltz, he tells me that while Hawk-Eye tries to place the cameras in consistent locations at every stadium, "you can't have necessarily the same exact longitude and latitude of every single camera in every single stadium." He adds, "Yeah, I'd have to look at the data to understand why players feel that way and understand whether it's a placebo effect or whether it's a true environmental issue."

An hour before first pitch, I run into the Reno Aces' 31-year-old cleanup hitter Phillip Evans out behind the center-field wall. Evans has played in 121 MLB games, and he says it's damaging to have a different definition of a strike in Triple-A and the big leagues. "Especially with how much movement there is every year, for the guys going up and down every 10 to 15 days, that's going to hurt these guys," he says. "You're here to get back to the Show and get ready to help the team win, right? So why have a completely different set of rules?" For Evans, it's hard not to feel at the mercy of a mad scientist thousands of miles away. "We're just like guinea pigs down here."

If the bio for @UmpireAuditor ("The worst calls of the day, every day") didn't give a sense of the 85,000-follower X account's view toward human umpiring, the banner photo showing that they'd been blocked by the MLB Umpires Association account bangs the point home. When he's not dunking on Blue, Dylan Yep, the man behind the account, does data-analysis work for a couple of public-defender offices in California. "I played in college a little bit, but I was honestly mediocre on my best day," he says. "And I always had sort of a contentious relationship with umpires, as many baseball players do. Especially mediocre ones."

To Yep's eye, the sport is at a technological crossroads. "Every single thing that's done on the field is basically propagated on social media in milliseconds… it feels like we're at a very unsustainable point," says the man whose sizeable social-media following is part of what's made imperfect umpiring so difficult to accept. "You can't take away instant replay; you can't take the box away." For Yep, robo-ump is the obvious next step.

"MLB umpires are probably the very best in the world at what they do. But we're also making them do something that humans simply aren't well equipped to handle," he says. "Tracking a 98-mph slider floating across the zone and then pinpointing its location in three dimensions is just not what the human eye was meant to do, let alone do correctly 100 times a game."

But Yep is the only person I spoke with who was unequivocally pro-robo ump. Even those who thought umpiring needed a technological boost favored what's called the Challenge System. During the Triple-A beta test, three games a week are played as full ABS — robo-ump makes the calls with no ifs, ands, or buts — and in the next three a human ump calls the game and each team has three challenges that must be taken instantly by a pitcher, catcher, or hitter. Players, coaches, and even MLB Commissioner Rob Manfred say the Challenge System is the most likely to eventually reach the big leagues.

If that compromise — humans empowered, and sometimes overruled, by technology — is the solution that wins the day, it's a fascinating outcome. The league would seemingly be saying human error has a place in the game. With advanced enough technology, the human umpire — who gets calls right 97% of the time based on MLB's metrics and 92.5% based on Yep's — should be made obsolete. But, so far at least, the unintended consequences of perfection aren't seen as worth that 3- or 7.5-point improvement. It may very well take an egregious, high-stakes miscall in an unforgivable spot — England's disallowed goal in the 2010 World Cup is credited with bringing goal-line technology to soccer — for the imperfect, but recognizable, status quo to become untenable.

The MLB's league office told me they aren't yet able to offer on-the-record interviews about the AAA beta test of the automated strike zone. One AAA team's director of media relations said they weren't permitted to comment on the Robo-Ump; the umpires calling the games I attended in Sacramento said they could talk about anything except ABS. Attempts to speak with MLB umps bore no fruit. But there was one umpire who agreed to talk.

Calvin Baker started umpiring in 1996 after a lifetime playing baseball and has worked his way up to the Atlantic League. He was one of the first to test the ABS system before it was rolled out across that independent league in 2019. There were plenty of times he disagreed with the call that had been relayed to his earpiece during the three-year test of TrackMan's system. "As a matter of fact, there was a couple of times I was apologizing as I was making a call," he says. But Baker is far from a Luddite; he agrees with the umpires he's talked to who are resigned to their robotic future. "They realize it's coming, so it is what it is."

To Baker's eye, human umpires will still be behind the plate — "you will have a plate umpire there to direct traffic, so to speak" — but balls and strikes will eventually be the province of the machines. The MLB league official agrees, telling me the Jetson Robot Home Plate Ump is not in their plans. Still, Baker worries that something intangible will be lost. "Fans go to see an argument in any sport," he says. "If you just have baseball going through the motions, I think it loses something."

River Cats pitcher Drew Strotman agrees. "By making it so robotic and objective, it just eliminates arguing with the umpire, which to me, is also part of the game," he says. "When you go to a hockey game, the crowd gets most into it when there's human interaction of players getting angry with each other and potentially starting a fight. Big-league stadiums get the loudest when there's frustration." Billy Evans, the Hall of Fame umpire who called balls and strikes from 1906 to 1927, once made a similar argument. A perfect umpire, he said, "would kill off baseball's greatest alibi — 'We wuz robbed.'" (All that said, fans themselves seem tentatively in favor of robo-umps. A 2023 Seton Hall Sports Poll found 52% approved, 28% disapproved, and 20% were undecided.)

Legislating out the high fastball should theoretically increase contact and lower strikeout rates. Strotman also believes that an elevated automated zone is risky because it would create a window for pitchers to throw seemingly sky-high breaking balls that were technically strikes. Remember: Baseball is a sport where everyone finds the outer limits of every rule — without a human arbiter, the second-order effects could make the automation transformative rather than restorative.

Strotman's walk totals are way up this season along with those of many other pitchers in Triple-A, but he balked when I asked whether ABS was to blame. "Growing up, my parents would go, 'Oh, the umpire's strike zone was small!' I'd always shrug that off: 'Trust me, they were good. I just needed to be better. Yada yada yada,'" he says. "Now, there's no concept of that. It is what it is. It eliminates the umpire and just puts a mirror back there."

To replace an umpire with a mirror — removing subjectivity that players, managers, and fans have long screamed about — seems in line with MLB's goal of improving the game. But accuracy above humanity might just stray from their agenda. The game has been umpired by humans forever; the 76 full-time umps who call MLB games are much more thoroughly trained and scrutinized by the league. They're clearly better than the ones behind the plate during the sport's midcentury peak.

And what do we lose when the human ump is gone? Players tell me they worry the field-first catcher will go the way of the dodo. And a few mentioned that forcing pitchers to attack hitters inside the zone would lead to more flamethrowers and fewer pinpoint-control, Greg Maddux types. And that's all before the unintended consequences — it's hard to prevision the edge cases a league full of MIT grads will discover and exploit. Despite the Seton Hall Sports Poll's findings, the animating goal of all these changes is a return to a game in which more pitches are put in play so that fielders can flash leather and batters can run the bases. Pitching coaches training the next generation of starters to throw cutters, sliders, and looping curves that nip an invisible line that only the optical-tracking cameras can see may be more symptom than cure.

The ABS technology should eventually be more accurate than a human, if it's not already. And glitches like the ones in Triple-A (the fifth through seventh innings of the first game I attended were called by a human ump because the cord connecting the ABS system to the umpire's headset stopped working) would most likely be ironed out before MLB implementation. Is that enough? Will fans cherish the accuracy more than the chance to yell at an umpire from the stands or from their couch? In Sacramento, the few fans making noise were almost all commenting upon the ump's eyesight. "Are you blind?" Well, kinda. None of them seemed to grasp that the man behind the plate was just announcing the calls made by the hawkeyes in the sky. On a warm summer night, a couple of beers deep, I'd imagine it'd be much less fun for them to yell at the engineer in row 17.

As I pondered baseball's robotic future, a video popped onto my X timeline: "This Robot shoots better than steph curry & klay thompson " The video shows a humanoid hunk of metal slowly pulling a basketball back and firing — from the foul line, the 3-point line, half court — right through the net. It's powerful and deflating: perfection, sure, and a technological marvel, no doubt. But sport, art, work, and so much else is about human error as much as it is about human excellence. We don't watch to see people get it right every time; fallibility puts virtuosity in starker relief. Seeing Shohei Ohtani whiff through a hanging slider or Tiger Woods blow a gimme putt reminds us that even the great ones are not infallible. Merry Clayton's voice cracking on "Gimme Shelter" and the white smudge on Edvard Munch's "The Scream" give texture to the masterpieces. Perhaps umpires are just detested and ancillary enough to be replaced. Surely, no one relishes the correctly called third strike more because the call before was missed. But from everything I heard, the flesh-and-blood umpires we love to hate are safe. At least until further review.

Joseph Bien-Kahn is a freelance journalist based in Silverlake. He covers film, sports, true crime, and oddballs for GQ, Vulture, Sports Illustrated, and Businessweek, among other magazines.