Perhaps more than any other area of baseball, measuring outfield defense has always been made more difficult simply because perception, in so many cases, is not reality. Which outfielder made the better play -- the one who made a flying grab to catch the ball, or the one who made the exact same play look easy because he was faster or took a better route?
You know which one of those plays made it onto nightly highlight reels for the last three decades, and for so much of that time, it was the eye test that ruled the day. But in the back of your mind, you always knew that there's no such thing as style points when grading defense. You knew that sometimes the play that didn't look like anything special was actually extremely special.
Of course, sometimes the plays that look great are exactly that, but we'll get to that in a second. Today, we're going to take what we hope is a step forward by introducing one of several new Statcast™ metrics we'll roll out in 2017: Catch Probability. (See also: Hit Probability.) It's a simple number that can be applied to every tracked batted ball to the outfield, and it's on a scale of 0-100 percent, where a zero percent Catch Probability is "that ball is never, ever caught" and a 100 percent Catch Probability is "caught by everyone, always."
That's important, because we know that not every fly ball is created equally. An out may be an out in the scorebook, but there's a very different amount of skill required to catch the lazy fly ball that has a 95 percent Catch Probability as opposed to the sinking liner that has a 10 percent Catch Probability. Regardless of how it "looks," we should attempt to credit that difference in difficulty accordingly.
So that's exactly what we're doing. Since Statcast™ tells us the exact start position on the field for each fielder and also tracks the hang time of each batted ball, we know the two most important pieces of data to define the difficulty of a catch opportunity are: 1. How far did the fielder have to go? 2. How much time did he have to get there? (Which direction he was running is also important, and while it's not incorporated right now, it will be rolled into the formula early in the season.)
For each of those two data points, there are important specifics to point out. First, we're using "Distance Needed," not actual "Distance Covered," and that's because a fielder who runs a poor route and artificially adds more feet to his path shouldn't get extra credit. Second, we're starting the clock at the pitcher's release, not the point of batted ball contact, and we're calling that Opportunity Time. Why? Because we've seen great fielders like Kevin Kiermaier specifically talk about reading the catcher's signs to anticipate where the pitch will be and get a head start in that direction. That's a skill, and that deserves credit.
You can see how these two measures add up to Catch Probability in the chart below, and you can use an interactive verstion right here.

Since we can measure the fielder's starting point, we can begin to cut through the noise of shifting and positioning that has become prevalent over the past few years and has made things more difficult for the present metrics. It doesn't fully eliminate that issue -- we'll explain why in a minute -- but it's a big step forward.
In a second, we'll have some leaderboards and more details. But you didn't come to an article about measuring great defensive plays expecting to not see some fantastic highlights, so here, for example, is Billy Hamilton doing something insane. This is going to be a recurring theme:

As you can see, Hamilton was 71 feet away from Kevin Plawecki's hit, and he had just 3.9 seconds to get there. While he made it look incredible with his flying leap, that's not really the spectacular part of the play -- it's that the Catch Probability was a mere seven percent. More than 90 percent of the time outfielders are faced with that play, they don't get there.
Now, let's compare that with another play that looked great, when Matt Kemp tracked down this Justin Turner hit from early in the year. While the dive stands out, he also had to go only 65 feet, less than Hamilton, and he had 4.3 seconds to get there, more than Hamilton. That combination combines for a 75 percent Catch Probability -- that is, setting aside what it looked like, three-quarters of the time, that play gets made.

Hamilton's is what we call a Five Star play, and we mean that seriously. Based on the ability to quantify each play, we've crunched the numbers and come up with identifiers based on the Catch Probability.
0 to 25 percent --- 5 Star play *
26 to 50 percent -- 4 Star play **
51 to 75 percent -- 3 Star play * 
76 to 90 percent -- 2 Star play **
91 to 95 percent -- 1 Star play *
Beyond that, you don't merit a star, and the bands get smaller as they get easier because the frequency increases so much.
So who's the best? Well, in terms of Five Star plays, while Hamilton had nine last year, he finished second behind Ender Inciarte and Adam Eaton, who both had 10. Here's one of Eaton's, where he ran 99 feet in 5 seconds to make a grab that had a Catch Probability of just 16 percent.

Hamilton did miss the final month of the season due to injury, of course, and in terms of percentage of Five Star plays made, he was one of just two regular outfielders to grab at least 30 percent (38, to be exact). The other, perhaps unsurprisingly, was Kiermaier (30 percent). In terms of team outfields, the Reds and Twins tied for most Five Star plays made with 18; this is just a taste, of course. 
You probably have a few questions. Let's answer them.
What does this number include?
Each tracked batted ball to the outfield was assigned an expected Catch Probability based on distance needed and opportunity time, compared to the Major League outfield average of balls tracked by Statcast™ since 2015. They were then given a star rating as outlined above.
What doesn't it include?
Initially, direction wasn't included, though the metric was updated to account for that in May, 2017. In addition, an update for the 2018 season accounted for the difficulty of plays near the wall. Catch Probability is also currently effective only for outfielders. As research progresses, we hope to expand it to include infield play as well.
What's the remaining positioning effect?
In a word, opportunity. Even the greatest fielder can't make a great play unless he's got the right conditions, and if a team's positioning is so good, it could be hard for a player to have enough distance to get the chance to make a "great play." For example, our extremely preliminary look at a positioning metric suggests that at least in center, William Fowler had the smallest average distance to cover. It's something to consider, now and going forward.