Atlanta's first famous baseball park was also home to a giant magnolia tree in center field

Atlanta didn't always have Major League Baseball. Before the Braves moved there in 1966, the city was home to the Crackers of the Southern Association and the Black Crackers of the Negro Leagues.
Both teams played their home games at venerable Ponce de Leon Park, nestled downtown in Old Fourth Ward. Although it was a fairly standard ballpark by most accounts, its best and most unusual feature was that it had a magnolia tree directly in center field.
To figure out why a ballpark would have a magnolia in center requires going a little bit further back in time -- to the 19th century.

The area that eventually housed the ballpark was near a patch of natural springs, which became a popular spot in the mid-1800s for Atlantans to relax, enjoy the nearby amusement park and take in the water's supposedly restorative powers. They were named the Ponce de Leon Springs in tribute to the explorer who was said to have sought the Fountain of Youth. In 1890, an artificial lake for boating and bathing was built across the street from the attraction, below tracks from the Atlanta Street Railway and near a magnolia tree that had been planted during the 1860s (as seen above). 
Before long, there were new plans for the lake. Its land was owned by the Georgia Railway & Electric Company, which purchased the Crackers in 1906. The team had played at a number of different locations in Midtown, Mechanicsville and Piedmont Park, and the new owners wanted a more permanent residence.
They decided to drain the lake, because it was good land to use in an already-popular area. On that site, Ponce de Leon Park was built and opened on May 23, 1907. While the lake was gone, the magnolia remained. It was even in play until 1946.

Baseball was the obvious calling card, but Ponce de Leon Park also served as a football field during the early years of Georgia Tech under the legendary John Heisman, as the Yellow Jackets played there until moving to their current home in 1913.
A fire in September 1923 burned down most of the wooden ballpark, but fortunately, it was reconstructed in time for the following season, thanks in large part to funds from owner Rell Spiller. It was then called Spiller Park for many years but eventually reverted back to its original name.
The area around the ballpark changed over time, as Sears Roebuck constructed an imposing building across the street where Ponce de Leon Springs and its connecting amusement park once stood. That building is now Ponce City Market, and its continued presence makes for a good landmark to give an idea of the modern location of Ponce de Leon Park.

As the area changed, still, the magnolia remained. Although it was 462 feet away from home plate, a couple Hall of Fame sluggers did reach it. Babe Ruth had two famous exhibition homers in Atlanta, one of which hit the tree while the other landed in a railroad car that then traveled 752 miles to Joplin, Mo.
The other man to do it was third baseman Eddie Mathews, who coincidentally later starred across town for the Braves. One of the 32 homers he hit as a mere 18-year-old in 1950 landed in the magnolia, stunning everyone at Ponce de Leon Park. "Nobody could believe it," said Crackers coach Whitlow Wyatt.
By the 1960s, Major League expansion was in the air, and since Atlanta was viewed as a great potential new market, the old ballpark's days were numbered. Mayor Ivan Allen Jr. gave the green light for the kind of venue to attract a Major League team, and so Atlanta-Fulton County Stadium was built not far from City Hall. The Braves liked it and relocated from Milwaukee to Atlanta in 1966.
Ponce de Leon Park was demolished a year before the move, eventually giving way to a strip mall. The railroad that once ran along above the ballpark is now the Atlanta BeltLine walkway, and this is the modern view looking down on the area, with the aforementioned Ponce City Market on the left side:

An excellent photo edit on Reddit captures all the change in the area since the days of Ponce de Leon Ballpark. There's almost no evidence it was ever there ...

... but still, the magnolia remains.