I know this thread is old, but I went down a long Google tunnel looking up information about this old race track. I can tell everyone with 100% certainty that the above aerial is NOT the location of the old Fair Grounds. This is what the location looks like now:

This is now a park, and you can see from my shot that some of the track is still in existence. I went and walked this park and the remaining track elements, and this was no race car dirt track. It looks more like it was a horse track at some point in time.

The correct location for the track is almost exactly 1 mile due West of the location in the above pictures. The original fair grounds was converted into one of Charlotte's early (maybe very first) indoor shopping malls in the late 1960's. We lived at the time, and in fact I still live, about 2-3 miles from this location, and I went there many, many times as a kid. There is literally nothing remaining of the old track facility, one would never be able to identify it from any aerial photo after the completion of the mall.