It has been stated over and over, but although you will see some interesting things there, I have not encountered any issues. I think anytime you drive a couple miles into the woods in a remote and very rural area, you run the risk of being taken advantage of by a criminal. Protect yourself properly, hopefully by staying close to some escape if things don't feel right. This applies anywhere, not just in Fayette County.
I have done a lot of risky thing pursuing trout, and fishing Dunbar is not on the top twenty list.