I think they're kind of opposites too, in attitude, though facial hair may be an overlap in appearance. I respect the flatbrimmers a heck of a lot more.
Flatbrimmers are generally bums of some obsession. The term trout-bum fits if they are into trout fishing. Yes, they'll have the latest and greatest equipment and trendy flies, while travelling all over the world on a shoestring budget and a diet of spaghetti o's and pop tarts to fish all the famous places, catching legitimately awesome fish and posing each just so to make them look even bigger. Perhaps even really getting into photography to better show off how awesome they are.
Remember, they're essentially the surfer culture. That's who they are. It's just expanded to other things beyond surfing.
A hipster, on the other hand, is pretty much a sub-genre of yuppie. They are young, urban, and well off, artsy and judgemental group who attempts to be edgy in style. Always looking for something new while shunning the established. As such, they can't become obsessed with anything, because doing so means being identified with the group that's into that thing.