I have been on both sides of this fence. When I am canoeing and can clearly see that someone is fishing water right in front of me, I politely say that I want to disturb the water as little as possible, and if there isn't enough depth for me to paddle behind them, I tell them that I am going to take my line and paddle out of the water.
I never say; "Are you catching anything?"
Jim Casada is/was a popular outdoor writer from western North Carolina. He frequently fished the Nantahala River, which is heavily kayaked and rafted. He believes that as long as the water is minimally disturbed, a trout doesn't know the difference between a watercraft, floating log, or passing cloud.
As a canoeist, I try to be as respectful of fishermen as possible.