In my opinion, replying to someone's followers-only post should offer a new privacy option to make your reply visible only to THEIR followers, instead of YOURS.
If I post in followers-only mode, I may be trusting that my post will only be known by my followers. But when a follower X replies, this exposes half the conversation to X's followers while denying my followers who don't follow X the opportunity to interact with the reply. In the end nobody is happy.
@TerrorBite I have to admit I sometimes like having the /threading/ restricted to critters who follow both me and the other account, but the leakage is still a thing and still makes me uncomfortable.
Pleroma even puts these broken thread bits /on followers' timelines/ [if they're on a Pleroma instance, not the critter(s) posting].