Jossy like every fan of any rugby club I have been refereeing games from the stands since I was 4 years old.
Don't get me wrong I am a committed London broncos fan and have been for ages so yes I watch games through the broncos sunglasses and everyone is against us especially when we lose, and I shout for off side just like everyone else, understanding the offside rule helps though!
But answering the question, yep I am a qualified and experienced community level ref, and an RFL refs tutor. I am lucky enough to get to interact at times with the Refs from elite level down.
Trust me they call what they see, they don't give preference to one team over another, they would prefer to never have to blow the whistle in a game apart from re starts and scoring.
Every game is reviewed, if they are penalising too much it's looked at to try and see why, is it just the teams or is it the ref not managing the game or being out of position? Positioning fitness communication even the tone of the whistle is reviewed.
If a ref is giving a short or long 10 one way it's highlighted. And they self review as well as independent.
So when they get it wrong, they know about it.