There are immigration rules and there are also published guidelines for the ECO.
95% of the internal guidelines are in the public domain for all to see.
Refusals for settlement visa's are not made routinely, and are not made on the whim or mood of the ECO
Take at look at the statistics for the refusal rate against applications.
Yes of course mistakes get made. No-one is perfect. We all make mistakes at some time. But at least there are processes and procedures available that hopefully capture and correct most mistakes.
Of course it will help it the applicant and sponsor make themselves aware of the rules.
I personally have no doubt that ECO's and ECM's try their best to be impartial and to help.
In my experience refusals are not made lightly.
If anyone feels they have not been treated fairly why not request reconsideration or appeal. Period.
Plenty of visa applications I have become aware of are not only weak but also non-compliant.