Your Husband and his dad seem like very weak people. Worried at upsetting the little Brothers/sons.

Is there any members of the family or his friends you get on with well. Your hubbys mum any aunts, any of his mates or neighbours?

maybe it needs to be told to him by another person, some blokes are thick when it comes to feelings and emotional things they need it spelt out to them in BIG LETTERS and sometimes from different people.