I have the horrible feeling that this guy would do more than punch the kids - from what I read, I suspect that the insecurity and ostracisation is the sort of background which can lead to child abuse.
Traveller, I will repeat what others have said - Get out now, while you can, and don't look back.