I've been in abusive relationships and the first thing I want to say is, this man won't change. If you're prepared to put up with it, he'll carry on doing it and might get worse over time. Also, abusive men can seem very plausible and charming to outsiders and he will do everything he can to get other people on his side and to isolate you.
This is a very bad situation for you and your children. My advice is to leave your husband straight away. If necessary, go to the Police who take domestic abuse seriously and if you need to go to a refuge, can help you with that as well.
If you haven't had sex with your husband, I believe you can apply for a divorce straight away. You can apply for maintenance too. If your husband threatens you, you can get an injunction to make him stay away from you.
To assist with this, keep a diary of all the bad things he does - day and time (and don't let him know you're doing it).
It would be better not to see the other man until you've sorted things out, because your husband could accuse you of having an affair and convince everyone (including the legal system) that he only started behaving badly because you were seeing another man. Don't give him any ammunition to shoot you with.
Whatever you do, don't rush into another relationship. The man seems nice now, but so did your husband once.
None of this gets you a visa to stay here, it's true, but you and your children are worth more than the misery that is your life at the moment.
