No that's not possible.
An EEA family permit is to allow you to either accompany or join your partner as long as your partner is an EEA National..... but a UK national.
British citizens are not considered EEA nationals for the purposes of the European rules relating to exercising treaty rights in UK.
Your husband must excercise treaty rights outside of UK