This is a little bit of a complicated and much misunderstood issue, so I will just give you the facts and you'll have to make the risk assessment yourself.

An EEA Family Permit is a form of 'entry clearance' to the UK issued by UKBA
The EEA Family Permit is always issued for validity of 6 months. (means you must enter UK within that validity)
But you will not be an overstayer after the expiry date. You can safely stay for as long as you want.
Whether or not you make application for your free Residence Permit from UKBA is your decision, as technically it's not mandatory.

However that Family Permit is not a visa and it's also not proof of legal residency.
It is not a Schenghen Visa and it is not considered to be a legal Travel Document.

Here's what UKBA state:-
You do not need to obtain documents confirming your right of residence in the UK if you are a family member of an EEA national.

However, you may be inconvenienced if you do not obtain this confirmation, as:
you may have difficulty proving that you are lawfully resident in the UK;
if you leave the UK, you will usually need to obtain an EEA family permit before returning here, in order to guarantee readmission as the family member of a qualified EEA national; and
you may find it difficult to obtain or change employment.
There's too many 'ifs' and 'buts' and 'maybes' for most folks.
There are certain laws and rights, which is all very well if you're completely conversant with them and you always carry a complete set of documents.
Even then you will likely find a major problem to re-enter the UK

Personally, I would strongly suggest you make application for a Residence Permit. It's will certainly be of big benefit in many ways, not the least of which is travel.

Do not make any confirmed travel arrangements withouth having the correct documentation (Residence Permit)

Eventually, you need to do some research and make your own choices.

Given that your husband is an Austrian National, who would like to travel back to see family every now and then, I would have thought you would have applied for your residence card as soon as you arrived to UK.