http://www.ukba.homeoffice.gov.uk/wo...mesticworkers/

They are allowed to change employers while they are in the United Kingdom, but they must still be employed as a domestic worker.

I assume, if the family move out of UK, then they are no longer employed as a domestic worker and therefore have to leave also.