Like boss mentions check your exchange and whats happening locally with your line etc.
While many are happy with Virgin heard a few horror stories up there with BT's finest moments regarding connections and very sloppy work locally...
One work colleugage had a ongoing battle with virgin who had been taking short cuts with his entire street's connections and im all sure we have heard of someone have a lazy BT engineer in the past make a bodge of a installation.
If its ongoing could be something like that.
I'm guessing you have eliminated updates on pc's or phones slowing the process down?
You have eliminated someone logging onto your wifi?
You have eliminated as Terpes mentions some interference of some sort?
Have you checked the logs of your router when it slows down and just before and after?
