I would do it if I were you, otherwise you'll probably have to bribe someone at the airport to let you board the plane or even worse, they could just refuse to let you board full stop. And you'll probably get the same trouble again everytime you go there, at least until you have permanent residency here in the UK. I would say you were extremely lucky to manage to leave the country without the CFO sticker once already.
Iain.