I've found this info about people with ILR, that you might not know about...

Anyone who, on arrival in the UK, requires the permission of an immigration officer to enter the country is "subject to immigration control". This includes holders of ILR who always need to seek permission to enter the UK despite their settled status. This effectively means immigration officers have the right to question them as to the nature of their stay, how long they have been away, what their profession is, etc, etc, and ultimately make a decision not to admit them.

This is an important point because holders of ILR DO NOT have the automatic right to enter the UK. Each and every time an ILR holder presents himself for entry to the UK, his entry is at the discretion of the examining immigration officer.

People who are not subject to immigration control have an inalienable right to live in the UK (right of abode or in the case of EU citizens, treaty rights) and therefore require no permission to enter the UK. Of course, they are still required to present their passports on entry to prove this right. Eu citizens' rights to live in the UK are not entirely inalienable, as they can still be deported and barred from the UK on public security grounds.