Just as well you didn't say or do anything, you may have infringed on their human rights !! and no you don't have any in that situation.
I'm working on "not saying anything", I've managed not to have my head beaten in so far but I think it's getting close. I can't stop myself from opening my mouth when people behave badly but forget sometimes that I'm not as quick or as strong with my hands as I used to be. I get away with some of it cos of my loud, deep Glasgow accent but doubt if it will work much in the future
I had a go at 4 young guys in a parked car that threw all their KFC litter onto the pavement, one got out of the car to have a go at me but I stood my ground, luckily for me he bottled it and they drove off.
My pet hate at the moment is people that put their feet on the seats on the London underground, I want to soooo much kick them behind the knee so they fall on their a$$.... but of course I can't do that. so as I said it's work in progress this "not saying anything".
I guess a good beating from 4 or 5 young guys may help me to deal with it !