Maybe he misunderstood your long look at him.

Maybe he thought you were flirting with him, Hence the hand in his pocket and moving closer to you.