I agree with Copa. So he thinks you’re an icy bitch. You don’t have to be nice to him if he’s gonna act like a douche.
If you find yourself stuck with him again and he says something else stupid, I would go with, “When you say things like that it makes it really awkward for me and I don’t know how to respond. When I’m regularly in situations with people who make me feel like that, I generally make an effort to avoid those people. I don’t need to discuss our past, and in fact I would prefer that you didn’t as its uncomfortable for me. If that’s difficult, then I understand if we need to keep our current relationship scaled back to small talk strictly when necessary.”