The act of being pulled away by a stranger must have been scary but lack of reaction from other people would have been scarier, in my opinion.

I feel sorry that you had this bad experience in one of the most beautiful cities in India. Hope this never happens to anyone.