Some people on the street will try to pick a fight with you. You'll know them because their speech will appear in red.
They are actively looking for someone to brawl with, so when you're not looking for a fight, keep an eye out and be careful not to get too close to them.