Got yelled at in Altstadt for speaking English

Was minding my own business on a Wednesday afternoon (3pm) when a woman heard me and me friend(German) speaking English and out of nowhere started complaining about “englander” and then started yelling in English that everywhere she goes she can only hear English now.

After 7 years here in Düsseldorf this is the first time that happens and wow me and my friend just got speechless.

What you folks do in cases like this?