Fight or flight mechanism has worked well for our survival over centuries. Go back to the ancient settlers. If they saw a lion or a tiger killing one of their tribe, having not seen the danger the first time they were exposed to it, the second time it entered their village they would be armed and either kill it or cause it to fear them and withdraw. Thereby saving their people and averting the threat. But what if the elders told the tribe to approach the tiger and reason with it? And it killed them. Would one trust the judgement of the elder of the village? Is the threat lessened because those in power say it is? Perhaps not all lions and tigers will kill you? And if one in 100 is benign should we be off our guard?

So, yes, actually if something or someone (in this case an ideology) has a track record of causing harm we should (if we have a survival instinct) FEAR it and FIGHT it.