Meeting Jewish settlers can be an unsettling experience.

Not because they really are the peculiar, pale-skinned wreckers of peace in the Middle East that so much of the Western media makes them out to be, but for the opposite reason – because they’re surprisingly normal.

It is testament to the intensity of anti-settler sentiment in the Western press that I ­– a non-Jewish consumer of copious amounts of news – arrive in Ma’ale Adumim, a settlement on the West Bank, expecting to be greeted by men with a Torah in one hand and shotgun in the other.

My prejudices are pricked over coffee in the sun-scorched outdoor section of a trendy café, where settler leader Dani Dayan uses maps and biblical evidence, rather than clenched fists, to try to convince me of his case. “Israeli settlers are the most hated people in the world,” he says, sounding defensive and sad. He doesn’t use the phrase “the West Bank”, preferring Judaea and Samaria.

The world has been fed “a lot of misinformation about us”, he says. Looking at recent media discussions of Ma’ale Adumim, he may well have a point. Around five miles from Jerusalem and surrounded by the Judean desert, it has a population of 40,000, making it the largest settlement.

So, not surprisingly, it’s a source of controversy. But sometimes the controversy is fuelled more by fury than facts. A few years ago, the Israeli group Peace Now announced to much violent head-shaking among Western liberals that 86 percent of Ma’ale Adumim was plonked on once privately owned Palestinian land; after consulting official documents it had to revise its figure somewhat, concluding that actually only 0.5 percent of it is built on such land. Misinformation indeed.

You don’t have to be a fan of settlers’ cause (I’m not) to see that Mr Dayan has a point when he says settlers are the most hated people in the world. You could hang out at the United Nations building for months and not encounter a single person with a good word to say about Israel settlers.

In the Western media they are depicted as a bunch of immoveable fanatics whose convictions will condemn Israelis and Palestinian to years of more division. German magazine Der Spiegel says they’re the “stumbling block” to peace. The pro-Palestinian Washington Report on Middle East Affairs tells us “settlers and suicide bombers block the path to peace”.

Some of the Western discussion of the settlers is uncomfortably racial in tone. In her book Behind the Wall, the British cultural commentator Bidisha described Israeli settlers as “the most abnormal people”, “odd recluses” who are “completely unworldly”and “look like zombies”. They have “peculiar children”, she says.

There’s an ironically colonialist bent to such commentary – ironic because where this critique of the settler movement presents itself as right-on and anti-imperialist, it sometimes rehabilitates old libels about peculiar foreign peoples, only this time aimed at white Jews rather than black Africans.

There are two reasons why this slamming of the settlers is out of order. The first is that these people weren’t beamed into the West Bank from outer space. They were encouraged to venture there, sometimes explicitly, sometimes implicitly, by past Israeli officials, especially in the post-war periods of the late 1940s and late 1960s-early 1970s. They believed, with good reason, that they were doing their bit for Zionism.

You can’t now turn around to them, 20, 30, 40 years later, and say: “Oh, sorry, that was all a big mistake. Pack your bags, dismantle your homes, demolish your schools, leave your jobs, and come back to Israel proper.”

And the second reason anti-settler sentiment feels wrong is because it seems to me to be a cut-and-dried case of scapegoating.

Settlers have been turned into a fanatical force we all love to hate, the “zombies” everyone can point a big fat finger of blame at in a bid to disguise their own failings on the Middle Eastern front.

American leaders, UN officials, even some Israeli politicians – all now dump blame for the sheer difficulty of resolving the Mid-East conflict on to the shoulders of settlers. Strikingly, even friends of Israel are adopting anti-settler postures.

It’s pretty clear why – they are trying to deflect criticism from the state of Israel itself by joining in the anti-settler chorus. They hope that if they nod along while the settlers are turned into the pariahs of international affairs, then they might help protect Israel from harsh criticism through re-aiming all the bile it normally gets at those “bad Israelis” – that is, the settlers.

This is a mistake, for it intensifies the isolation of the settlers, and might further radicalise some of them, while also detracting attention from the fact that no one has yet come up with a really good idea for the future of the Middle East.