Responses to the open letter to the organizers of the poll about Israel that reverberated around the world

Responses to the open letter to the organizers of the poll about Israel that reverberated around the world.

From: “Amiram Goldblum”

To: “‘Maurice Ostroff’”

Date: Mon, 29 Oct 2012 12:28:11 +0200

Dear Maurice,
Thank you for your phone call.
I attach the responses to all your questions
I hope that you will find them satisfactory. There is no way to get more professional responses than from that person.

With best regards,
Amiram

Information from the chief statistician

Dear Amiram,

Here are the replies to the questions that you sent me together with the letter from Mr. Maurice  Ostroff that you forwarded me.

1. Inreply to Mr. Ostroff question, from the data that I got from Dialog, the sample included 82 Olim from theRussia(who came after 1991), and the distribution of their religious affiliation was as follows:  66 Non-Religious, 6 Traditionalists and 10 Religious. None of them was Ultra-Ortodox. The distribution mentioned in Mr. Ostroff’s letter counts the 82 persons twice (as Russians and in the respective religious categories).

1A. The design of the poll was good, and optimal under the constraints of the sample size. In particular I learned that the design used in that study is similar to that used in the majority of the polls performed for the major media.  Specifically, the   design is based on a random sample with quotas for sample sized in each domain (defined by the religious affiliation for the non-Russion) approximately proportional to the proportions of the domains in the population.

1B. Each individual from the population (who has a telephone) had an equal probability to get a phone call from the interviewer.

1C. Furthermore, since given the probabilistic nature of the sampling process, the sample sizes from each domain cannot be exactly equal to the proportions in the populations, I’ve learned that the data were  weighed by the correspondent  coefficients.

2. Since the newly coming Olim fromRussiaare by far the largest group who arrived in the recent years , I think that it is reasonable to analyze their opinion as a special group, and to weigh them according to their prevalence in the population. As for all the other groups of Olim they are represented in the sample by the power (and the beauty, in my opinion) of the random sample. Obviously, as a matter of research, we can focus on any group and then increase their particular sample size accordingly, but that has no bearing on the reliability of the results for the entire population.

3. The sample size of 500 is the routine sample size for the polls in the media inIsrael. A sample size of 1000 is better, as a sample size of 5000 is even better than 1000. The effect is on the margin of error (or, in technical terms on the lengths of the confidence intervals). A sample size of 500 has a margin of error of about 4.4% (with probability of 95%), while a sample size of 1000 has a margin of error of about 3.2% (with probability of 95%). A sample size of 500 is certainly more than reasonable.

4. As for the importance of places of residence my reply is similar to that regarding the different groups of Olim. As far the results regarding the population, the randomization takes care of that. But of course, if one wants to know what the people in Kibutzim think, one can build a specific sample (called stratum) for that sub-population and increase that sub-sample accordingly. Again, no bearing on the reliability of the findings regarding the entire population.

5. All the people who have telephones are part of the population (in particular, if you want, residents across the Green line).

Letter from M. Ostroff to Dialog

Dear Sima

I am a freelance writer and as discussed I am trying to understand the recent apartheid poll

According to a press report, the respondents are made up as follows

1. The report says that 503 respondents took part but as you will see the total is 577.

It is also not clear if the Russians are part of the various religious categories in which case they should not be counted in the total. The total would then be 497.

Where does the 503 come from?

2. As replies from Russians are shown separately in the report don’t you think that we should compare them with olim fromUSA,Iraq, South Africa etc?

3. Don’t t you think that the sample should be at least 1,000 as used byGallup?

4. Is it not important to factor in the places of residence as people inHaifahave different views than Bnei Brak or Kibbutzim?

5. Were any residents across the Green line included?

I would appreciate your response

Many thanks

From: “Amiram Goldblum”

Reply by Professor Goldblum

To: “‘Maurice Ostroff'”

Thanks Maurice,

Please find my response attached, all in brackets  between your lines. I hope that this will be satisfactory, or if not, Maybe you should call other experts to check.

Not a single criticism until now, despite hundreds of reports, was directed to the TOP professional manner in which this Poll was taken.

Best regards,
Amiram

Reply by Maurice Ostroff with Professor Goldblum’s responses in italics and brackets after each relevant item

Dear Amiram,                                     October 30, 2012

Thank you for the response from your chief statistician which I read with interest. It explains several aspects that were not clear.

Dealing seriatim with his numbered paragraphs I reply as follows

1. Thanks for the clarification of the religious affiliations of the Russian respondents. The information is significant. You will observe that although I included the Russians in the total in my letter to Dialog asking for clarification, I did NOT include them in my letter to you. In that letter to you I showed a total of 495 as opposed to the 503 mentioned in the press statement. No explanation has been given for this difference but I guess that it possibly comprises 8 persons who refused to participate. (Amiram Correct – or they dropped in the middle of questioning…)

1A) – Noted

1B – Noted

1C  I agree that sample sizes from each domain cannot be exactly equal  to the proportions in the populations, but if they differ too widely the results become distorted.  It appears that the randomization process was ineffective as  Ultra orthodox comprised 12% of the respondents whereas according to the CBS they comprise 8% of the population. In effect they were over represented by 50%. I would appreciate more details of the manner in which the data was weighted to take this into account.

(Amiram. What you quote is a LAMAS survey from back in 2009 which refers only to grownups over 20 years of age (attached). This morning there is a report that nearly 14% of this years class are Haredi who are exempt from service. It is expected to grow to 20% in 2020. There are also quite a few Haredi who join. So at the age of 18 years the percentage is already nearly double the one that you quote. Also, the change from 2009 to 2013, nearly 4 years, with Haredi having the largest natural growth in Israel, I am convinced that the number chosen for the survey is absolutely correct, even probably too low. There is therefore no need for any weighting. And this is a tough message for the future because Haredi are the most racist of all. http://www.haaretz.com/news/diplomacy-defense/number-of-ultra-orthodox-men-exempted-from-idf-service-reaches-all-time-high.premium-1.473349)

2. I agree that it is reasonable to analyze the Russian opinion but my question has not been answered as to how you arrive at an average total for the entire population when the Russian responses are included side by side with the responses of religious categories (in which the Russians are included) as in the following example of question 8 translated into English. (All figures are  percentages)

Are the Russian votes counted twice; once in the relevant religious column and again in the Russian column? And how is the total column arrived at? Is it the average of all columns or is the Russian column excluded?

(Amiram. Russian votes were not counted twice : only Haredi, Orthodox, Massorti and Secular were used  for calculating the percentage in the last column).

Note that the total in the traditional column does not equal 100%

(Amiram. This is the result of rounding off and is much smaller than the 4.5% error of the whole survey, in fact ANY survey. Ultraorthodox total is 101%, Traditional is 99%, of course we have the numbers after the digit, but this is not the way any poll is reported. And certainly has not a tiny bit of importance.)

3. Gallup and other major organizations use sample sizes of between 1,000 and 1,500 because in their opinion this provide a solid balance of accuracy against the increased economic cost of larger and larger samples

(Amiram. In Israel??  A sample size of 1000 has a margin error of greater than 3%. All Israeli DAHAF and other election polls are between 500 and 600. Check the famous poll for the minister Kachlon today by Rafi Smith. 

.   It is a function of population size and the precision that you wish to obtain. Here is one site that suggests to have between only 50 to a few hundreds in a sample:

http://www.c-service.co.il/searchResult.asp?Msg=4891

And here is from Tel Aviv university (~600 respondents):

http://www.peaceindex.org/indexMonth.aspx?num=200&source=newsletter

and another one with 516 :

http://www.idi.org.il/media/1256611/%D7%9E%D7%93%D7%93%20%D7%94%D7%A9%D7%9C%D7%95%D7%9D%20%D7%90%D7%95%D7%92%D7%95%D7%A1%D7%98%202012.pdf

There is a larger presentation of issues on Wikipedia:

http://he.wikipedia.org/wiki/%D7%9E%D7%93%D7%92%D7%9D

If you have problems with the statistics, why don’t you call other statisticians inIsraelto find out ?

And, do you think that a bit more precision (from 4.5% error to 3.2% error) will change the results of ANY poll ?

The sample size you mention is in theUS:

http://www.gallup.com/poll/101872/how-does-gallup-polling-work.aspx

Where population is ~50 times larger than inIsrael, and its spread is more than 1000 times larger than Israel)…

4. The place of residence cannot be discounted especially in a sample of only 500. I disagree that it has no bearing on the reliability of the findings regarding the entire population.

(Amiur Your opinion is adamantly at odds with the vast majority  of Israeli polls. If that is what you think, you should not believe ANY poll in Israel, including the statistical survey…

If a poll of ~600 is used to probe elections for 10-12 parties, you need a MUCH smaller sample if you only have 2-3 possible answers. All depends on the number of responses possible, and the size of the population that you probe.Gallupmay have 1000-1500 for a 350 million people country….and a spread of population over a continent…)

5. Noted

Sincerely
Maurice

From: “Amiram Goldblum”

To: “‘Maurice Ostroff’

Subject: RE: REPLY RE STATISTICS

Date: Wed, 31 Oct 2012 20:10:12 +0200

Dear Maurice,

This issue is really not of central importance. We can spend years in discussing.

My opinion is that there is no need to weigh because of what I wrote you. Maybe they weighed something else, not the Haredim ? I did not examine others. I do not really care. I was not involved in that part and do not wish to be.

CBS May 2010 is a report about 2009. I did not write 2000, I wrote 2009. I gave you the proportion of Haredi in this year’s IDF draft (13.8%) and the prediction for 2020 (20%).

Pollsters are not CBS. I suggested that you talk to pollsters of elections. I went through a CBS

Questioning myself – they go into homes for 2-3 hours of interviews and they ask about ALL aspects of life, which is why they need a much larger sample. You yourself quoted Gallup polls of 1000-1500 in a country that is 1000 times large in size and 50 times in population compared to Israel.  There are also polls in Israel of more than 3 million people on election day.

The criticism that you quote has nothing to do with statistical errors but with the character of a single question, by Mina Zemach.  The criticism by Shani Mor who Wrote it for BICOM London was already answered by me (attached) reproduced below]  and will get coverage in Friday’s Jewish Chronicle. I attach my long response to his work, that is clearly driven by the political needs of British Jewry, which is understandable. However, even Shani did not raise ANY statistical issues as the ones that you raise.

I am not a statistician but as I repeated a few times already, I have not heard a single criticism of the statistics work on that poll.

I am afraid therefore that I can not help you any more in sorting out the Issues that you are spending much too much energy to discuss , IMO, rather than dealing with the real ones which are the tremendous increase in racist trends among Jews in Israel compared to what we were 20-30 years ago, and its going to become only worse unless we move quickly to solve the conflict with the Palestinians.

Please direct all further questions on statistics to someone else who knows better than me. Why not call CBS ?

With best regards,

Amiram

Poll on racism and ‘apartheid” – questions answered / Amiram Goldblum

A group of 8 people, myself among them, spent a few months since the beginning of 2012 questioning the political situation and concluding that the chances for a solution to the Israel‐ Palestinian conflict are quickly diminishing as settlements are being constructed at an ever accelerated pace and the 2‐staet solution is experiencing a slow death.  All 8 members served many years in the IDF and reserve service, mostly as officers, participated in the wars ofIsrael, and were associated with the Zionist left /liberal parties – Labour and Meretz and other services to the state. In numerous discussions, we questioned what are the Israeli public thoughts about the stalemate in the talks and the disappearing hopes for a peaceful resolution. That is why we decided to commission a public opinion poll, and the Israela Goldblum Fund, a family fund for improving relations and promoting collaborations between Jews and Arabs inIsrael, was able to financially support it.  Unfortunately, neither NIF not J‐Street wished to be associated with a poll that probes racism and apartheid trends among Jews. Due to the shocking results, they will have to reconsider soon.

We spent a few weeks to choose the questions, to discuss and modify them based on the reactions of some of the best pollsters inIsrael, and finally gave the green light to run the phone poll by DIALOG pollsters in early September. The poll was performed with perfect statistical measures and balances just like all currently reported election polls, in terms of the random picking of respondents who have phones, the sample size of about 500,  and its distribution among the different religious sectors (Ultraorthodox, Orthodox, Traditional, Secular).  As most questions had very few (3‐4) optional responses, the precision of the results is greater than that of any of the polls that purport to predict the voting for 10 or more political parties.

This high level of professional attitude on all sides is the reason to reject any criticism of this poll as “poor social science” and “bogus” by someone who did not even point to a single problem with the actual poll. Indeed, the publication of the poll in Haaretz suffered of a few problems. Haaretz and Gideon Levy already apologized for some of the misleading titles and statements. But those were problems in the publication of the poll, not in the poll itself.

It is also extremely important to remember that the Israeli public by and large is very politically involved, knows much more about the intricacies of the various issues concerning the territories due to military service and to that issue being central in the discussions at home, in the supermarket, the class, synagogue, and mostly anywhere else. Also, respondents would ask the pollsters if they are not clear about the meaning of a question, or refuse to answer otherwise. This is an Israeli typical attitude, not comparable to other countries.  The level of the “do not know” responses was between 2% to 15%, except for 3 questions: 26% do not know if there is discrimination against Olim (Newcomers) in government jobs, 21% about discrimination against Arabs in government jobs, and 30% if the boycott on S. Africa resulted in the end of apartheid.

I shall focus here on 4 poll issues that seem to be of major importance and at which criticism was directed:

1)   In case Israel annexes Judea and Samaria, 69% wish to forbid the 2.5 million Palestinians from voting to the Knesset  (19% wish them to vote, 12% do not know) The main criticism was that as most Israelis support the two‐state solution, there is no point in asking them about a hypothetical situation. This criticism implicitly assumes that polls determine government policies. According to the recent “Peace Index” (April 2012) of Tel Aviv university, 58% of Jews do not believe any more in the two state solution. Settlement construction continues at an unprecedented pace and the Knesset speaker MK Rivlin opened the Rabin Memorial session 2 days ago stating that Oslo has failed and that there is only a possibility of one state solution, a “Jewish democratic state” between the sea and the river (Jordan). It is most probable that Abu Mazen will soon “return the keys” to the Israeli PM and Rivlin’s statement will become a fact, withIsraelforced to control the whole of theWest Bank.  The respondents wish one thing (70% wish to continue talks with the Palestinians) but are realistic ( 67% believe that they will fail, “Peace Index” April 2012) .

There need be no official annexation. The very control over the WB suffices to declare that if the 2‐staet solution is not reached very soon,Israelmay become an apartheid state on a National, not racial or ethnic basis, as it will not allow the 2.5 million Palestinians to participate in the electorate. The 69% did not have to use the “A” word.  What they ask for amounts to apartheid.

2)  Asked about some separate roads to Jews AND to Palestinians, 24% responded that it is “good” and 50% said that “it is not good but inevitable”. 17% said that it is not good and should be stopped. 9% did not know. Given the right to decide, who would those 50% join ? Would they take the risk, or would they be guided by the  “inevitable” ?. The answer seems to be clear: 74% prefer, for whatever reason, to have SOME separate roads.

3) Question no. 10 stated: A famous American writer boycottsIsraelclaiming that it exercises apartheid. What is closer to your ideas : we should boycott her, not respond, or invite her toIsrael? The immediate next question, no. 11, was : “considering the claim of the American writer that there is apartheid in Israel, which of the following is closer to your ideas: There is no apartheid in Israel (31%) , there is apartheid in some subjects (39%), there is apartheid in many subjects (18%). 11% did not know. We expected that 90% or more would oppose the claim of the American writer, as it was clearly smearingIsraeland directed by us to sound as being bad. Therefore it does not matter if respondent know exactly what apartheid was inS. Africaor what it currently is

in many other countries. Overall 58% believe that there is apartheid in various field of life here, no matter if it is insideIsraelof pre‐1967 or in the “greaterIsrael”.

4)  The most worrying immediate issue is the racism of Jews against Israeli Arab citizens. It is clearly fed by the political situation and the lack of the two state solution, but is at an unprecedented and frightening level and supported by other recent polls. With 33% Jews who wish to prevent the 20% Arab citizens of Israel from voting (Q6, 59 support them Q6 found that 41% are against voting rights to Olim in their first year in Israel), 59% of Jews in favor of discriminating Arabs in government Jobs (Q4, 34% support them), 47% for “transfer” of some Israeli Arabs to the Palestinian authority (Q14, 40% against), 49% support “the claim that the state should take care of its Jewish citizens rather than the Arab citizens” (Q7, 49% against), and 42% respond that they are “troubled’ if an Arab family would live nearby or in the same building (53% are not troubled, Q8) and 42% will be troubled if Arab children would be in the class of one of their own (49% are not troubled, Q9). It does not matter what 47% mean exactly when asked about “transfer”. They clearly wish to get the Israeli Arab citizens “out of their view”. It is very helpful that there is still a small majority against discrimination, but minor political events could reduce that small majority quickly.

Criticism of this large volume of responses that indicate an extremely high level of anti‐Arab feelings in Israel amounts to not understanding and not realizing or worse, not willing to deal with this grim situation , amounting to burying one’s head in the sand. Just imagine what would happen had such a poll been published inFrance, about the attitudes of French Christians to French Jewish citizens. Sure, no place can be exactly compared to another. Israeli MKs criticize the recent results of Ukrainian elections, with an anti‐Semitic party gaining 9% of the votes. The Anti Arab parties in the Knesset are at a minimum of 32% (Lieberman’s Israel Beiteinu 15, Shas 11, United Torah Judaism 5, National Union 4, Jewish Home 3) joined by more than half of the Likud MKs.

Whoever claims that this is a flawed poll, or poor social science, did not wish to understand it but decided to use any argument to fight against it. This may be understandable given the huge distribution of the results over the world and its use, already, by anti‐Israeli organizations and media. But lovers ofIsraelshould read it carefully and helpIsraelto quickly return to the two‐state solution path, the only path that can save us from becoming an apartheid state. Only by returning to that solution canIsraelcontinue to be a democracy, cherishing the v words of its declaration of independence of May 14, 1948: “full social and political equality of all its citizens, without distinction of race, creed or sex”.