MSNBC/WSJ poll oversampled African-Americans to get numbers

[carlotta]

I guess you can get polls to say whatever you want if you choose to oversample a particular group. Now the cat is out of the bag. We can safely conclude from the MSNBC/WSJ poll that Obama wasn't hurt by the Rev. Wright among African-American voters. I'd like to see the same poll done with a random sampling of voters. I'd be willing to bet the results would be different.

"In addition, we oversampled African-Americans in order to get a more reliable cross-tab on many of the questions we asked in this poll regarding Sen. Barack Obama's speech on race and overall response to last week's Rev. Jeremiah Wright dustup. - MSNBC First Read"

[woolldog]

1. I guess you don't understand what oversampling means?

[loveangelc]

2. Not for the entire poll, for the racial questions and for questions seeking to see what they believe

about the speech. he has much higher favorability among whites than hillary.

[UALRBSofL]

6. Y'all know these polls are not accurate, none of them

So when I see a poll I don't really agree with it.

[democrattotheend]

3. His numbers among white voters look good.

I did a little analysis http://purplestatepundit.com/blog/election08/latest-nbc-poll-obama-is-in-surprisingly-good-shape/">here. He is losing the white vote by a smaller margin than Kerry lost it in 2004, and he is doing better among white voters than Clinton.

[carlotta]

10. well, here's what the AP says

"Yet people familiar with Obama's remarks were about evenly split between those who said they felt reassured about his feelings on race, and those who said they still had doubts. Slightly more said Obama has said enough about race than said he needs to address it further.

In all instances, whites were more dubious than blacks about whether Obama had handled the issue successfully. Democrats were far more supportive than Republicans, while independents were likelier to be divided.

Blacks have solidly supported Obama in the Democratic presidential contest, while whites have tilted toward Clinton. ... .."

[writes3000]

4. Just because they asked more black voters than usual doesn't mean that they...

gave them a higher percentage of the vote.

They just wanted to get more of a snapshot with those voters on those issues.

[olkaz]

5. I take it that you haven't enrolled in a statistics class before? n/m

[book_worm]

7. Okay how do you explain new polls out of CT, CA and OR showing O doing

much better in those states than Hillary against McCain?

[countingbluecars]

8. I read it differently.

http://firstread.msnbc.msn.com/

[Teaser]

9. You are only allowed to ever post again if:

You explain to me what the following means:

1) oversample

2) reweight

A simple, three line explanation will allow me to restore your posting privileges.

[ashrob123]

11. This is a common survey technique

when you want to see what a particular group thinks about a subject. A random sample would may have yielded a non-representative sample of African-American voters and therefore they would not have been able to say one way or another what African-Americans think about the Wright situation. It is very possible that in a random sample of 100 people (for instance) that you would only get 5 African-American voters far less than the general population (~13%). Would you have rather they did that and then not be able to say anything reliable because they didn't have enough of African-Americans that responded?

[woolldog]

12. What it means to oversample:

oversampling is a common survey research method used to allow more precise estimates of the opinions of subgroups. The results for the oversampled groups are then weighted down to their known proportion of the population, so that the overall results are still representative of the country (or whatever) being polled.

[kirby]

13. Oh No! My CD player Oversamples TOO!!! Yikes!!! n/t

[Adelante]

14. This may be helpful to your understanding of oversampling in polls

Pollsters sometimes "oversample" a survey sub-population in order to increase the reliability of the results for that group. More interviews means less potential random sampling error. Before tabulating the data for the full sample, however, they "weight" back the oversample its correct proportion with the larger sample.

I checked with Gary Langer, the director of polling at ABC News, and he provided a few additional details. The ABC Polling Unit started with a nationally representative sample of 1,803 randomly selected adults interviewed between March 29 and April 4. Of these, 660 described themselves as baseball fans (on the survey's first question). Of these, 64 were African-American.

The pollsters wanted a bigger and more reliable sampling of African-Americans. So they continued calling from April 5 to April 22 and interviewed another 476 randomly sampled African Americans, of whom 139 were self-described baseball fans.

Thus (adding everything up), the ESPN/ABC survey interviewed 799 baseball fans, including 203 among African Americans. Before tabulating the data, however, they weighted the combined sample of 2,279 (the original 1,803 plus the oversample of 476 blacks) in a way that reduced the proportion of African-Americans to its correct value as determined by the U.S. Census.**

This practice is not at all unusual. The intent is to generate more statistically reliable results by race, not -- as Brown puts it -- to "generate racially charged results."

http://www.pollster.com/blogs/even_polls_about_baseball.php

[Jane Austin]

15. Talking Points Memo asked NBC

if the oversampling applied to the overall results or was just to get a broader sample of African-Americans.

They replied that it was just to get a sample size that was reliable and did not affect the overall results.

[Shakespeare]

16. Perhaps a little education on topic will help:

From TPM (but don't let this get in the way of your faux outrage):

Over-Sampled?
This evening everyone is chewing over the results of the latest sounding of the Democratic primary race provided by the new NBC/Wall Street Journal poll. A number of readers, questioning the results of the poll, have written in flagging this passage in NBC's Chuck Todd's analysis of the poll ...

In addition, we oversampled African-Americans in order to get a more reliable cross-tab on many of the questions we asked in this poll regarding Sen. Barack Obama's speech on race and overall response to last week's Rev. Jeremiah Wright dustup.
Given Obama's overwhelming support among African-American voters, if the poll had a disproportionately large number of African-Americans in its sample, that would definitely throw the results of the poll into question.

But I'm pretty sure that's not what Todd is saying. What I think he means is this: In order to get a statistically reliable subset of African-American voters, they over-sampled this category. (Remember, African-Americans account for only about 13% of the US population. So that subset of a regular poll doesn't really have a large enough sample to ensure a low margin of error.) They then re-weighted these results to come up with topline (everybody put together) numbers that adjusted for that oversampling.

Got that?

In any case, I don't know that. But from my experience I'm pretty sure that's what it means. I'd be very surprised if a major media outlet would release a poll like this without more clearly flagging that the numbers were skewed. In the meantime, we've shot off some emails to people involved with the poll to get confirmation one way or another. We'll let you know what we hear.

Late Update: A source at NBC confirms that this is correct. The results are weighted, as I described.

--Josh Marshall

http://talkingpointsmemo.com/archives/185845.php

[EffieBlack]

17. Pollsters "oversample" all the time - why is this the first time we've heard which group they've

oversampled?

I know why - they are trying to undercut the poll results by telling us "Don't take this poll seriously - All of those BLACK people skewed the results!"

Funny - when pollsters oversample white people, it's all fine and dandy.

[Mass]

18. Do you even understand what you are talking about.

Reas what others have posted and get informed rather than to create fake outrage! For the rest, I take every poll with a grain of salt.

Anybody who has any ideas how polls are made knows that.

[TacticalPeek]

19. It would only take a couple of mouse clicks for them to show both results.

They could simply show the results with and without the oversampling and solve this thing: did it make any difference and if so, what?

[powergirl]

20. That helps Sen. Clinton

If they did not oversample African-Americans, Clinton's numbers would be lower. There's no "cat out of the bag" b/c they stated their methodology. It's not a secret, wasn't meant to be.

:banghead:

[TacticalPeek]

23. What's your basis?

"If they did not oversample African-Americans, Clinton's numbers would be lower."

Do you have data that you base that on? Link? I find nothing at MSNBC or TPM that supports it.

[malik flavors]

21. From MSNBC.com:

From NBC's Domenico Montanaro
Hart/McInturff, the group that conducts the NBC News-Wall Street Journal poll, explains what "oversample" means.

TO: NBC News And The Wall Street Journal
FROM: Hart/McInturff
DATE: March 27, 2008
RE: Sample For March 24-25 Survey

As you know, the sample for the March 24-25 poll on race included an “oversample” of 100 African American voters. There has been some confusion as to exactly how these extra interviews were integrated into the survey; we hope this memorandum will clear up any misconceptions.

The main sample for the survey was a cross section of 700 registered voters nationally. As is the case with all of our usual polls, this sample is statistically representative of voters across the country, accurately reflecting the gender, age, educational, geographical, and racial makeup of the electorate. The column in the topline document labeled “All Voters”, as well as nearly all of the subgroups listed in the survey crosstabs, are among these cross section of 700. Eleven percent (11%) of these interviews -- or 77 interviews -- were with African Americans, which accurately reflects African Americans’ proportion of the electorate. Thus, African Americans are NOT over-represented in our national sample.

In addition to this national cross section, we interviewed an extra 100 African Americans to analyze the opinions of this group with a greater degree of statistical reliability. We combined these 100 only with the 77 African Americans that naturally fell into our national sample, for a total of 177 interviews with African Americans; these extra interviews were not combined with the full national sample of 700. The column in the topline document labeled “African Americans” shows the responses of these 177 respondents, as do the subgroups in the crosstabs for African Americans, African-American men, and African-American women.

The table below shows the margins of error for the three groups whose responses are shown in the topline document:

National cross section of voters: 700 interviews, +/- 3.7%
White voters: 520 interviews, +/- 4.3%
African-American voters: 177 interviews, +/- 7.4%

[Nia Zuri]

22. No they were not overrepresented in the total. They were over sampled-- Big Difference

Blacks were oversampled in order to get a large enough readable sample, but when they were weighted back into the total at the same levels as are represented in the population. This is a very common sampling technique.

[Robbins]

24. BS

If Blacks were oversampled he would have had a lead over Hillary and a bigger lead over Mccain.

[Shakespeare]

26. You never took a statistics class, did you?

Take a look at my post upthread from Talking Points, which explains in very simple terms how the poll was conducted.

[Occam Bandage]

25. Oversampled and overrepresented are not the same thing. Don't post on what you don't understand.

[ecstatic]

27. Will your 3rd post include subtle race baiting as well?

[bigtree]

29. by your standard, YOUR post is 'race-baiting'

ANY mention of race is considered 'race-baiting' by Obama supporters. I've been dealing with issues of race (first hand) my entire life. I really don't believe folks appreciate or understand what racism really is when they make criticisms like yours.

[karynnj]

28. You are completely misunderstanding this

By saying that they intentionally over sampled AA, they are NOT saying that they are overweighed in the analysis. The design of the sample is clearly not a simple random sample - nor are most professionally designed samples.

The intent was to be able to make meaningful statements, broken down by race. If the sample were simply a simple random sample, the sample size that would happen for AA would be too small to get any precision worth speaking about.

You are fishing here.