Manual recounts of random precincts OR machines for each ballot type?

[IndyOp]

Okay, I am discussing the possibility of doing mandatory manual random audits of opscan ballots in the fall elections with real honest-to-gosh elections officials. I don't know if I am getting anywhere, but I am learning a lot and finding that the two officials who really have doubts about the system recommend a different recount strategy than I initially suggested.

As typically discussed: Mandatory manual random audits means that we randomly select precincts from each county until we get to 2% or 5% or 10% of the total votes cast in that county.

What the typical recommendation does not take into account - possibly - is that the most common errors in electronic voting result from errors in the "ballot definition files" - the files that are written before each election that indicate races, candidates, and determine how the votes are counted.

In my county there are 92 precincts, and dozens of different ballot types.

If, in my county, we randomly select precincts until we get to, say, 5% of total votes cast, that might be as few as 5 precincts - so only 5 of the dozens of different ballot types are audited.

An error in 1 or 2 ballot types could thow an election (especially for small, local races). If randomly selection does not include one of those ballot types we would not find the error with an audit. So, I now wondering whether we should "audit randomly selected machines for each ballot type" until we reach 5% of total votes cast in the county.

Note: In our county we have central-count optical scan ballots for absentee and early voting - we can audit these at the central-count location. We have DRE's for election day and obviously cannot audit these.

PLEASE POST your thoughts on potential benefits/problems with "auditing randomly selected machines for each ballot type" as opposed to "auditing randomly selected precincts in each county" -- given that we have central-count system so we can't send counting teams to the precints on election night anyway...

The :evilfrown: IS in the details!

:kick:

[Kelvin Mace]

1. First, let me be sure I have all the details right

BTW, what state are you in?

You say you are using OpScan for absentee ballots, but DRE (no paper) for election day?

As to which type of sample will be appropriate, let's see if we can get Febble & Co. in on the discussion.

[IndyOp]

4. 58 out of 92 counties in Indiana have DRE's without VVPBs...

So, voting on election day in these 58 counties is paperless. :(

The wrinkle I've been working, however, is that Indiana has no-fault early voting and 40 of the 58 DRE counties use optical scan paper ballots in early voting (the same ones they use in absentee-by-mail voting).

Why? You have to have some kind of paper ballot for absentee-by-mail voting, and it seems that with some DRE systems, counties purchase a complementary opscan machine (central count) for the absentee ballots.

I am encouraging people to vote early on opscan ballots. We still need to ensure that the paper ballots are handled with appropriate security and I am advocating mandatory manual random audits for the Oct/Nov elections. There is NO WAY we will have new laws in place to require the random audits, but we may be able to get agreement from some Boards of Elections.

[Bill Bored]

5. The best ways to deal with paperless other than getting rid of them are:

Parallel monitoring where you take some machines out of service at random and vote on them all day, comparing the totals to hand counted totals of each ballot as they are cast. See the Brennan Center report, "Machinery of Democracy."
<http://www.brennancenter.org/presscenter/releases_2006/pressrelease_2006_0627.html>
And of course auditing the ballot defs before the election as discussed.

[IndyOp]

11. Do you know anyone who has gotten permission for parallel testing?

I just can't imagine the Boards will agree voluntarily to pull a machine(s) for parallel testing on election day when that will slow voting at some polls.

:shrug:

[OnTheOtherHand]

2. hmm

Be sure to rope in Bill Bored, who has thought a lot about BDF issues.

I would hope that there is a more efficient way to verify the ballot definition files than by hand-counting votes (and actually I'm not sure that would catch all the possible problems anyway), but I won't assume it. A small test deck run in every damn precinct might be best. Treat that as a question, not an opinion!

A 5% audit certainly won't render a local election hack-proof regardless; that opens into bigger questions.

Let's suppose that hand counts are a good way to verify BDFs. Stratifying the sample by ballot style increases the complexity, and potentially decreases the transparency of the sample-draw, but I don't think those are crucial objections. In some circumstances it might reduce the statistical power by skewing the sample toward smaller precincts, but of course if it compels a larger sample to begin with, I don't see how that will be a major concern. A table of precincts by size and style might help us to get a handle on this. (If the styles are essentially randomly distributed, then stratifying by style shouldn't appreciably affect the power.)

[IndyOp]

12. Hey, OTOH...

In some circumstances it might reduce the statistical power by skewing the sample toward smaller precincts, but of course if it compels a larger sample to begin with, I don't see how that will be a major concern.
I was hoping that the fact that it will compel a larger sample take care of this problem. But until I figure out how precinct size varies across and within ballot styles I won't have a clue about whether this is a problem...

A table of precincts by size and style might help us to get a handle on this.
Okay - I might be able to work on this.

[OnTheOtherHand]

19. OK, although I'm not losing sleep over it

At the end of the day, a BDF is just one more way of stealing votes, so if you have a solid audit regime in general, and then you are maybe adding to it to improve your coverage of ballot styles -- and if you have a good L&A test to begin with (which is by no means a given!) -- then you are in pretty good shape.

I think that in addition to random recounts, candidates should have some discretion to demand recounts of a small number of precincts where the results seem quite bizarre -- and if the results do turn out to be bizarre, then they should get to keep going. I haven't tried to figure out the details, of course.

[BeFree]

3. Every machine

In every precict should have a hand count audit of the results.

I'd say an audit of 20% of the ballots might be enough, but smarter folks than me could come up with a number that would give us a 99% confidence in the audit and the counted results.

This type of thinking is what needs to be included in any HR 550 type bill that pretends to have real election reform take place. The present 2% audit of something or another is a crock.

Good work InyOp.....NGU.

Tell you what my latest project is... maybe you can help. Trying to get my local small town paper to request of the BoE an allowance to audit my county's vote on opscan.

[Kelvin Mace]

8. Could you tell me what your credentials are to make these sweeping

statements?

First, they can't audit "every precinct" because the majority of the counties have paperless TS systems.

HR-550 specifies 2% as a minimum, and the statisticians I have spoken with are compfortable with that number given the number of ballots involved. Upon what basis do you state "The present 2% audit of something or another is a crock."?

Tell you what my latest project is... maybe you can help. Trying to get my local small town paper to request of the BoE an allowance to audit my county's vote on opscan.

Have you asked the BoE yourself?

[OnTheOtherHand]

20. strictly, it depends on the number of ballots

Sorry to be nitpicking at your expense again, but something I've really been trying to get across is that the appropriate sample percentage depends a lot on the race. Statewide in North Carolina or Ohio, 2% should be fine, as long as they do it right. In House races -- never mind local races -- not so great. Even in some small states, not so great (I haven't bothered to try to work out the 'tipping point').

Of course that doesn't mean that a 100% count is obligatory. (I don't think BeFree really grasps sampling theory. Does he really think that an exit poll of maybe 2000 people is conclusive, but a count of 70,000 or 100,000 actual ballots is worthless? Whatever.)

[BeFree]

21. You don't think?

Thanks for the constructive criticism.

But still, here's what you wrote... a piece:

...2% should be fine, as long as they do it right. In House races -- never mind local races -- not so great. Even in some small states, not so great...

We look forward to you finding that "tipping point". Good luck, you do that you get a Nobel Prize. But since you can't, we'll just stick with an audit of every machine. Real simple that way, and what we need is simple.... not some "good here, not good there". KISS, KISS.

Audit every machine oughtta just about do it, don't you think?

[Bill Bored]

6. A real L&A test can also be used to test BDFs.

But you have to cast a random number of votes for each candidate or issue and that could involve quite a few ballots.

This is where ballot styles come in too. You want to test at least one of each identically configured machine with the above test, and that means one of each ballot style -- at LEAST!

None of these tests protect against the possibility that the machines will either fail or be attacked using their real-time clock to trigger a time based exploit. That's why parallel monitoring can help and so can the manual audits -- if you have paper to audit!

[IndyOp]

9. I think I've asked you this before - but I am gonna ask again:

Can you direct me to a clear, complete description of how to conduct an L&A test. I think this is a realistic possibility for our central count opscan.

And I am eagerly awaiting the paper from VoteTrustUSA on AUDITS!!

[Bill Bored]

16. Doug Jones:

http://www.cs.uiowa.edu/~jones/voting/testing.shtml#landa

"The tabulation system should be tested by recording test votes on each machine, verifying that it is possible to vote for each candidate on the ballot and that these votes are tabulated correctly all the way through to the canvass; this can be done, for example, by casting a different number of votes for each candidate or issue position in each race or contest on the ballot."

I would strongly recommend that this actually be a RANDOM number of votes for each candidate or issue position in each race or contest on the ballot, and I think the professor would agree. That way, even if having a different number for each would result in too many ballots, at least you'd be assured of testing random combinations of votes.

[Bill Bored]

18. Just to clarify...

Different is good, random is better, different AND random are best!

Does that make sense IndyOp?

[Bill Bored]

7. Also, with THAT few precincts, you should ALWAYS choose individual

MACHINES or scanners because you'll have a greater number of machines than precincts (unless you have one scanner serving a bunch of precincts at the same polling place???). I think you might mean polling places and NOT precincts though. A precinct should only have one ballot style, shouldn't it? I guess it depends on how you define "precinct."

In general, it's always better to randomly select from the largest number of precincts or voting systems or whatever the units are that you're auditing.

[IndyOp]

10. We have central count opscan for early & absentee ballots -

Early voting is at one location (clerk's office) and absentee ballots are all mailed to the clerk's office. All ballots are run through the one and only opscan on election day at the Courthouse - anyone can go watch.

The one and only opscan reader has to be programmed to read all of the different ballot styles. A code goes on the bottom of each ballot that tells the machine which precinct that ballot belongs to (how to count the votes on that particular ballot style).

It occurs to me that, in my case, we cannot randomly select machines - because we only have one central count opscan reader machine -> :blush:

In general, it's always better to randomly select from the largest number of precincts or voting systems or whatever the units are that you're auditing.

Okay - I'll try to follow that suggestion here:

Even though people do not go to different precincts in early and absentee voting, their ballots are marked with precinct numbers instead of a 'ballot style' identifier.

Let's imagine that a ballot style is programmed once and then the precincts that all use that single ballot style are directed to that code ->

For precincts 1, 2, 3, 4:
"1" = Hill
"2" = Sodrel

I think we get a thorough audit by randomly selecting a precinct from each of the different BALLOT STYLES. Let's say...
Ballot Style 1 is used for Precincts 1, 2, 3, 4
Ballot Style 2 is used for Precincts 5, 6, 7, 8, 9, 10
Ballot Style 3 is used for Precincts 11, 12, 13

We would randomly select one precinct from Ballot Style 1 group; one precinct from Ballot Style 2 group; one from Ballot Style 3 group.

If, after selecting one precinct from each ballot style group, we did not reach 5% of total votes cast, then we could go back and select more precincts from the larger ballot style groups?

---

In Indiana counties with all opscan ballots - early voting and election day voting - I am thinking that the most effective audit would, again, be to select precincts from ballot style groups, too. Not just randomly select precincts, but randomly select precincts from each of the ballot style groups to make sure that each ballot style was programmed correctly.

I have an aggravating voice in my head saying that if we select any 5 or 10 precincts we won't have sampled each of the different ballot styles.

[Bill Bored]

15. How about this rule of thumb?:

Consider each RACE as a separate auditing process.

When you do an audit, you are trying to find out who is most likely to have won each race on the ballot. Some people refer to this as a verification protocol, but I think this is really just semantics.

When the BDFs are programmed, they must always consist of races. Races are assigned to districts, or they can be jurisdiction-wide and of course precincts comprise districts. Ballot styles are then GENERATED from the race/district/precinct combinations.

Diebold defines a ballot style as a unique collection races and for once I agree with them!

So when you do an audit, you should actually be auditing each race. A race can be on one ballot style or more than one ballot style, one district or several districts, one precinct or many precincts. So what I would do is list ALL the races on the ballot in the county and then pick from among the various precincts (or machines) to do the audit of each race. If you end up auditing several races in the same precinct or machine, that's OK as long as they're all selected randomly.

[diva77]

13. you are then giving equal weight in your audit to a ballot type that may

represent a couple thousand people versus another type that represents hundreds of thousands (well perhaps not that many - not sure of max. number of people that end of being represented by a particular ballot type). that leaves more room for irregularities that go undetected in areas with more people using a particular ballot type

pls. forgive me if this argument is redundant - not sure if anyone else made this point

perhaps try this: have random selection of all precincts; then, above and beyond that, include random selection of any ballot types that were not randomly selected in 1st round

make sure to print out results on machine as soon as polls close and post results at precinct

but still no guarantee of safety/transparency of elections; plus what printout sez might not be what memory card that goes to tabulator sez

optiscan for early and DRE for election day - I would have expected DRE for early voting

[IndyOp]

14. I think the quickest way to make this decision is to find out

precinct sizes and which precincts share the same ballot style/type. Then I can determine if the possibility of skewing toward small precincts will be an issue or not. If so we could randomly select and then go back and pick one precinct for any ballot style groups not represented in the original sample.

There are about 18 counties that will be using DRE for early voting and election day -- those counties have absolutely no paper to audit. :(

[Einsteinia]

17. Do machine & hand!

Both systems can be subverted and so use each to check each other. Only be certain that there are two different groups of people overseeing them, so that no one can do what is called "hacking the stack." Hacking the Stack is where you know the outcome of one and make the second method match.