The proposed Sarasota software audit team looks good

[eomer]

The team proposed by Yasinsac includes several computer scientists who have a proven track record of exposing specific problems in e-voting systems and of opposing paperless e-voting and are nationally prominent election reform advocates. Specifically, Ed Felten, David Wagner, and Matt Bishop (Michael Shamos has been a proponent of paperless e-voting so I don't include him on the list). If the team goes forward as proposed and these three are allowed to do their job freely then we can have a good chance of getting to the truth about the iVotronic machines.

Here's the proposed team:

The following members shall comprise the initial team of the principal investigators:
Alec Yasinsac, Associate Professor, Computer Science Department, Florida State University
Mike Burmester, Professor, Computer Science Department, Florida State University
Breno de Medeiros, Assistant Professor, Computer Science Department, Florida State University
Ed Felten, Professor, Computer Science Department, Princeton University
Michael Shamos, Professor Computer Science Department, Carnegie-Mellon University
David Wagner, Associate Professor, Computer Science Division, University of California-Berkley [sic]
Matt Bishop, Associate Professor, Department of Computer Science, University of California-Davis

http://election.dos.state.fl.us/pdf/FSUstatementWork.pdf

[Tandalayo_Scheisskopf]

1. Who is...

The hardware weenie? I would suggest a noted hardware engineer to be included in this team.

[jody]

2. Does the "Project Scope and Organization" limit the study group?

2. Project Scope and Organization.
The sole purpose of this project is to conduct a scientifically rigorous static software analysis on the
iVotronics version 8.0.1.2 firmware source code to determine and identify flaws, vulnerabilities or
anomalies, if any, that may have potentially caused, contributed or otherwise created the higher than
expected under-vote rate in the District 13 Race. The project team for FSU shall exercise well-known
software analysis techniques at their scientific discretion including code walkthrough, automated code
scanning, debuggers, and other techniques including but not limited to those documented in Section 5.

[philb]

3. Is this analyis likely to be useful? The experts said that problem likely with individual machines

ballot definition files or programming for the about 50 or so machines that had undervotes more than 20%

This doesn't seem to deal with that, and may just be a distraction to take attention away from the most likely problems- that still haven't been allowed to assess

the screen design and programming on those machines with extremely high undervotes.

We already know that all machines were programmed with a ballot design that was designed to take attention away from the
Congressional race and produce undervotes. But that does not appear to have been the biggest problem.
The disappearing votes appear to have been the biggest problem, and that was caused by programming.


I don't think the source code audit for the generic code is likely to be very useful.
Am I missing something?


Are they looking at the programming of the most problematic machines?

[Contrite]

5. What about the PCA cards?

Is anyone looking at those? It took Hursti only 24 hours to spot the fatal flaw in Diebold's memory card architecture.

[philb]

21. The Diebold memory cards were for opti-scan systems- how much alike

to the opti-scan memory cards are memories on the ES&S touch screen systems?

its clear manipulation can occur at many steps in the process

BDFs, compilers, or with the compiled data
all should be checked as there clearly was a major problem that affected the election outcome.

But the audit apparently has shown that the individual memory logs for about 16% of voters on the touch screens show
no vote for the Dist 13 race, so the problem apparently occurred prior to compilation.

[dlaliberte]

13. Don't hold too much hope for this audit

If there is going to be any source code audit, they better somehow determine that the source code really resulted in the binary code running in the machines. I don't know a good way to do that.

Auditing of the binary code in the machines must be done, and that is difficult.

Certainly they must examine problematic machines - no point examining the ones that are not problematic, unless they are going to compare one against the other to find out that there is some difference - that by itself might be enough to prove tampering.

Most important of all --- Even if they find nothing wrong with any of the code in any of the machines, that does NOT mean there was no tampering. All evidence can be removed automatically and invisibly such that there might appear to be nothing wrong after the fact. The only way to find out for sure what happened in those machines during the election is to examine them during the election itself, not weeks later.

But proceed with the audit. If there is evidence of tampering still lying around, I won't complain.

[Bill Bored]

4. Phib is correct. What about the ballot definition files?

Also, can you tell us the results of the retrospective "parallel" testing of the DREs? Did they in fact cast the same ballots as the voters tried to do, including votes for Jennings or Buchanan on the undervoted ballots?

And have the ballot definition files been audited YET?

I have not read the press accounts of this since the initial ones because I think they will be full of hype, inaccuracies and too much dependence on "source code", the definition of which most people don't understand vis a vis ballot definition files.

So give us the scoop eomer, because I know YOU KNOW the difference! :)

Thanks!

[eomer]

6. I don't know whether they will have access to the BDFs.

But I believe that the disappearing votes malfunction that was testified to in voters' affidavits has got to be primarily caused by something in the software.

It's true that there may be a trigger of some kind that is in the BDF. You've correctly pointed out to me in the past that there might be a way of using a hidden straight-ticket option somewhere in the BDF to trigger clearing out all the Democratic votes cast up to that point. That trick doesn't seem to have been used in this case because of the particulars that we know but there could well be some other similar trick that can be put into the BDF.

But it should be fairly easy to spot those kinds of possibilities by looking at the source code.

The thing that I'm worried about is not that there is some simple trigger that can be hidden in the BDF -- I think the investigators will be able to spot that kind of problem just by looking at source code -- rather, I'm worried about the kind of errors that can be programmed in C that can be very difficult to spot in the source code. For example, some piece of compiled code that gets run sometime between the casting of the vote on the voting screen and the display of the review screen that has an unexpected effect on some part of memory that it should not. For example, a pointer somewhere gets incremented incorrectly and so the vote for some down-ticket race gets stored over the vote for the House race rather than in the correct part of memory that was allocated for it. This type of problem would not be easily noticed in the source -- and the interaction that causes the problem could be much more complex than what I described. C is infamously a language that delivers a number of unconstrained capabilities that allow a programmer to totally trash his own data if not careful. Or (what's worse in this case) to write very tricky code that intentionally modifies data in memory in a way that is very difficult to decipher and predict all the runtime behavior of just by looking at source code.

Bottom line, I didn't mean to imply that we can now rest assured that the state audit will do everything possible to get to the bottom of it. As several posters have pointed out, even if there are three guys who are highly qualified and determined to figure it out, there is still the question of what they will have access to and how they will be constrained. But even though that's the case, it is still a hopeful development to see these guys on the audit team. There is a fairly good chance , if you ask me, that they will find what caused the malfunction. And even if they don't find that, I believe it is almost certain that they will find serious problems of some kind or another in the software.

_{Edit: minor wording}

[Wilms]

7. Is it really easier to check the BDF by looking at the source code?

I think not (though I ain't a coder).

In addition to reviewing the source code, it seems the BDF should be checked.

I realize the desire is to review the code that was loaded on particular machines as that could differ from the files on the server side, and that the machines don't have the ability to open the BDF file for review.

Still, it is (and would have been--both after and BEFORE the election) easy to look at the BDF on the server and see if anything is set wrong. Further, I'd be surprised if it would be easy to check it in a de-compiled state.

FWIW, I heard through the grapevine that someone working on the case didn't believe it important to check the BDF directly to see if a straight party option was enabled. They believed that wasn't at issue because Florida doesn't have straight party voting. Argh! The machine most likely has the option even if FL law doesn't allow it's use.

This investigation leaves me worried they aren't going to adequately audit the system.

[eomer]

8. No, not easier, but still easy.

It would be easiest to check the BDF possibilities by looking at the BDFs and the source code together.

So I'm not saying (and haven't said) that they shouldn't check the BDFs. I'm just saying that most of what they need to check can probably be checked without having the BDFs. I could be wrong but that's how it looks to me.

Regarding the BDF and the stuff about the server and decompiled state, I don't think we have a server, as such, in this architecture. The DRE is disconnected and acting on its own, not working with a server. And the BDF would not be compiled so there is no need to decompile it. It may be encrypted but I doubt that it is. It is probably just a data file that can be read directly. I assume that it does not have any capability of expressing logic (like a scripting language). It should be just what it sounds like -- a file that contains data describing the races, the order they will appear on the ballot, and configurable options associated with those races.

Regarding the straight ticket grapevine, I can't imagine that none of these experts will want to check out that possibility.

I'm also worried about the audit but it is mainly over whether the team will be given access to whatever they need (like the BDFs, the hardware components, and so on) and be free of constraints (like prohibitions on divulging serious problems that they find).

What I meant to communicate in my OP was that having these three guys, who are truly expert and truly dedicated to transparent and fair elections, look at this particular source code is quite an important development. I can't believe that they will not find serious problems in the software and at the very least publicize those problems in general terms. Of course, I thought that the 2004 Ohio recount was sure to result in a full hand recount and never imagined the lengths to which they would violate the law and get away with it. So maybe I'm totally wrong and this will be just a farce of an investigation like we've seen so many times in the past. Only time will tell.

[Bill Bored]

14. First of all, BDFs are not source code.

They are election-specific programming -- configuration data. If they are not interpreted code, as Diebold uses, they will not be human readable on the iVotronics. So we are talking about reading things that would look like checkboxes and menu options on a server, that will look like bits or bytes on the DRE. Unless the team knows exactly what they're reading, or the BDF can be de-complied, if it is complied, they will have a tough time figuring out what it all means. If it is decompiled, it still won't look the way it does on the server where it's configured so they will need to translate it, or just look for anomalies without knowing what they're looking at!

Also, have you seen this regarding ES&S's stuff?
<http://www.votetrustusa.org/index.php?option=com_content&task=view&id=1475&Itemid=51>
This actually says that the BDF modifies the object code! Lot's of luck sorting that shit out!

And one other thing:
What you describe about a down-ticket race overwriting a vote in an up-ticket race would be indistinguishable from the BDF hack I envision in this case, although as you say, the BDF is not the only way to do it. It's just the easiest and therefore the most likely way, which is probably why it hasn't been checked after all the weeks this thing has been going on.

[eomer]

16. Yes, the BDFs are data.

I thought that's what I said but I guess my wording wasn't clear. The BDFs are configuration data and that data is used by the software to draw the display and to otherwise control how the software performs. So the best way to investigate is to look at both the BDF and the source code of the software at the same time.

Unless the BDF is encrypted then a skilled developer should be able to "read it". Whether it is an ASCII file or a binary file (there are editors with a facility for reading a binary file) it can still be read by a developer. I've done this kind of thing many times. In fact, on one project I had to write my own debugging utility program that would display the bit-level representation of data that had started out as UNICODE text characters in Java but was then encoded into hexadecimal by a process that intentionally mimicked a particular encoding used in Visual Basic, was run through several obfuscation algorithms, and then stored in a binary file. It is definitely possible for a developer to "read" this kind of a file and figure out what the contents mean and how those contents will relate to the source code's branching and displaying of the screens to the user. They may need to write their own utility for reading the file and displaying something meaningful to the investigator/developer, but they can definitely do it.

Regarding the BDF modifying the object code, I don't think it actually modifies the object code of a compiled program -- it probably is just stored on the same storage medium alongside the object code of the programs. That makes the checksum verification more complicated but that's all it does and this team should be able to sort that out (if they have even have access to some of the actual machines, PEBs, and are therefore able to do the checksum verification themselves).

I guess I still haven't gotten across my main point in a way that is understandable. The software should not allow the behavior that was reported, no matter what is in the BDF. If the software does allow that kind of behavior by way of some trigger in the BDF then that is a major flaw in the software and should be discovered and treated as such. One area of investigation (perhaps the principal one) should be a very narrow focus on how the software can produce that behavior. Is there code that directly expresses that behavior? Or is there some way that the code produces that behavior but only through a pointer getting messed up or a variable overrunning its capacity?

If the behavior occurred as reported then there is a major flaw in the software. Not just in the BDF, but in the software itself.

[Bill Bored]

19. I agree with all this but see Post 18. And I have 1 question:

If there is no developer present, but only investigators, will it be as easy as you say to reconstruct the BDF and determine its effects on the object code?

[yowzayowzayowza]

22. Speaking of BDFs as data:

What of the software which builds the BDFs? I don't see where itz included in the scope of this project either.

[Wilms]

23. Nah

That stuff is on the server side.

And, it seems, they ain't gonna get a look at it.

[stevenstevensteven]

11. BDFs have yet to be tested

The latest word I have is that K. Dent has not released the CF memory cards or the PEBs, which were supposedly locked away since ED. So, they have yet to conduct a comprehensive review of the BDFs, which is clearly where the problem lies. While all of the audit work so far is great overall progress, they have yet to investigate those items most likely at the root of the problems in HR13.

Unfortunately, I do not believe that any of the proposed team members have any experience with the ES&S iVotronic. So, it will probably be a while before they are able to isolate or replicate the problem, if at all.

[Bill Bored]

15. What about the so-called parallel tests?

Have they tried this yet on the machines with the undervotes?

[eomer]

17. Regarding the reported behavior where a voter successfully selected

a choice on the voting screen, the choice showed up on that voting screen, and then the choice disappeared when the voter got to the review screen: do you think that the software (which ultimately controls just what options you can configure in the BDF) is flawed if it can produce such behavior just because an election worker configured the races a certain way in the BDF? If you ask me, if the software can ever produce that kind of behavior (no matter what the election worker has configured in the BDF) then the software itself has a major flaw.

When you say the problem is most likely in the BDF, are you referring to the same reported behavior that I describe above or are you talking about one of the other reported problems?

[Bill Bored]

18. The answer is YES to all of your questions.

For example, say about 1/2 the voters are Democrats and would have voted for Jennings. And half of the voters who voted "No" for one of the non-partisan county-wide judicial races were also Democrats who would have voted for Jennings. The sequence of:

1. Vote for Jennings;
2. Vote No for Judge Woppner;

could cause the vote for Jennings to disappear by the time the summary screen is displayed. This can be set up purely as a ballot definition exploit.

The only thing that would have to be true for this to work is for the default behavior of the iVotronic to be to cancel a previously cast vote for a candidate of Party A (in this case Democratic) once a subsequent straight party vote is entered for Party B (in this case No for Judge Woppner).

If you look at any of the county-wide judicial races on the Sarasota ballot, about half of those "No" votes are equal to most of the undervotes in the Jennings race. So the math works out so far. The part that's harder to work out is whether they are actually correlated by precinct. But since there were multiple MACHINES at each precinct, this correlation may not be necessary for the theory to be true. Precinct size is also a confounding variable unfortunately.

So does anyone have machine-level vote data? All I have are the precincts.

[eomer]

24. But wouldn't the likely behavior of the software be

to switch the vote rather than to cancel it? I was with you until I thought some more about it and realized that a disguised straight-party selection somewhere down the ballot would most likely switch the vote, not cancel it. So you wouldn't see undervotes -- you would see added votes for Buchanan. In fact, you would probably see a lower than expected rate of undervotes because the straight-party selection would fill in a vote in some cases when the voter had intentionally undervoted.

My line of reasoning here depends totally on what the software would do in the case of a straight-party selection when some individual races have already been voted but I don't see any logical reason why it would cancel the vote. Clearly a straight-party selection should mean to force the vote to be for the candidate of that party, not force it to an undervote.

[Wilms]

25. I think the straight-party option would behave as it is programmed to.

That would include Bill's scenario.

But Bill will probably show up and offer his take on it.

[Bill Bored]

26. Yes, it depends on the default behavior of the iVotronic

when a straight party vote is applied to a race in which a candidate of another party has been previously selected. Since straight party controlling races are typically at the top of the ballot and the exploit puts them further down, it's anyone's guess whether the system will remove the previous vote or switch it. This is why these things need to be tested. There are NO standards for this unless it's specified in state law such as in Penn.

Also, you wrote:
"In fact, you would probably see a lower than expected rate of undervotes because the straight-party selection would fill in a vote in some cases when the voter had intentionally undervoted."

Actually in this scenario, intentional undervotes would only go to the Republican. (Did you know that in Georgia 2004, there were virtually NO undervotes for President on 26,000 DREs? I mean ZERO!)

So what I'm suggesting is that the default behavior is to cancel the vote if it was for Jennings and give it to Buchanan if it was an actual undervote. Since the actual undervote rate was relatively small, as seen on the paper ballots, this would only pad Buchanan's vote by a few percent, but it would remove double digit percentages of Jennings' votes, assuming of course that voters had difficulty noticing and/or correcting it.

[stevenstevensteven]

20. YES

I agree with Bill Bored.

[autorank]

9. Shamos...???

Isn't he the proponent who freaked out in Pittsburgh when the machines didn't work? It seems Charles Stewart of MIT turned a bit on this one and maybe the "word" is out, dump the suckers. If I'm anyone of these guys, I'm thinking Democrats run both chambers,I may get paper if I fool around. Hope so.

[eomer]

10. You're probably talking about March 2006 when Shamos refused

to certify Sequoia DREs for use in Pennsylvania. That sent Allegheny County (Pittsburgh) scrambling to buy ES&S iVotronic DREs with just a few weeks left before the May 2006 primary.

I don't know what Shamos is saying after the November election but right up to it and for the last couple of years he has been an energetic opponent of paper and proponent of paperless DREs. He's also the guy who put up the $10,000 DRE hack challenge in August 2004, betting that no one could hack into a DRE chosen by him (http://euro.ecom.cmu.edu/HackingChallenge.htm).

[dlaliberte]

12. Unfair hacking conditions

Michael Shamos' page describing "The DRE Tampering Challenge" (http://euro.ecom.cmu.edu/HackingChallenge.htm ) has conditions that are not a fair comparison with real elections. This shows that he doesn't really understand the nature of the problem with electronic voting machines. Three problems:

1. He gets to choose which machine you get to hack. But he might choose one that actually is secure, as they all *should* be. I would not claim that it is impossible to secure a machine, but I do claim that it is impossible for us to know that all machines are secure.

2. After you make the hack, he gets to inspect the machine for 24 hours in any way he chooses. But that is not what we get to do in actual elections where possibly hacked machines cannot be inspected until long after the election, if at all.

3. He wins if he can detect the hack. But we are not claiming that a hack is undetectable if inspections are allowed during the actual election time frame.

A more reasonable counter challenge would be something like this:

1. We get to choose any machine we want to hack from a set of so called "certified" machines used in actual elections.

2. After the hack, he doesn't get to inspect the machine, but merely gets to use it in a mock election, where the only thing we find out is what voters might want to tell us in an exit poll, and what the final count is.

3. We win the challenge if any hack was successfully made that changes the vote count, regardless of whether the voters notice the changed vote count. That's what we have to live with after all.

We prove that the hack changed the count by calling back each voter and comparing an internal paper trail with what the voters thought they had voted for. Hmm, but perhaps we faked the internal paper trail. Or perhaps we can't trust the voters to remember what they really voted for. Well, gosh, there seems to be no way to prove the vote was hacked without something like a voter verified paper ballot, just as we need the paper ballot to prove that it was not hacked. Hmmm....