The Japanese contingent led by Mr. Aiko Toyoda spent 3 hours before a House Committee this afternoon. Most of the testimony and the questions behaved like ships passing in the night, where the answers given by Mr. Toyoda and his executives bore little resemblance to the questions asked except in the sense that the subject was about Toyota automobiles. Toyoda was particularly evasive when questioned about the possibility that the sudden uncontrolled acceleration (SUA) could be caused by the electronic throttle control system. It was a little disconcerting that the Congressional committee didn’t have a software/systems engineer there to help guide some of the more technical questions. However, as a (retired) software engineer I am prepared to offer my own perspective on this deadly problem below the fold.
Before I start, let me lay out some facts. The American auto manufacturers have had their own computerized system for years, along with the Japanese car makers. However, the American manufacturer’s computer “black box” contains memory that records engine performance events continuously providing data that can be subsequently be downloaded by external “reader” equipment which is available on the commercial market. Toyota executives were very careful to say that data from their car’s computers cannot be downloaded to be read by any commercially available “readers”. You have to peel back what they actually said. First of all they said that “their car’s computers cannot be downloaded to be read…..” Further, they said that they will not be ready to support commercial “readers” until middle or late 2011, and they have just started working this month on the idea of making their vehicle’s computer data downloadable. I believe what they didn’t reveal was the fact that currently none of their cars are actually capable of recording all of the engine performance events, so there is no post crash data to download!! Most Americans who are familiar with the data recording provided by the “black boxes” in American cars just assumed that all cars would have this feature included as a standard. Now I think we are finding out that this was an incorrect assumption.
During the testimony it was inferred that the reason that Toyota did not have this data download capability in their cars/trucks was that they wanted to protect proprietary information. Mr. Toyoda and his executives acted if this inference about the company being over cautious about exercising proprietary control over their car’s computer generated data was something that they were loath to reveal to the public. This bit of acting on Mr. Toyoda’s part was a clever ruse to cover up the fact that Toyota has no in-vehicle recording capabilities for saving the engine control computer’s events for later analysis. So the real reason why “the data” can’t be downloaded and read is that it simply doesn’t exist.
I have no idea how many computers are in Toyota cars and trucks that are dedicated to engine control and performance, and this is of little consequence. What is really important is the software control architecture and the numbers of lines of code that make this puppy run. The exact structure of the design is also of no consequence. But what is important is how the master control computer handles and processes interrupts from other real-time events in the system, and how distributed the control functions may be. These parameters are important because the complexity of software testing and verification is directly proportional to the size of the population of these variables.
Let’s take a simple example. I have a pedal controller on the floor in my make believe vehicle on the right hand forward side of the driver’s seat. This pedal has an electronic component that sends out pulses, the rate of which is proportional to just how hard and fast I press down on the pedal. Those pulses are transmitted to a port on the throttle control computer in the engine compartment. This computer decodes the pulse rate and based on its calculations it will send an electrical command signal to an electromechanical ratchet device that will mechanically move and through linkage to a valve increase the flow of gasoline into the carburetor. Likewise, if I let up on my foot pressure on the gas pedal a return spring will bring it back upwards towards its no acceleration position and the pulse rate input to the throttle control computer will also decrease in frequency. This causes the throttle control computer to put out a reverse command to the electromechanical ratchet decreasing the flow of gasoline into the carburetor and so the car slows down.
Now let’s suppose that I add a performance analysis computer that is independently monitoring the flow of gasoline and air into the carburetor along with the actual rate of acceleration and the pulse rate from the driver’s foot pedal. Let’s further suppose that this performance analysis computer performs some calculations and decides that the car is not accelerating fast enough in accordance with data from the driver’s foot pedal, so it sends a command to the carburetor’s throttle ratchet device to increase the amount of gasoline for some brief period of time. The purpose of the performance analysis computer is to attempt to enhance the overall performance and response of the vehicle’s engine. Now we have a situation of shared control over the vehicle’s engine and software testing and verification just got more complex by a magnitude. If we encounter a runaway acceleration situation how do we determine which computer’s software is at fault? Is it the main throttle computer or is it the performance analysis computer? The only way to determine fault during any real-time operation is to record all events. This is why airliners all are required to have on board “black boxes” the recordings of which can be used for post crash analysis. We have reached a point with computerized street vehicles that an on board event recording “black box” must be mandatory equipment.
Don’t get me wrong, I’m not blaming Mr. Toyoda and his company exclusively; he’s just the one who got caught with his hand in the cookie jar. It was bound to happen sooner or later and it just came up on the Toyota product. With all of the new electric cars on the drawing boards and in manufacturing, the prospect of a breakdown in computer software Quality Assurance was just a matter of time. I wonder how many computers are built into the Chevy Volt or in the Tesla.
Others have suggested that the computers in the Toyota vehicles may have experienced some sort of outside electrical interference in the environment and this might be the cause of the SUA crash events. I rather doubt this and such a problem is easily fixed by installing shielded cables in the cars wiring harness runs. If I’m right about Toyota not having an onboard engine event recorder for the vehicle’s control computers, this will result in some major changes for the company. Not only will they have to add the flash memory to store the engine events, but each event will have to be identified, defined and then coded into the software. Then they will have to make all of the distributed control computer’s events accessible to be recorded. They will also have to add the data collection download capability for an external reader. Finally they will have to build a test simulator capable of generating simulated external physical event signals. The simulator will be capable of being programmed by test case software which has been designed and written to make sure that random or unexpected real-world events don’t cause the vehicle’s computer software to malfunction. The test simulator would also provide constant monitoring and evaluation (via downloads to the external reader) of the response of the vehicle’s computer software to test case events. This is a definitely a lot more work than just taking the floor rugs out of the driver’s compartment or installing shiny new shims to fix the “sticky gas pedal” problem.
