Two software errors surfaced after the Boeing Starliner crew ship launched in an unpiloted test flight last December that ultimately failed to dock with the International Space Station due to one of these errors, according to NASA.
The official statement
The Space Agency revealed that an independent review board “found the two critical software defects were not detected ahead of flight despite multiple safeguards.”
Douglas Loverro, director of spaceflight operations at NASA Headquarters, said that the newly discovered issues are more than pure software errors.
“Too put it bluntly, the issue that we’re dealing with is that we have numerous process escapes in the software design, development, and test cycle for Starliner,” Loverro said.
The errors
It turns out that the errors themselves are more like symptoms rather than the actual problems. According to Loverro, the real problem is that they had numerous process escapes that allowed the mistakes to sneak in.
The software running on the Starliner consists of a million lines of code. That will be the main focus in the future of the mission, as the team re-centers their attention towards fixing issues of the code.
Loverro believes that oversight from NASA was insufficient: “That’s obvious, and we recognize that. And I think that’s good learning for us. The independent review team didn’t just have recommendations for Boeing, it’s got recommendations for us as well, and we’re going to take all those to heart.”
The ill-fated flight
The Starliner’s failure to complete the orbit insertion is a result of incorrectly tweaked software. There was a big discordance between the spacecraft’s internal clocks and the code it was running. That was all revealed by the Atlas 5’s flight control system.
The code of the Starliner was meant to retrieve time data during the final countdown. At that point, the Atlas 5’s clocks were accurately set for the launch sequence.
Unfortunately, the code retrieved the time data that was part of an earlier countdown sequence. The result was an 11 hours offset that messed up the timing of downstream post-launch events, including the orbit insertion burn.
The correction of the problem
The problem was finally detected and corrected – Engineers began checking up other critical sequences of software as a means of precaution. Frankly, this uncovered an additional issue- The software in charge of controlling thruster firings required for safely jettison the service module of the Starliner was also poorly configured and set for the wrong phase of flight.
If the problem hadn’t been corrected, the result would have been catastrophic. The service module’s thrusters were undoubtedly going to fire in the wrong sequence, driving it back into the crew module. That could have resulted in a tumble or even permanent damage in the protective heat shield of the ship.
Jim Chilton, a senior vice president for Boeing Space and Launch, said that, even though a detailed analysis was not carried out at the time, “nothing good can come from those two spacecraft bumping back into one another.”
Continuous mystery
Engineers are still analyzing the cause of the communications glitches that prevented a quick correction of the timing issue at first.
Surprisingly, high background radio noise (possibly from phone towers) might have been a cause of all the problems.
However, NASA was expecting software defects, especially in complex spacecraft code, as they revealed in a statement. Nevertheless, they are still firm on their belief that Boeing software quality processes “either should have or could have uncovered the defects.”
