Whatever the cause of the mishap, I doubt that Exomars will come out of December's ministerial meeting unscathed.
Ministers are politicians; if they simply allocate 300-odd million Euros for something that then
ends up a smoking heap of ruins on Mars, there will be questions asked.
At the very least, some additional oversight will be imposed - perhaps in the form of an external review panel -
to ensure that the EDL system that is to deliver Exomars 2020 will work end-to-end. In the worst-case scenario,
there may be an increase in cost for additional testing and a delay to 2022 or later.

Reading that paper about the radar testing is a bit disturbing. From what I read the unit was not tested in actual flight conditions but instead simply by sending it simulated echoes. I understand we cannot recreate Mars in drop tests but we can certainly put a unit in some sort of bouncing vibrating box hanging from a parachute under as tough conditions as we can manage and force it to report back its findings over very varied terrain. Anyone have any idea why that wasn't done if I'm reading things right?

$$$, (or in Schiaparelli's case €€€)
Galileo's jammed HGA comes to mind for me, with insufficient testing after the long period in storage. Though to be fair, we are all backseat flying here on this board, and our speculation is just that. Until we get official word from ESA or raw data is released we should give the team the benefit of the doubt. In the meantime, we can wait for the HiRISE imagery of the site.

You don't subject your flight hardware to harsh conditions, you take an identical unit out for "qualification testing". IIRC, in the case of MPL, for example, this involved bolting a radar to an F-16 and diving it at the ground.

There are limits to the fidelity of testing, but simulations are a key element. For landings many thousands of simulated landings are done to build up statistics on the distribution of behavior.

With all that, MPL was doomed by a couple of missing lines of code that would have taken 5 minutes to write. The Genesis SRC crashed because its G-switches were installed backwards. Stupid mistakes happen.

Lack of hysteresis in state transitions.... it's sort of both. It's not necessarily on the state machine designer to create hysteresis for a noisy sensor. The senor could instead be improved. But sure, it's easy to do that sort of thing in code-- hold down the power button for 10 seconds to force reboot-- if the need is communicated.

The radar unit was tested from cranes in Italy in static suspended tests and mounted on a helicopter in dynamic flight tests in Morocco (possibly also due to not that dissimilar terrain?).

See here for pictures of the flight tests.

Those event-triggered multi-threading systems are very hard to test and debug.
There may have been set a timer, which triggers the thruster shut-down if some event hasn't occurred within this time. If, for example the velocitiy variable is initialized with zero, and adjusted by the radar after an assumed time, a delayed setting of the variable may return standstill, although the variable simply hasn't been set. This could be resolved to some degree by adding an invalid flag. But how should the system behave in case of an invalid variable? Default would still be thrusters off, since waiting too long near ground with thrusters on would prevent a landing, and thrusters on during flight would be wrong, too.
There are certainly thousands of possible errors of this kind you don't see in a fixed set of simulations. With the actual data, they can now set-up a new series of unit tests to get closer to real conditions near Mars. And once the error is nailed down, some will certainly say, that this could have been known before. But before the real-world test, finding a needle in a haystack is much easier.
I'm wondering now, whether another test with a low-cost lander should be performed, or if the expensive payload should be risked without a prior fully accomplished landing test.

I thought the radar comes on line only ONCE the backshell have been released. Now, it's said that parachute and backshell have been released too early. I'm a bit confused...or wrong.

A sensible designer would never use such a structure for a flight control application. There have been well-understood flight control algorithms in use for almost 50 years that can be tested. Note that Viking landed on Mars with a very simple computer the very first time. http://history.nasa.gov/computers/Ch5-6.html

Just to clarify on the question of proving computer program correctness…

In the study of formal program verification, one seeks a mathematical-style proof that a program will correctly meet the formal requirements that are defined. This is, as was noted, quite hard to do for complex systems. Moreover, a program that is proven correct may be part of a failing system because the formal requirements are not an accurate statement of the real world application. For example, an airplane autopilot program that assumed that winds are always under 10km/hour might be proven logically correct, but lead to immediate failure because the real world has winds higher than that.

I doubt if any spacecraft failures take place because an algorithm could have been proven correct in the above sense, but the effort to prove it was not made or was not tractable in a computational sense. It's far more likely that ignorance, oversight, or negligence would arise in defining the formal requirements or – more likely still – the notion of proving an algorithm correct in the formal sense was never part of the development process in the first place, for the very good reason that it's usually not tractable for complex systems.

Formal program verification, as the literature defines it, is not a part of any software development process that I've been around. I'd liken it to a doctor trying to use the periodic table of elements to diagnose a patient's illness. Technically, an illness pertains to how your atoms are arranged, but that's not a useful level of description for a sick person, in most cases.

For example, the two problems that afflicted the Cassini-Huygens relay: (1) The Doppler shift in radio frequency because of the relative velocity of Cassini and Huygens during the mission was not put into the formal requirements. Moreover, empirical testing that would have revealed this would have been unthinkable. The algorithm was absolutely correct – assuming that Cassini and Huygens were not moving very fast. The fact of their high relative velocity during the mission was simply excluded from the design phase, but fortunately this oversight was caught before arrival at Saturn. (2) The software that was supposed to turn on radio receivers listening to two partially-redundant radio channels did not turn on one of the two receivers, which meant that Cassini transmitted it in vain, and half of the images that it took were never received, and lost forever. Here, I'm not sure that enough information was released to the public for us to say exactly what went wrong. It becomes a bit arbitrary as to whether the algorithm was correct, but the specifications written incorrectly, or if the specifications were written correctly but the algorithm was not proven correct. My sense is that the paradigm of logical program verification is probably not a good description of their development process, but that may not be public information.

I have a PhD in computer science and I develop flight software for a living (among other things) and I use whatever tools I can, including formal correctness. But I guess I'd agree that it's not common.


My understanding is that this was user error (miscommanding) rather than a software failure per se.

Software can be complex, and the simpler it is, the more likely it is to be right. Landing on Mars should not be that hard from a software perspective.

What a wonderful insight into the computer system on the Viking landers. Interesting that they say in there that a system could be run completely off preset timings but it was decided on that mission to allow the craft to fly itself a bit based on accelerometer input. Instead of using radar to decide altitude they basically used computed speed and timing which says to me that one can know fairly accurately where the craft is at any time without sophisticated add ons. Hindsight seems to say that radar is a wonderful addition that enhances the entry but nothing replaces the basic science and physics that can model the path and timings very precisely. I suppose there's a nerd factor that takes over when designing complex systems, a pull to reject simplicity. Sometimes, as in the case of the Curiosity sky crane, it all comes together but we all had nightmares about all the things that had to go exactly right. In many ways the real genius was shown in the very simplistic idea of the airbag delivery system, something that ESA could have certainly used for its first attempts since their lander was so light. I understand they wanted the base of the lander on the actual surface so once the airbag is deflated and retracted flip the thing off onto its head. Oh the ease of armchair quarterbacking. Laughs.

You are reading way too much into that basically non-technical history. Radar is essential for Viking or any Mars lander that has to do anything based on altitude, as the altitude relative to the ground can't be determined any other way.

I agree with mcaplinger: Landing on Mars should not be that hard from software perspective
Flight/lander software is robust (50 yrs of it) and even newer and tested architectures are out there (Deep Space 1, Remote Agent, late 90s)

An executive like Remote Agent could have said something like: "No Parachute, listen to me, not the Radars, stay with me until I say so...we are on a timeline and we have minutes to go....and as for you Radar, stop sending me crap, and get me good altitude and velocity and quit scaring Parachute... and Rockets, don't even think about firing for only 3 seconds.... everybody get back on the script, and back on the timeline..."

I doubt that the primary problem was radar, as it does not explain the early release of the backshell. My gut feeling is that for whatever reason (bad accelerometers?) the state vector deviated too much from reality and basically the lander though it was lower than it really was and could not incorporate the radar data into the state vector and ended up ignoring it.

Here is my timeline of what was supposed to happen... and what happened
My suggestion is that the heat shield did not come off (or hung on) and spoofed the radars, computer, and rockets
Where is heat shield on the surface, and how close is it to the other hardware?
HiRise should supply answers.
(ESA times, altitude, and speeds)
Click to view attachment

That would be a good theory, but the CTX image shows a possible impact (see http://www.planetary.org/blogs/emily-lakda...pdate-ctx.html)
HiRISE should confirm soon enough...

Out of curiosity did the test lander fail before or after planned decent images?

There is an extra ")" at the end of your url. This one should work.

Here is a detailed technical description of TGO, including the EDM.
Have a closer look at this graphics, particularly at the Radar Doppler Altimeter (RDA) block within the Surface Platform subdiagram. You'll see a thin blue line injecting the block, meaning connection via CAN bus b.
Access to the propulsion sysem via CAN (again via the technical EDM article) :


Communication over CAN bus systems is usually performed via sending function blocks.
Systems are usually described by message sequence charts according to the MSC-2000 "standard". This would normally mean asynchronous exchange of messages between processes with all the difficult testing.
This doesn't necessarily mean, that the system design of TGO works exactly like this, but I would be surprised, if not.

See also page 9 of this presentation:

There's always laser altimeters for altitude and doppler Lidar for relative velocity.

Might be worth it for the 2020 surface platform to check into what's available, especially inhouse with ESA members and associates. OPTEL-D with AIM heritage comes to mind as a family of something that could possibly be further downsized to a sufficiently low footprint.

I'd think, they'll "simply" fix the glitch(es) or parameter settings, without fundamental changes of the design.

Edit: Altitude and veolcity of the EDM were determined by radar:

HiRISE images out : http://www.jpl.nasa.gov/news/news.php?feature=6663 Definitely heat shield separation, but a strange streak going east of the lander crash.

Radar is an extremely robust and mature technology and the larger beam footprint and longer wavelength can be an advantage. The landing applications for lidar I know of are for terrain hazard avoidance, not altitude/velocity determination per se, for example https://www-robotics.jpl.nasa.gov/publicati...aejJGCD2002.pdf

AFAIK, every successful Mars lander (and Moon lander, for that matter) has used radar.

That is definitely a strange streak. Maybe that's the mini-version of a crater ray: The splash of some small-scale debris from the impact site. Another thought: Perhaps that's a location where a small, random feature in the sub-dust topography had less dust adhering to it. The smoothness of the curve, however, doesn't look like typical rock morphology on that scale. Maybe the impact stirred up a mini dust devil?

It could be that a pressurised tank was let free in the impact and the curved streak represents the path the tank took while it was depressurising. Just throwing ideas out there, I doubt we'll ever know for sure.

http://photojournal.jpl.nasa.gov/figures/PIA21130_fig2.jpg

There is no sign of the curved "tail" in the CTX image taken last week, so it may have developed since then from debris or disturbed soil blown by the wind.

Click to view attachment

This is a comparison of the CTX and HIRISE images as best I can match them by the eye. I guess the tail could be lost by the low resolution. There might also be some dispersion of the debris in the time between the images.

There's also no signs in the CTX of the rather prominent craters just west of, and south of, the impact site. Either those craters are also a few days old or the CTX image doesn't really have the resolution and/or signal-to-noise ratio to see such finer details.

I think this is the right solution. This may be a trace of a rotating and ruptured tank. This explains the arc.

Little snippets of information still coming out, one saying the lander swung too much while on the parachute confusing the the altimeter causing the computer to look for radar which wasnt on (maybe). Computer triggers remaining decent sequences, even science instruments turned on, thinking it had landed !

Could the lack of curved tail be explained by different lighting angles between the two images?

It is a resolution issue, but I don't agree that it is not visible in CTX. There is a dark pixel in CTX at exactly the right place.


Phil

Could someone cognisant speculate on how hydrazine might behave if exposed to martian surface conditions?
For instance, would it burn, explode, quickly evaporate, or linger?

MSL's hazcam showed the skycrane explosion as a big plume, but there wasn't enough detail to tell if it was smoke from some chemical reaction/combustion or just a dust cloud from the impact itself. They obviously didn't want to risk exposure to toxic hardware to take a closer look with the rover though....

First, one needs to know the exact chemical. According to this ESA site, the propellant has likely been MMH (monomethylhydrazine), and "MON" as oxidizer, a mix of nitrogen oxides.
According to table 11 of this paper, MMH has a boiling point of about 260 K near 600 Pa. The freezing point of MMH is near 220 K, but it can stay liquid down to 207 K.
Hence MMH may freeze overnight, but evaporate away over daytime, unless it isn't decomposed on impact.

If the impact didn't cause immediate decomposition, it's likely, that the presence of MON formed a hypergolic mix with MMH, resulting in an explosion, and consecutive decomposition of the remaining MMH.

No, the lander was a monoprop system using only hydrazine (N2H4). http://spaceflight101.com/exomars/schiaparelli-edm/

Same thing with MSL.

I wasn't able to find the phase diagram of hydrazine, but I would expect most of it to have evaporated under martian ambient pressure.

I should have remembered that from a few days before...
If it's really pure hydrazine with no water content, it freezes near 2 C.
According to this article, the vapor pressure of solid hydrazine at 0 C is 2.60 mm Hg or 2.60 hPa x 1013.25 / 760 = 3.47 hPa. Therefore at estimated 6 hPa on Mars, there should be a narrow temperature range, where hydrazine would be an oily liquid.
But due to the zero partial hydrazine pressure on Mars, it will boil, then evaporate until it freezes, then sublimate.

But again, it is likely to decompose in an explosion on impact:

Or on contact with the surface material on Mars, especially with the iron oxides bearing dust:

This may be dependent of the temperature, but it's reasonable to assume, that the hydrazine in the tanks was warmed and liquid on impact.

If the hydrazine contains some stabilizing additive, odds for staying stable on impact might be a little better. But it will nevertheless react or decompose in the Martian environment after days or weeks.

Just noticed an interesting quote-



And a prior article about a 170-120 Kelvin difference because of the tides.
http://www.planetary.org/blogs/guest-blogs...atmosphere.html

Now, that's a twist I had not thought of. Without running through pv=nrt,
a difference of 50 kelvin degrees, at 120 degrees kelvin, is a huge change in pressure.

Just like a homerun baseball travels farther through thin summer air, shorter through dense fall air,
perhaps part of the story here is that Mars' atmosphere has some major density fluctuations?

That's always been a fact of life for EDL. It doesn't look to me like it affects the terminal part of landing, it mostly means that the time spent on parachute can vary by a significant amount. It's basically why you need the radar, you can't rely on absolute pressure sensing to figure out your altitude relative to the ground.

Colour HiRISE imagery released: http://www.esa.int/Our_Activities/Space_Sc..._site_in_colour
http://www.uahirise.org/releases/esa-edm/second-image.php

The white spots are real objects, parachute moving in the wind...

Here is a refreshed version of the Schiaparelli KML file for GE. I added several ground overlays with the recently released HiRISE color and black & white images. Do zoom in to see them. All images can individualy be made visible or hidden.

There are many white dots around the crash crater, which are interpreted as pieces that broke off Schiaparelli.
Rapidly toggle on and off the color image of the parachute to see it flip (interpreted as caused by wind, just as MSL's parachute).

The HiRISE b&w image's size is 6.82MB, so the KML directly loads it from the ESA web site (instead of being included in the KMZ). This takes a little time and requires the user to be online.

Fernando
Click to view attachment

Here is the HiRise color image of the Schiaparelli Impact site.
4X and de-convoluted
My thoughts:
* the three big white spots are the three main fuel tanks for the retrorockets
* the arcing line off the right is the pressurized helium tank, spewing (its purpose was to to pressurize the the fuel tanks as each emptied)
* as always ... your mileage may vary
Click to view attachment

ESA says in the article cited below, that the investigation should be done by the end of November. The fuzzy 'software glitch' starts to comes into more focus:

"Fundamentally there’s a software issue here between the radar and the on-board computer system,” Mark McCaughrean, a senior science advisor at ESA, told the Associated Press. “The radar was giving inconsistent info on where it was.”

http://www.csmonitor.com/Science/2016/1105...sh-site-tell-us

Similar information in this new BBC report:

http://www.bbc.com/news/science-environment-37898565

"The onboard computer had some problems taking data from different sources, and defining correctly the altitude...."

Here is an interesting article from ESA on a new microprocessor for space applications.
http://www.esa.int/Our_Activities/Space_En...issions_smarter
Interesting to note that the radiation hardening process, despite progresses, still results in larger circuitry (65 nm) and much smaller speeds (250 MHz) as compared to non-space qualified microprocessors (e.g. the z13 at 22nm and 5GHz).
Fernando

The Italian news agencies are running several stories today about the Schiaparelli EDM Lander.
I counted six articles -- all roughly the same
The craft was built in Italy for ESA. Thales Alenia Space is the spacecraft builder in coordination with Russia’s Roscosmos. The official ESA (interim?) crash report is due on November 24, 2016

Enrico Flamini, 'scientific coordinator' at the Italian Space Agency (ASI) relates what went wrong with the Schiaparelli EDM lander.
• He reports that the lander was ‘in wild oscillation’ on the parachute
• When the altimeter read 2000 meters, the gyros reported -10m (below the surface)
• He said the computer believed the gyros and eventually the lander crashed
• He also said that the stratospheric balloon drop tests of the whole system were never done, to save a million euros
• ASI wanted the Swedish Space Corporation to do the ‘stratospheric throws’, but ESA gave the work to a Romanian company that eventually didn’t do the job
• ESA then 'made itself content' with computer simulations of the landing provided by a British company

Here is one (of many links available)

Sciaparelli Crash http://www.quotidiano.net/tech/marte-schiaparelli-sonda-1.2690005

Gyros don't measure altitude, and as noted upthread, IMUs can't be used to determine absolute height above the ground. So this part of the story makes little sense. Let's just wait for the report and ignore media misunderstandings.

As noted, this isn't a complete/consistent account. It seems to remain open as to whether software caused the crash or if the software simply responded futilely to an already hopeless situation.

Fluid dynamics are not trivial to model computationally, so using computer simulations to validate parachute descent seems like a bad option, but saying that now benefits, of course, from hindsight.

The last time the US did actual flight-like testing of supersonic parachutes was for Viking (not counting the LDSD flights, which both failed.)