[HN Gopher] Mars helicopter employs advanced control techniques ...
___________________________________________________________________
Mars helicopter employs advanced control techniques to survive in-
flight anomaly
Author : dougmany
Score : 112 points
Date : 2021-06-18 19:03 UTC (3 hours ago)
(HTM) web link (control.com)
(TXT) w3m dump (control.com)
| ealloc wrote:
| The article says the problem was a dropped frame from the camera,
| but that just further piques my curiosity:
|
| Presumably they use some kind of Kalman Filter, but those are
| easy to program to account for missing frames, or frames at non-
| discrete timepoints, perhaps even for screwy camera images if the
| programmer had a reasonable prior for the likelihood of it
| happening. Kalman Filters by design account for measurement
| error.
| jcims wrote:
| It seems like the issue wasn't that there was a dropped frame,
| it's that the time slot for that frame got filled by the next
| frame, then every subsequent frame was off by one resulting in
| a persistent timestamp offset of the vision data from reality
| for the remainder of the flight.
|
| I didn't read into it too much so I may not have all the
| details right, but I think this is the gist of it.
| ealloc wrote:
| That would be a straight-up, avoidable software/hardware bug:
| The incoming timestamp is incorrect, and garbage in is
| garbage out.
|
| That would make me curious how the timestamp error occurred:
| software, hardware? Camera or Navigation code? I assume they
| have very high standards, what was the process failure point?
| cm2187 wrote:
| Or timezone bug! Always hard to test.
| tonyarkles wrote:
| Thank you for your comment, because it triggered an interesting
| chain of thoughts about a semi-related problem I'm working on
| at work.
|
| Usually with a Kalman filter, you're taking into account the
| spatial measurement error (gyro-measured roll rate error,
| accelerometer-measured acceleration error, etc) but I don't
| think I've ever encountered a system that explicitly modelled
| sensor latency variation relative to timestamps. Based on the
| description of the problem they encountered here, I suspect
| what happened is that it lost a frame but didn't adjust the
| "photo timestamps" appropriately; every frame that came along
| afterwards would have had an incorrect timestamp? Even if the
| Kalman filter was set up to handle "this photo was taken 20ms
| ago" when doing its forward integration, if they didn't model
| "this photo was taken 50ms ago but is reporting that it was
| taken 20ms ago" then you'd pretty readily get the kinds of
| oscillation they were getting.
|
| Edit: yeah, just like the sibling comment said :)
| doublerebel wrote:
| HoloLens provides a timestamp with every frame from each
| sensor, which can be used for sensor fusion outside of the
| system usage. _Windows.Perception.PerceptionTimestamp_ can be
| used for either recorded data (e.g. camera) or for future
| predictions (e.g. predicted device position). The predicted
| latency is also used to adjust the render to ensure the
| viewer 's perspective is correct even though the draw calls
| may be lagging slightly behind the viewer's position.
| jollybean wrote:
| Can anyone hint why they wouldn't use a gyro?
|
| When it's on land, they can make the gyro reliably point 'down'.
| Then at least during flight they know which way 'down' is.
|
| Would this be too fragile for Mars?
| yupper32 wrote:
| They have an inclinometer, so they know which way is down.
| tonyarkles wrote:
| When the article refers to an IMU, they're referring to a
| combination of sensors; most commonly on Earth that'd be a
| 3-axis gyro, a 3-axis accelerometer, and a 3-axis magnetometer
| (compass). The problem with just using that is accumulated
| error and drift. On super small aircraft like this one, we
| usually use MEMS parts instead of big spinning physical gyros,
| and they don't have great long-term performance.
|
| MEMS Gyros measure angular rate, not absolute angle; to compute
| an actual angle, you're taking the integral of the rate from
| t=0 to now. Any small errors in the measurements add up quickly
| to give you completely nonsensical results. For drones-on-
| Earth, we use a variation of the Kalman filter to combine
| short-term and long-term measurements. As an example, an
| accelerometer requires a double-integral to turn into position,
| so errors accumulate _very quickly_ , but we can correct those
| errors using GPS. The accelerometer and its integrals give us
| really quick acceleration, velocity, and position updates (at,
| say 500hz), and then the GPS is used to correct the long-term
| position and velocity (at, say 5hz).
| kklisura wrote:
| Here's a research paper that might provide more information:
| "Vision-Based Navigation for the NASA Mars Helicopter"
| https://sci-hub.se/10.2514/6.2019-1411
|
| > This paper provides an overview of the Mars Helicopter
| navigation system, architecture, sensors, vision processing and
| state estimation algorithms.
| anticristi wrote:
| Time to set up a GPS constellation on Mars? Somehow forgot that
| this navigation aid -- that we take for granted on Earth -- is
| missing on this other planet.
| aeternum wrote:
| It might be enough for the rover to just drop a few radio
| beacons at various distances.
| idiotsecant wrote:
| Yes, but wouldn't help with this particular problem. This was a
| roll/pitch issue, not fundamentally a position issue.
| bonzini wrote:
| It stemmed from incorrect estimation of the position and thus
| of the velocity, which the helicopter attempted to correct.
| LeifCarrotson wrote:
| I think it would be an APS constellation, because Martian
| orbits like Areosynchronous get their prefix from the greek
| Ares.
|
| That would be really cool, I wonder if it would be faster and
| more precise because there would be negligible human-produced
| radio noise to interfere and the Martian ionosphere is much
| thinner, or if it would be worse because the thinner and weaker
| atmosphere and magnetic field don't protect the receivers from
| solar noise.
|
| I wonder if a rover could drop a few beacons for time-of-flight
| 2D triangulation, it's not like they're moving hundreds of
| miles away over the horizon.
| pilsetnieks wrote:
| But the G in the Global Positioning System doesn't stand for
| Geosynchronous...
| Sharlin wrote:
| Indeed satellite navigation sats aren't even on
| geosynchronous orbits.
| JoeAltmaier wrote:
| GPS doesn't have to be satellites. It can simply be beacons on
| the ground. Often used terrestrially to get a few more decimal
| points on a fix.
| mfsch wrote:
| Not sure what techniques exactly you're referring to, but the
| commonly used techniques for GPS enhancement (e.g. RTK [1])
| are not simple distance measurements between GPS receivers
| and base stations but rather both stations measuring the
| satellite signal and using the difference of the signal in
| both locations to improve position estimates.
|
| I imagine it's possible to set up a ground-based network, but
| you would need a high density to cover large surfaces (you
| want to see at least four stations from every position). I
| also imagine that it would be difficult to get accurate
| vertical positions if the stations are all in the same
| horizontal plane.
|
| [1]: https://en.wikipedia.org/wiki/Real-
| time_kinematic_positionin...
| Gwypaas wrote:
| With the fuzzy GPS signal you had DGPS correcting it by
| broadcasting the current change from a known point. [0]
|
| Before GPS LORAN [1] and it's versions was used relying on
| ground stations
|
| [0]: https://en.m.wikipedia.org/wiki/Differential_GPS
|
| [1]: https://en.m.wikipedia.org/wiki/LORAN
| anvandare wrote:
| Pretty sure SpaceX's first deliveries to Mars (orbit) will
| include just that. I think they could probably use Starlink
| satellites without much modification (most important one I can
| think of would be more panels to cope with the reduced solar
| irradiance)?
|
| The GPS constellation has 32 satellites. A GPS satellite weighs
| ~2000kg. Starship should get 100-150 tons to Mars - the math
| checks out! (Disclosure: I got all my rocket science from
| sending Kerbals to their doom.)
|
| So, MPS for Mars, LPS for Luna? Just imagine the amount and
| quality of rover footage we could get if each of them had
| (constant!) 1Gbps uplink to Earth...
| TheAdamAndChe wrote:
| It's funny/amazing to me how we lived without GPS for
| thousands of years, but it's become so integral to our
| society that it's one of the first things we're setting up
| there before visiting.
| anvandare wrote:
| > The first and most moral responsibility of Freeland will
| be establishing a decent brewery on the new planet.
| (Civilization: Beyond Earth - Hutama)
|
| Sure, our ancestors (who could make their way and thrive in
| the wild) would probably consider me (who gets lost the
| moment I make a wrong right turn and depends on the local
| supermarket) a complete idiot, especially since I couldn't
| recreate _any_ of the technology my daily life so depends
| on... But I 've got central heating and clean tap-water, so
| I'm happy with the exchange.
|
| The higher we climb the ladder of technological dependency,
| the more calamitous it becomes should we ever fall off.
| Best not to look down. (I think I'm supposed to bang rocks
| together to make fire, right?)
| ghaff wrote:
| >(I think I'm supposed to bang rocks together to make
| fire, right?)
|
| Won't work very well unless you have flint specifically
| :-) Even if you know the techniques, starting fire
| without _any_ manufactured tools or manufactured tools to
| make the devices is really really hard.
| usepgp wrote:
| Its possible to just spin a dry stick against another dry
| stick with your hands. takes < 5 minutes to get a fire
| going.
| closeparen wrote:
| One of the main risks for early explorers was simply
| getting lost.
| bombcar wrote:
| GPS is also "easy" to setup if you're already coming in
| from space.
|
| I wonder what the "useful minimum" for satellites would be
| if you wanted, say, relatively accurate positioning once
| every orbit or each day.
| quesera wrote:
| I get the humor, but if you think about it:
|
| Almost all of the technologies that we would need for
| survival and productivity on Mars are recently-invented.
| CamperBob2 wrote:
| The problem of determining position (specifically the
| longitude part; latitude is relatively easy) has been one
| of the most pressing technical problems in human history.
| See https://en.wikipedia.org/wiki/Longitude_rewards .
|
| No kingdoms offered huge prizes for the first electric
| light, or the first antibiotic, the first radio, or the
| first computer. But geolocation has always been considered
| a big deal, and it arguably wasn't well and truly solved
| until the advent of satellite navigation. It'd be
| surprising if it weren't being planned for the next
| generation of Mars orbiters.
| the8472 wrote:
| > satellites without much modification (most important one I
| can think of would be more panels to cope with the reduced
| solar irradiance)?
|
| atomic clocks
| gpm wrote:
| Are those actually necessary?
|
| Not an expert, but the whole network is on communication
| (with perfect understanding of the satellites locations no
| less), shouldn't they be able to run an ntpd like protocol
| to stay in sync, and just sync ground clocks to "starling
| consensus time" instead of "real time".
| cryptoz wrote:
| There is a pretty decent description of the problem here:
| https://physicscentral.com/explore/writers/will.cfm
|
| I'm also not an expert so I don't know if you can solve
| it with syncing. Maybe you can. But that doesn't sound
| like a "just" concept to me - any solution to this is
| going to be complicated.
| ncmncm wrote:
| The control system is clearly designed wrong. Navigation input
| should not be able to affect the closed-loop control system
| directly. It should affect only the calibration, incrementally.
| If that had been done, there would have been no flight
| instability, just disagreement between the IMU and navigation
| about how far they had flown.
|
| There are certainly people involved in the project who could have
| explained this to them. I hope they are learning fast.
| baybal2 wrote:
| > There are certainly people involved in the project who could
| have explained this to them. I hope they are learning fast.
|
| This is how nearly all modern high-end quadcopter drones fly -
| navigation using optical flow camera sensors short circuited
| directly into attitude/motor control loops.
|
| I guess there is not a small chance they entrusted helicopter
| autonomous operations programming to people with quadcopter
| background.
| ncmncm wrote:
| It is one thing to feed averaged relative motion into the
| control system, entirely another to feed in absolute
| position. The flight instability in response to navigational
| position error is incontrovertible proof of a mistaken
| design. Fixing the off-by-one coding error just papers over
| the design error. Testing clearly failed to detect the
| mistake.
|
| I would not be at all surprised to learn that commercial
| quads share the design mistake. I _was_ surprised to learn
| that NASA professionals copied it into a Mars probe. But,
| notably, not into the vehicle that delivered the lander.
| londons_explore wrote:
| Put tape over the downward facing camera on almost any
| quadcopter and you'll soon find out why...
|
| It turns out sideways drift accumulates very quickly - and so
| quickly that unless you are a very practiced drone operator
| its very hard to compensate for by hand.
|
| GPS compensates somewhat, but obviously that isn't available
| on mars (or indoors).
| isatty wrote:
| Terrible article - rehashed old news with a new title and even
| though it's on "control.com" has no mentions of what said
| advanced control techniques are. Atleast it has pictures.
| iancarroll wrote:
| It's cool to see how they compensated for the IMU inaccuracy. I
| bought an IMU off of Amazon once and tried to use it to measure
| the position of a steering wheel. This worked for one rotation,
| but as it mentions, it's incredibly hard to compensate for the
| error margin -- as you integrate more measurements, the error
| builds up irreconcilably to the point where it's a useless
| instrument.
|
| I am sure the one on the Mars helicopter had more precision
| though :)
| HALtheWise wrote:
| I can't think of a polite way to say this, but as someone who
| professionally develops drone software, both of the software
| failures experienced by Ingenuity have been embarrassingly
| amateur at a technical level.
|
| The first failure, which delayed the initial spin test, was
| described as a "watchdog timeout", which for anyone not familiar
| with embedded development basically means the code crashed. We
| all write code that crashes, but I am having trouble thinking of
| an excuse to justify the fact that their code crashed before
| takeoff, on Mars, and they didn't see it coming. There is nothing
| about sitting on the ground on Mars that shouldn't have been
| tested repeatedly on earth, and testing in production is _really_
| not the right way to do Aerospace development (although Boeing
| Starliner would beg to differ)
|
| Similarly, there are a huge number of things that can and will
| result in dropped frames when running Linux on a Qualcomm mobile
| chip, and having a software stack that infers frame timing purely
| from the sequence number is brittle, and would definitely not
| have passed code review and testing where I work (I actually
| checked, we do have a robust solution). If I had to guess, I
| suspect the root cause of the dropped frame wasn't actually
| anything exciting like a cosmic ray, but instead was some run-of-
| the-mill event that would have been caught by a couple hours of
| flight testing on Earth. Either way, it shouldn't have made it to
| Mars.
|
| I'm sure that there are a lot of great engineers working on the
| Ingenuity project that _don't_ write these sorts of bugs, and am
| glad that theae amateur fuckups (barely) haven't crashed the
| drone before it has been able to do some incredible technology
| demonstration work.
| fermuch wrote:
| It might be related to the hostility of the environment. The
| chips aren't radiation proof, so it is expected to have some
| bit flops due to radiation.
| dougmany wrote:
| After five successful flights, the Mars Helicopter had a minor
| incident during its sixth voyage that was fixed using advanced
| control systems.
| njoubert wrote:
| I see no mention of "advanced control techniques"? Sounds like
| there is just a limit on roll/pitch angles and a limit on
| distance applied. Saying "advanced control techniques [for
| multirotors]" sets an expectation for something along the lines
| of Tedrake's Underactuated Control approaches:
| http://underactuated.mit.edu/
| idiotsecant wrote:
| Sensor fusion is part of the control system. Sensor fusion that
| incorporates visual information is outside of the typical
| classical controls sphere and is likely considered part of an
| advanced controls segment of the devices firmware as opposed to
| the bog standard deterministic controls algorithm. You need not
| gatekeep here, this is a real term.
| njoubert wrote:
| The article title is still misleading. The sensor fusion was
| the root cause of the malfunction, it was not the advanced
| control technique that was deployed to survive the anomaly.
| mrandish wrote:
| My understanding from reading an earlier explanation from
| JPL was that it appeared the issue wasn't so much from the
| missing frame itself but rather the software not being
| tolerant of the frame count being off.
|
| IIRC, it seemed more like a missing test case or scenario
| in the software validation suite might have been the
| ultimate root cause.
| arbirk wrote:
| Agree. It's interesting that they don't treat the optical
| and gyroscopic systems as two separate sensors with the
| possibility to disregard one if it disagrees too much. A
| quick restart of the visual system would have been the most
| optimal solution #2020-hindsight
| afrodc_ wrote:
| How would you know which one is wrong in this scenario?
| jessriedel wrote:
| I don't think this is gate keeping. Regardless of the
| terminology used within the field, the intended audience of
| this pop article will interpret "advanced" to mean actually
| unusual or at the cutting edge. Relying on the public's
| misinterpretation of jargon ought to be criticized.
| krisoft wrote:
| > ... intended audience of this pop article ...
|
| Are we reading the same page? I see datasheets linked in
| the navbar, an article about an EtherCan box, and white
| papers about encoders. If any site is then certainly this
| one has a very specialist audience in mind.
|
| Besides i believe what constitutes "advanced" gets wider
| rather than narrower when you move towards a less
| specialised audience. To the generic population even two
| stacked PID loops would qualify as "advanced" while
| practitioners would call that "conservative" or "old-
| fashioned".
| baybal2 wrote:
| > Sensor fusion that incorporates visual information is
| outside of the typical classical controls sphere and is
| likely considered part of an advanced controls segment of the
| devices firmware as opposed to the bog standard deterministic
| controls algorithm.
|
| Not so advanced as very correctly pointed out by
| https://news.ycombinator.com/item?id=27554949
|
| Motor-attitude control loop cannot, and should not be "fused"
| with navigation layer, whether INS, or optical flow sensor.
|
| The bog standard deterministic controls algorithm here is
| "bog standard superior." Computer vision/optical flow sensors
| have much lower precision than gyroscopes, and
| accelerometers, let alone aerospace grade ones.
| caconym_ wrote:
| Do any of these commenters have experience designing small
| "helicopter" "drone" avionics systems using off-the-shelf
| hardware, as in Ingenuity? For instance, you mention
| "aerospace grade" inertial sensors, yet Ingenuity
| apparently uses COTS components in that role^[1].
|
| There are smart people on HN, but unless they have actual
| professional experience designing similar vehicles, I'm
| inclined to give the JPL folks the benefit of the doubt.
| It's also worth considering that they are the only team
| that has ever flown such a vehicle on another planet, which
| imposes constraints that even other experts may not be
| familiar with; that they were working with a limited
| budget, and making heavy use of existing open-source code;
| and that, generally speaking, we weren't in the room.
|
| On the other hand, maybe they really are incompetents whose
| avionics system is "designed wrong", and who do need these
| things _explained to them_ so that they can _learn faster,_
| as the commenter you linked so delightfully put it. In that
| view, I guess their groundbreaking achievement is probably
| more dumb luck than anything else--maybe JPL should do some
| firing and hiring so that things don 't turn out worse next
| time!
|
| ^[1] https://spectrum.ieee.org/automaton/aerospace/robotic-
| explor...
| shadowgovt wrote:
| I wish they had gone into more detail. Unfortunately, my
| experience is that, yes, such limiters do count as "advanced."
|
| First time I got my hands on a working mounted arm, I was
| cautioned again and again the need to run any new program in
| low-speed mode first. Because the arm had no limiting logic and
| would cheerfully power-bomb its own base with all the force and
| torque its motors could muster if I told it to.
|
| Preventing that is still considered an "advanced technique."
| rsp1984 wrote:
| Fusing video with IMU for navigation is called VIN (visual-
| inertial navigation) or VIO (visual-inertial odometry) and the
| field has made enormous progress over the last 10-15 years. It's
| the same technology that the iPhone uses for all its AR features.
|
| Dropped frames are one of the easiest things to handle. Yes, the
| visual feature tracking depends on frames of video but even cheap
| phone IMUs these days are good enough to dead-reckon for a second
| or two, especially when embedded into a sensor fusion framework,
| so the prediction errors resulting from a single lost frame
| should be very minimal and not enough to throw off the tracking.
|
| That's why I find it hard to believe that the VIN in use by the
| Mars Helicopter (part of a multi billion dollar program) wouldn't
| be able to deal with a dropped frame. It just doesn't add up. I
| suspect that the situation is much more complex than what the
| article suggests and that more things went wrong than just a
| dropped camera frame.
___________________________________________________________________
(page generated 2021-06-18 23:00 UTC)