Help - Search - Members - Calendar
Full Version: PMA Actuator problem?
Unmanned Spaceflight.com > Mars & Missions > Past and Future > MER > Opportunity
alan
CODE
02259  12:27:42  p2864.04. 2    0   0   1   0   0   1   pancam_PMA_actuator_diagnostic_L2

What's a PMA actuator?

edit: found it, PMA is the Pancam Mast Assembly, they must be determining why the pancam didn't turn when the drive direction mosaic was taken.
Tesheiner
Saw those sequences too. huh.gif
The PMA actuators are those *very* important motors which command navcam and pancam yaw and tilt.

Glad to know the pancam tracking web is being updated again and hope there's nothing wrong with those actuators.
fredk
Yikes, I didn't realize the sol 2257 pancams were supposed to be drive direction:
http://marsrovers.jpl.nasa.gov/gallery/all...nity_p2256.html
And presumably the subframes were supposed to crop on the Sun, but it wasn't in the field of view. ph34r.gif

Those pics should've come down some time Monday, which would explain why there was no mention of this in the PS report.

Obviously the azimuth is opposite to what it should've been for drive direction, but the top of the R1 frames is just a few degrees below the horizon. So if the rover tilt is right, it's possible that the pancam altitude was correct, and just the azimuth was wrong. I'm sure we can all speculate on what that could mean, but we really need some information before our imaginations run too wild.
centsworth_II
QUOTE (fredk @ Jun 2 2010, 06:22 PM) *
...presumably the subframes were supposed to crop on the Sun, but it wasn't in the field of view....
Maybe they better stop taking those pictures of the sun. It would not be good to have a tilt actuator fail with the Pancam pointed up. At least if it is horizontal to the surface something could be done with rover movements to get desired shots.
marsophile
QUOTE (centsworth_II @ Jun 2 2010, 03:30 PM) *
Maybe they better stop taking those pictures of the sun.


I believe the shots of the sun are needed to reset the precise attitude of the rover. They would have to come up with some substitute method of determining the fine attitude to correct the errors that eventually accumulate from dead reckoning.
jamescanvin
Yikes indeed! Loss of a PMA actuator would be a major blow.

Just been looking at the PDT. Interesting that the mast head didn't move in either el or az - I'm assuming that there have separate actuators for each?

The good news is the PMA_diagnostic thumbnails do show some change in both el and az. smile.gif

Code from the PDT site below - last two columns are az and el respectivly.

CODE
*Sun find*
/oss/merb/ops/ops/surface/tactical/sol/2257/opgs/edr/pcam/1P328550110ETHAHQ8F0006L8M1.IMG::117::684::0.266769::13.4113

*Drive Direction*
/oss/merb/ops/ops/surface/tactical/sol/2257/opgs/edr/pcam/1P328550320ETHAI00P2442R1M1.IMG::118::0::0.266769::13.4113
/oss/merb/ops/ops/surface/tactical/sol/2257/opgs/edr/pcam/1P328550356ETHAI00P2442R1M1.IMG::118::0::0.266769::13.4113
/oss/merb/ops/ops/surface/tactical/sol/2257/opgs/edr/pcam/1P328550393ETHAI00P2442R1M1.IMG::118::0::0.266769::13.4113
/oss/merb/ops/ops/surface/tactical/sol/2257/opgs/edr/pcam/1P328550428ETHAI00P2442R1M1.IMG::118::0::0.266769::13.4113

*PMA Diagnostics*
/oss/merb/ops/ops/surface/tactical/sol/2259/opgs/edr/pcam/1P328726770ETHAI00P2864L2M1.IMG::118::0::0.266769::13.4113
/oss/merb/ops/ops/surface/tactical/sol/2259/opgs/edr/pcam/1P328726824ETHAI00P2864L2M1.IMG::118::0::0.266769::13.416
/oss/merb/ops/ops/surface/tactical/sol/2259/opgs/edr/pcam/1P328726869ETHAI00P2864L2M1.IMG::118::0::0.266769::13.4097
/oss/merb/ops/ops/surface/tactical/sol/2259/opgs/edr/pcam/1P328726918ETHAI00P2864L2M1.IMG::118::0::0.0800684::13.4097
/oss/merb/ops/ops/surface/tactical/sol/2259/opgs/edr/pcam/1P328726956ETHAI00P2864L2M1.IMG::118::0::0.129489::13.4097
James Sorenson
I as well hope the issue with PMA actuators is resolved.

Also I might point out, whats with these lines in the PDT? The "cleatcam" images usually point to something wrong, or so I thought.

CODE
02259 12:32:23  118     4        1 p1394.01 1                          *               rear_right_haz_cleatcam_4_bpp_pri_23
02259 12:31:37  118     2        1 p1394.01 1                          *               rear_right_haz_cleatcam_4_bpp_pri_23
02259 12:30:27  118     0        1 p1394.01 1                          *               rear_right_haz_cleatcam_4_bpp_pri_23


jamescanvin
iirc, many of the actuators share control electronics (some of the wheels with some in the arm at least). I guess that the PMA shares with the right rear wheel and so these images are another diagnostic of the same problem.
James Sorenson
Thanks James.

Was there supposed to be commanded motion of the RR wheel at the time those Haz images were taken? If so judging from the thumbnails on the tracking site, I see none.
fredk
QUOTE (jamescanvin @ Jun 3 2010, 08:52 AM) *
The good news is the PMA_diagnostic thumbnails do show some change in both el and az.

Thanks for posting that data, James. Both az and el are tiny movements, but especially elevation. But the important question is how much movement was commanded. If there seems to be a problem, I could imagine that they commanded only small diagnostic movements to start. I really hope that was the case. We'll have to wait for some info to trickle out.
Tesheiner
I see not only tiny movements but also that those were commanded on a single axis at each sequence. I.e. tilt first, then yaw. Consistent with a diagnostic to see the behaviour (voltage & current ?) of each individual motor.
There's more diagnostic planned for thisol.

The next status report may give us a hint about the issue.
fredk
Sol 2259 pics are down:
http://marsrovers.jpl.nasa.gov/gallery/all...nity_p2258.html
http://marsrovers.jpl.nasa.gov/gallery/all...nity_r2258.html
You can see movement in the pancam sequence like Tesheiner described based on the data:
Click to view attachment
You have to look closely near the bottom edge to see the elevation movements, which come first in the sequence of 5 frames.

I can't see any movement in the hazcams, though of course we don't know if any was sequenced.
fredk
This is truly bizarre: this image somehow received the wrong timestamp:
http://marsrovers.jpl.nasa.gov/gallery/all...APP0666R0M1.JPG
The timestamp decodes as the beginning of sol 2260, which can't be right since we're long past Concepcion now, and it'd be dark at midnight anyway... huh.gif
jamescanvin
Some more very small motions of the PMA in tosols tracking data. And increments in the drive number indicating some commanded motion of the RR wheel I guess.

CODE
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328904931ETHAI04P2868L2M1.IMG::118::4::-24.0682::9.73096
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328904967ETHAI04P2868L2M1.IMG::118::4::-24.0679::9.73251
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328904994ETHAI04P2868L2M1.IMG::118::4::-24.0687::9.72787
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328905039ETHAI04P2868L2M1.IMG::118::4::-24.243::9.75514
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328905073ETHAI04P2868L2M1.IMG::118::4::-24.1207::9.736
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328905249ETHAI06P2867L2M1.IMG::118::6::-24.1175::9.73351
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328905270ETHAI06P2867L2M1.IMG::118::6::-24.1175::9.73351
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328912524ETHAI06P2868L2M1.IMG::118::6::-24.1175::9.73351
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328912561ETHAI06P2868L2M1.IMG::118::6::-24.1172::9.73505
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328912588ETHAI06P2868L2M1.IMG::118::6::-24.118::9.73042
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328912633ETHAI06P2868L2M1.IMG::118::6::-24.4231::9.77827
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328912668ETHAI06P2868L2M1.IMG::118::6::-24.264::9.75331
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328912865ETHAI08P2867L2M1.IMG::118::8::-24.264::9.75467
/oss/merb/ops/ops/surface/tactical/sol/2261/opgs/edr/pcam/1P328912886ETHAI08P2867L2M1.IMG::118::8::-24.264::9.75467
fredk
QUOTE (jamescanvin @ Jun 4 2010, 12:00 PM) *
Some more very small motions of the PMA in tosols tracking data.

Thanks again, James. Notice how the movements are very small, but the az/el positions are very differrent from sol 2259. I couldn't see any thumbnails at the tracking site to confirm the change in pointing. Even if it did change, it might be due to a drive, if there was a drive. Lots of ifs.
Tesheiner
QUOTE (fredk @ Jun 4 2010, 05:04 PM) *
Notice how the movements are very small, but the az/el positions are very differrent from sol 2259.

Not that different. Here's, for instance, the last shot on 2259 and the first on 2261. I *think* the data on James' two posts used two different reference pointings: rover relative azimuth/heading, and absolute azimuth/heading.

CODE
2259::1P328726956ETHAI00P2864L2M1.IMG::12:29:50::-24.0696::9.7111
2261::1P328904931ERPAI04P2868L2M1.IMG::12:36:19::-24.0682::9.73096
BrianL
If the pancam is unusable (and I'm not suggesting that is the case), what would be affected in terms of what the rover can and can't do?
Phil Stooke
Assuming it can take pictures but only in a fixed position... the rover can still drive perfectly well, using the fabulous HiRISE images for planning, especially since we are now past the most difficult terrain. At every stop it could do a quarter or half turn, taking several Navcam and Pancam pictures to help with that planning. Only the construction of large panoramas would be prevented.

Phil
djellison
Just to correct BrianL's choice of words there - it's not Pancam that's being tested here. Or Navcam even. It's the mechanism that points them. And whatever happens to it, we still have Hazcams.
fredk
As Phil said, taking pics separated by drive arcs would presumably be a substitute for a frozen azimuth joint. Pan/navcam are currently looking forwards, so I guess doing drive direction imaging would require some forwards driving at the end of a drive.

If the elevation joint is frozen (and, as you can see, that's a big if), at least it seems to be in a relatively convenient position for drive planning. It would be much harder to compensate for frozen elevation by rover movements where we are now, and probably impossible once we get to the flat tarmac.
jamescanvin
QUOTE (Tesheiner @ Jun 4 2010, 04:26 PM) *
I *think* the data on James' two posts used two different reference pointings: rover relative azimuth/heading, and absolute azimuth/heading.


Yes, sorry, I forgot that I had used rover relative last time.

QUOTE (djellison @ Jun 4 2010, 07:20 PM) *
And whatever happens to it, we still have Hazcams.


Of course there is also the slight concern over the RR wheel as well. unsure.gif
centsworth_II
It may be a good idea to stop taking 'up' pictures, sun or tau, if there is a possibility of the tilt mechanism sticking. If the actuator fails, they will need to find workarounds anyway. Maybe it would be best to start using the workarounds now. Best to have it stuck in a horizontal position.



elakdawalla
Maybe we should wait to learn a little more about the problem and whether it is transient or permanent before we start offering the rover drivers advice, hmmm?
centsworth_II
QUOTE (elakdawalla @ Jun 4 2010, 03:44 PM) *
...hmmm?
Hmmm...

laugh.gif
I know, I really don't want to be 'that guy'.


elakdawalla
I split all the PMA actuator problem posts to this new topic. Original drive east topic remains here.
Deimos
QUOTE (fredk @ Jun 4 2010, 07:14 PM) *
If the elevation joint is frozen (and, as you can see, that's a big if), at least it seems to be in a relatively convenient position for drive planning.


The PMA's camera-bar elevation mechanism is completely separate from and less overused than PMA azimuth. But the choice was made long ago that if it were to fail between uses, it should at least fail in a "convenient" position. The end-of-activity stow formerly pointed the cameras straight down. So the cameras are dusty than they'd otherwise be, but are protected against a much worse scenario. [I'm not suggesting an elevation problem, just commenting on the convenience--JPL will release info on the state of the rover when they are ready.]

As for some other posts ... I'll second Doug's point--Pancam is just fine. Not a hint of a problem, it takes pictures and they are posted where all can see. Even if the PMA could not aim it, it is conveniently attached to a mobility system.

As Emily suggests, much needs to be learned before choices are made. Further voluntary restrictions on imaging wouldn't surprise me, neither would a big reduction in Sun images specifically. But no one said up/down is at the crux of the problem. And you also have to weigh what you lose against the risk you face. You do not explore if you are one to just stop activities while paralyzed by fear of something going wrong. Sun images (a small subset of them) let the rover aim the HGA at Earth and currently are needed for long-term mobility (stand on one foot for as long as you can; repeat with your eyes closed; now you know what a rover feels like without checking fine attitude, relying only on gravity). Sun images for tau are less key. But imagine the 2007 dust storm without knowing if it was a failure in the power system. Orbital images would have confirmed the situation eventually, but would leave open a dual-problem path you'd like to rule out. Luckily, dust storms are quite unlikely for a couple months.
fredk
Thanks for the insights, Deimos. Thinking about the possibility of losing some functionality has the cool side effect of being reminded of or learning new aspects of how these rovers work, like the solar imaging/HGA connection. These rovers are truly amazing.
fredk
Sol 2262 images are down. More hazcam and pancam diagnostics. Pancams show similar small movements in azimuth as we've seen recently, though I can't discern any elevation movements.

Hazcams show no movements in any wheels, for what it's worth.
Sunspot
Has one or more of the wheels possibly stopped working too?
Deimos
See post #8 for James Canvin's good explanation of the wheel pictures.
Ron Hobbs
The most recent Rover Update says:

"The PMA was found to be okay, which does not explain the earlier fault. So the project is continuing the investigation with more diagnostic tests."

That is good news ... I hope.
James Sorenson
That is indeed somewhat settling news....
Burmese
Scott Maxwell seems to think things have all cleared up, judging by his excited twits. Would be nice to have some actual detail...
Poolio
QUOTE
Smart engineers make apparent hardware failure vanish into thin air using brainpower. Have I mentioned that I love working with geniuses?

Yes, that sounds very promising. I especially like the word "apparent" used in that context.
stevesliva
QUOTE (Burmese @ Jun 9 2010, 04:09 PM) *
Scott Maxwell seems to think things have all cleared up, judging by his excited twits.


Sounded more like he was referring to excited geniuses.

For the record:

QUOTE
After I make my TV show "Wow!" (showcasing wonders of natural world), I'm gonna make a "House"-like show about engineers. about 3 hours ago via Twitter for iPhone

There are lots of TV dramas where doctors or cops solve hard problems with detective work. Any shows where engineers do the same? Why not? about 3 hours ago via Twitter for iPhone

Smart engineers make apparent hardware failure vanish into thin air using brainpower. Have I mentioned that I love working with geniuses? about 3 hours ago via Twitter for iPhone

God *damn*, I love working with geniuses. Sometimes I get used to how smart my coworkers are, like the air I breathe. Today I remember. about 4 hours ago via Twitter for iPhone
fredk
QUOTE (Burmese @ Jun 9 2010, 09:09 PM) *
Scott Maxwell seems to think things have all cleared up, judging by his excited twits.

Maybe the jpl engineers fixed the office coffee maker. That would get some people excited...
Stu
Re. the pause in images appearing on Exploratorium. I'm told by the site Webmaster that they're having issues getting the JPL software up and running properly on the new system. It's in hand.
fredk
Finally some real (but still unofficial) information. From here and here:

QUOTE
Had problem with Oppy's PMA azimuth motor last week; thought we'd lost it (so couldn't look left/right w/o turning).

Careful work showed PMA azimuth motor is probably fine; problem lies elsewhere. Double-checking hypothesis in today's plan.


"thought we'd lost it" - Yikes!
Deimos
Big problem, to puzzling behavior, to "nothing to see here, move along".

There is plenty of activity, including drive direction stuff, in the next plan at the pancam site. Among the first things to use the PMA, btw, is a set of sun images for attitude knowledge. Interestingly, the "mars quake" experiment could not be done while the rover was stationary. There was too much time on the IMU since the last fine-attitude check, and drift was a concern. No sun image = no seismology (that's got to be among the less intuitive MER connections).

This was still a reminder of the mortality of all the actuators and the fact that PMA az is the most overused. So there may be some things done less often. But at least it was not an actual sign of degraded PMA performance.
BrianL
And one more ScottTweet ™ on the subject:
Assuming all goes well, tomorrow we breathe a huge sigh of relief and go back to normal. Friday, more driving to Endeavour!

And there was much rejoicing.
fredk
Lot's of detail in the latest Oppy update:
QUOTE
Diagnostics were run on Sols 2259, 2261, 2262 and 2265 (June 1, 3, 4 and 8). In every case, the diagnostics indicated a healthy PMA azimuth actuator. Further investigation now suggests that the [Mini-TES] is the origin of the PMA symptom and that the PMA was just waiting on a signal from the Mini-TES that never arrived. Anomalous symptoms were observed from the Mini-TES earlier on Sol 2250 (May 23, 2010). Investigations are underway on the instrument. Meanwhile, final checkouts are being performed to return the PMA to normal use and to resume driving.
fredk
The new pics show a completely new pancam pointing direction for the first time since sol 2257:
http://marsrovers.jpl.nasa.gov/gallery/all...10P2869L2M1.JPG
http://marsrovers.jpl.nasa.gov/gallery/all...12P2869L2M1.JPG
smile.gif
Zeke4ther
well that's good news... rolleyes.gif
Phil Stooke
(crawling through desert) "Gasp! A drive, a drive... Gasp! Must have a drive..."

Phil
fredk
From the new rover update, some more details about what happened with the PMA:
QUOTE
By June 10, the troubleshooting engineers had homed in on the [Mini-TES] as the origin of the PMA symptom. Since the Mini-TES and the PMA are inherently 'bonded,' whenever the Mini-TES completes a task, it signals the PMA that it is finished and the PMA is free to move. The exchange is a kind of electronic “handshake” between the two devices. The PMA, it seems, was waiting on a signal from the Mini-TES that never arrived. Without the ‘handshake,’ it really couldn’t move.
Zeke4ther
In Engineering we call that 'deadlock'
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2024 Invision Power Services, Inc.