Help - Search - Members - Calendar
Full Version: Stereo Interpolation For Mesh Generation
Unmanned Spaceflight.com > Mars & Missions > Past and Future > MER > Tech, General and Imagery
djellison
Any ideas on this guys? I'm always looking for something new and interesting to do with the imagery but as you all know I'm more an artist and certainly not a programmer but I'm wondering if there's anything out there to generate terrain maps from stereo pairs?

http://schwehr.org/photoRealVR/example.html is the sort of thing I'm talking about - and a lot of googling has found nothing.

Doug
CosmicRocker
Doug:

I ran across a guy who wrote part of Maestro for JPL while I was on the #space channel on irc.freenode.net tonight, and I asked him if he knew of any. He said "I don't know... mipl might release theirs if you ask nicely." That's all he said.

I assume he meant http://www-mipl.jpl.nasa.gov/.
djellison
Wasnt Justin was it ?

Doug
djellison
I've done some more hunting and found some VRML data sets for the Pathfinder site - and I put it together and came out with...

http://mer.rlproject.com/mpf_3d.wmv (4 meg WMV)

Now - obviously - with a single P.O.V. one can get visibility 'shadows' behind things - when your cameras have wheels - you can go to a different P.O.V. and fill in the visibility 'shadows'. If I had another MPF data set from, say, 10m away - I could use it's terrain maps to 'fill in' a lot of the blanks.

Now - what I would DESPERATELY like to do is this with MER data - as, for instance, one could re-build the heatshield impact site quite verbosely - as we've had Navcam stereo pairs from several locations.

Doug
tedstryk
Perhaps a few of those shadows could be filled in with Sojourner data.
slinted
I've been looking into this as well, mostly since the begining of the mission. Here's a couple references i picked up, that mostly outline the points along the chain of processing same as the webpage posted regarding Pathfinder:

This one shows some of the processing steps involved in the stereo pair vision used onboard for navigation and hazard avoidance: http://robotics.jpl.nasa.gov/people/mwm/visnavsw/aero.pdf
A bit about the image correlation : http://dynamo.ecn.purdue.edu/~gekco/mars/correlator_app.html
The CAHVOR camera model, used to linearize (remove lens distortion, to line up stereo pairs so that you only have to correlate features horizontally) : http://robotics.jpl.nasa.gov/people/mwm/cahvor.html

The software used by MIPL to generate the steps are all VICAR ( http://www-mipl.jpl.nasa.gov/external/vicar.html ) programs, whose helpfiles are available online at http://www-mipl.jpl.nasa.gov/vicar/vicar300/html

As far as I can tell, here are the steps involved:

Raw image -> MARSCAHV (linearizes images) ->
MARSJPLSTEREO (computes disparity map) -> MARSXYZ (computes xyz values for each pixel)

Then, using the xyz files, they can derive roughness maps, slope maps, reachability maps, etc...

As to the data that is already available, there were supposed to be Mesh products released with the PDS (some of the PDS directories even list them) but I can't find a search or directory that actually contains them.

What was released, however, might suit your purposes. The 3 letter codes for finding them are in parenthesis first for the non-linearized, then the linearized:

5.2.4 XYZ RDR (XYZ, XYL)
An XYZ file contains 3 bands of 32-bit floating point numbers in the Band Sequential order.
Alternatively, X, Y and Z may be stored in separate single-band files as a X Component RDR, Y
Component RDR and Z Component RDR, respectively. The single component RDRs are implicitly the
same as the XYZ file, which is described below. XYZ locations in all coordinate frames for MER are
expressed in meters unless otherwise noted.
The pixels in an XYZ image are coordinates in 3-D space of the corresponding pixel in the reference
image. This reference image is traditionally the left image of a stereo pair, but could be the right image
for special products. The geometry of the XYZ image is the same as the geometry of the reference
image. This means that for any pixel in the reference image the 3-D position of the viewed point can be
obtained from the same pixel location in the XYZ image. The 3-D points can be referenced to any of
the MER coordinate systems (specified by DERIVED_IMAGE_PARAMS Group in the PDS label).
Most XYZ images will contain "holes", or pixels for which no XYZ value exists. These are caused by
many factors such as differences in overlap and correlation failures. Holes are indicated by X, Y, and Z
all having the same specific value. This value is defined by the MISSING_CONSTANT keyword in the
IMAGE object. For the XYZ RDR, this value is (0.0,0.0,0.0), meaning that all three bands must be zero
(if only one or two bands are zero, that does not indicate missing data).

5.2.5 Range RDR (RNG, RNL)
A Range (distance) file contains 1 band of 32-bit floating point numbers.
The pixels in a Range image represent Cartesian distances from a reference point (defined by the
RANGE_ORIGIN_VECTOR keyword in the PDS label) to the XYZ position of each pixel (see XYZ
RDR). This reference point is normally the camera position as defined by the C point of the camera
model. A Range image is derived from an XYZ image and shares the same pixel geometry and XYZ
coordinate system. As with XYZ images, range images can contain holes, defined by
MISSING_CONSTANT. For MER, this value is 0.0.

5.2.7 Surface Normal RDR (UVW, UVL)
A Surface Normal (UVW) file contains 3 bands of 32-bit floating point numbers in the Band Sequential
order. Alternatively, U, V and W may be stored in separate single-band files as a U Component RDR,
V Component RDR and W Component RDR, respectively. The single component RDRs are implicitly
the same as the UVW file, which is described below.
The pixels in a UVW image correspond to the pixels in an XYZ file, with the same image geometry.
However, the pixels are interpreted as a unit vector representing the normal to the surface at the point
represented by the pixel. U contains the X component of the vector, V the Y component,
and W the Z component. The vector is defined to point out of the surface (e.g. upwards for a flat
ground). The unit vector can be referenced to any of the MER coordinate systems (specified by the
DERIVED_IMAGE_PARAMS Group in the PDS label).
Most UVW images will contain "holes", or pixels for which no UVW value exists. These are caused by
many factors such as differences in overlap, correlation failures, and insufficient neighbors to compute
a surface normal. Holes are indicated by U, V, and W all having the same specific value. Unlike XYZ,
(0,0,0) is an invalid value for a UVW file, since they're defined to be unit vectors. Thus there's no issue
with the MISSING_CONSTANT as there is with XYZ, where (0.0,0.0,0.0) is valid.

5.2.11 Terrain Map RDR
Terrain models are a high level product which are derived from the XYZ files and the corresponding
image files. The terrain models are generated by meshing or triangulating the XYZ data based on the
connectivity implied by the pixel ordering or by a volume based surface extraction. The XYZ files can
be viewed as a collection of point data while the terrain models take this point data and connect it into a
polygonal surface representation. The original image is referenced by the terrain models as a texture
map which is used to modulate the surface color of the mesh. In this way the terrain models can be
viewed as a surface reconstruction of the ground near the instrument with the mesh data capturing the
shape of the surface and the original image, applied as a texture map, capturing the brightness
variations of the surface. Specific terrain model formats such as VST, PFB, DEM and others can be
viewed as analogous to GIF, TIFF or VICAR in image space in that each represents the data somewhat
differently for slightly different purposes.
5.2.11.1 VST Terrain Wedge (VIS, VIL)
The ViSTa (VST) format consists of one terrain model for each wedge (stereo image pair), in a JPLdefined
binary format suitable for display by SAP. Each file contains meshes at multiple levels of detail.
5.2.11.2 PFB Terrain Mesh (ASD?, ASL?)
The Performer Binary (PFB) format facilitates the representation of a terrain surface as polygons,
optimized for use by the RSVP tool. The number of polygons at any one time may vary according to
site specific features, such as small rocks versus large boulders.


I know you said you weren't looking to do any programming, but that may be the only way to achieve a solution that doesn't involve waiting for the PDS release. There seems to be a lot of code available with stereo vision algorithms, but one in particular that might be helpful is the Open Computer Vision Library available at : http://sourceforge.net/projects/opencvlibrary/
CosmicRocker
QUOTE (djellison @ Jan 7 2005, 03:40 AM)
Wasnt Justin was it ?

It sure was. You might ask him if he can help you get that software.

BTW, that was a really nice VRML animation.
djellison
Looking at http://pdsimg.jpl.nasa.gov/cgi-bin/MER/sea...RELEASE_ID=0001 - terrain wedge and terrain mesh are listed as N/A

Poo

I may get in touch with Dr JB and see what the word is on these things

Doug
Bjorn Jonsson
This (generating terrain from stereo imagery) is something I'd be very interested in if I had the software to do so.

Ted Stryk mentioned something related in this thread:
http://mer.rlproject.com/index.php?showtopic=604

It has even occurred to me to attempt to write a program myself to do this. In theory it isn't difficult, the idea/algorithm is actually fairly easy to understand. However, in practice there are lots of complications like image noise, inaccurate geometric information and shadows to name a few. So in prcatice this isn't exactly easy to do.
djellison
It shouldnt be too hard if one starts with the depth maps - take this for instance

http://anserver1.eprsl.wustl.edu/navigops/...FF09BXP1933L0M1

It has RNL file IMG's that dont seem to make a lot of sense when img2png's

The XYZ files come in at 12+Meg each smile.gif

The data's sort of there, just not 'accesable'

Doug
erwan
I have generated some months ago a mesh for Eagle crater ("hand made", unfortunatly, from Opportunity images with help of Excel). A color panorama was wraped upon the mesh and a short movie, a "walk upon westside of Eagle crater" was synthetized with help of POV-ray. Here is the movie : outcrop02.avi
. Hope you will enjoy!
ilbasso
Erwan, I can't open the file -- the download button doesn't work in either Firefox or IE. Could you repost, please? I'm looking forward to seeing the movie!
djellison
I can download the file - but it wont play smile.gif

Doug
erwan
The movie requires codec intel indeo video 5.04 (I guess it's the problem). Playing is easy with realplayer, winamp, windows player on my computer. I cross my fingers....
MizarKey
That was pretty cool! Hope you do one for Endurance...

Eric P / MizarKey
aldo12xu
Hey Erwan & djellison,

Those are great videos. It'd be amazing if NASA would take the time to generate similar ones themselves, or at least make all the necessary data readily available. Has anybody mentioned the following Quicktime VTR links. Is anyone familiar with the process? What cheap program can be used for PCs to generate similar renderings? How could I convert them to .mov or .avi files?

http://www.panoramas.dk/mars.html
lyford
I use MakeCubic for Mac OSX, but I am sure there are solutions for PC as well.

It's fairly easy to take the panoramas that NASA puts up and convert them to a QTVR file - I have a few at

http://homepage.mac.com/lyford/FileSharing2.html

and plan to upload even more when I get a chance.
erwan
I dont know anything about Quicktime VTR generation. Anyway, although certainly tedious for amateurs like us, an ideal solution to render real 3d movies from MER datas is to compute a mesh, then wrapping an image on this mesh using a program like POV-ray. I found many solutions in this forum to facilitate this process, thanks to Doug, Bjorn, Slinted... (see topics Mer files and stereo interpolation...)
I will try maybe to compute a a future "3D image" following these steps:
-download useful XYZ or range IMG files from PDSnode
-convert these files to PNG using Bjorn program
-Choose multiple XY key points for each image, and report these points with corresponding Z values (readed in IMG file) on a table (using Excel for example)
-generate a text file convenient for the mesh, for example in POV-ray language

Thanks for any suggestion and improvement

Erwan
Sunspot
QUOTE (lyford @ Feb 1 2005, 05:30 PM)
I use MakeCubic for Mac OSX, but I am sure there are solutions for PC as well.

It's fairly easy to take the panoramas that NASA puts up and convert them to a QTVR file - I have a few at

http://homepage.mac.com/lyford/FileSharing2.html

and plan to upload even more when I get a chance.

Hi lyford, what software are you using to view the processed/calibrated IMG files from the MER PDS? I found an OS X version of NASA VIEW. Also I think GraphicConvertor might support IMG?
lyford
QUOTE (Sunspot @ Feb 1 2005, 10:18 AM)
QUOTE (lyford @ Feb 1 2005, 05:30 PM)
I use MakeCubic for Mac OSX, but I am sure there are solutions for PC as well.

It's fairly easy to take the panoramas that NASA puts up and convert them to a QTVR file - I have a few at

http://homepage.mac.com/lyford/FileSharing2.html

and plan to upload even more when I get a chance.

Hi lyford, what software are you using to view the processed/calibrated IMG files from the MER PDS? I found an OS X version of NASA VIEW. Also I think GraphicConvertor might support IMG?

I haven't used the callibrated files yet, I just use the press release ones right now. Quick and dirty, as they say.

GraphicConverter is the best! Here is the list of image formats it knows. It can do a NASA raster metafile, but I am not sure that's what you want, as well as something called IMQ.

ImageBrowser may be what you are looking for, and there is a whole slew of graphics (mostly FREE) listed at:

http://whatonmars.com/archive/_12-2004/07arc.htm

NASAView is for OS9 if I understand, but it may run in Classic mode.

I wish I knew more, I am really rank beginner hobbyist on this front. Now, if you needed MUSIC software.... Anymore and I will REALLY be OT.....
erwan
QUOTE
-convert these files to PNG using Bjorn program
Ooops! Bjorn program doesnt convert 32 bit IMG files to PNG... Anyway, thanks to Lyford (previous post, whatonmars link), one may download ImageJ software.
So Doug: with ImageJ we can easily open greyscale 32bit img files from PDS imaging node :file - import - RAW - *RNL*.img - image type: 32-bit real. Moreover, doing this, ranges are expressed in meter as intensity values, and measurement tool allow to save multiple pixel measurement on a text file.
At this step, I believe two improvements could help to compute more easily meshes :
1. Writing a plugin for ImageJ, allowing an automated mean range measurement for each, say 5*5 or 10*10 pixel squares from RNL images and edit a text file as the result : I cannot do such a work!
2. Writing for example an Excel macro who can transform the text file in true XYZ data, then in mesh text for each Navcam or Pancam images. I will try...
Finally, we may merge each image mesh with help of image header information (mast azimuth and elevation, MER attitude and location.
Any comment? What about the project to compute and share meshes for the two MERs?

Erwann
Sunspot
Heres what the PDS archive site says about NASAVIEW:

NASAView is a Planetary Data System archive product display tool that runs on Sun/Solaris, Windows/NT/95/ME/98/2000/XP, and Mac OS9/OSX platforms .

Does GraphicConvertor work with IMG files that youve tried?

GraphicConvertor is listed as Shareware, how much is it to purchase? I didnt see any instructions for payment when looking at the download page.
erwan
Sunspot I can't answer, I'm a PC user... You could visit GraphicConverter homepage. If you cannot open PDS IMG file directly in GraphicConverter, maybe you could open the file as a RAW format file...
lyford
QUOTE (Sunspot @ Feb 1 2005, 04:43 PM)
Heres what the PDS archive site says about NASAVIEW:

NASAView is a Planetary Data System archive product display tool that runs on Sun/Solaris, Windows/NT/95/ME/98/2000/XP, and Mac OS9/OSX platforms .

Does GraphicConvertor work with IMG files that youve tried?

GraphicConvertor is listed as Shareware, how much is it to purchase? I didnt see any instructions for payment when looking at the download page.

I can't get GraphicConverter to open .img files yet - working on it! I don't have the right settings for the RAW files... but it should work - I can see it in there....

You can download the demo which is fully functional - it takes longer each time you start it though until you pay the 30 dollar (US) shareware fee. I did and am VERY happy with this program. Even opens my old Atari files! laugh.gif

You have to download the Multiplatform version of the PDS tools to get the OSX native version of NASAView. Seems to work fine.

There's MultiSpec which seems to be able to import RAW files if you know the settings.

http://dynamo.ecn.purdue.edu/~biehl/MultiSpec/

erwan You got ImageJ to import them>? I seem to not be doing something correctly...
erwan
Lyford: I can open one band (greyscale) 32-bit IMG images with ImageJ (for example ranges RNL linearized images). I import the file requesting import raw and select 32-bit real format when requested. Images seems totally black but value datas are there and correct (please read Bjorn comments on "mer files" topic).
lyford
QUOTE (erwan @ Feb 2 2005, 05:18 AM)
Lyford: I can open one band (greyscale) 32-bit IMG images with ImageJ (for example ranges RNL linearized images). I import the file requesting import raw and select 32-bit real format when requested. Images seems totally black but value datas are there and correct (please read Bjorn comments on "mer files" topic).

Thanks - in my haste I forgot this was a 2 page thread. blink.gif
Looks like I have some bedtime reading!
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2024 Invision Power Services, Inc.