Help - Search - Members - Calendar
Full Version: How Much Data Do You Have?
Unmanned Spaceflight.com > Mars & Missions > Past and Future > MER > Tech, General and Imagery
Sunspot
Since the rovers landed i've saved 559MB of Spirit data and 402MB of Opportunity data, total: 961MB .... and im on dialup!! ohmy.gif ohmy.gif

It might actually have been slightly higher, I did lose some data when files wouldn't save properly - including one of those HUGE colour pans of Endurance Crater dry.gif

..........I only have 35.6MB of Cassini data so far wink.gif
slinted
way too much! heh

I'm going to need to get a new harddrive before the January release of (it almost boggles the mind) sols 90-180 on *both* rovers which by time spent on the surface, this one release will double the coverage of all PDS releases up to this point.

From the left eye of the pancam alone, counting just the raw jpg release (up to now) and the PANCAL IMGs from the PDS release (sols 1-90), i'm carrying 15.1 gig of images and probably triple that in working-copy, derived, or JPL color/mosaics. Its an overwhelming volume, produced just by the 1 megapixel left eye pancam CCD alone.

I can't help but think forward to the MRO and MSL. From the Mars Reconnaissance Orbiter telecommunications system webpage :
QUOTE
Over its two-year primary science mission, the spacecraft is predicted to transmit more than 34 Terabits--that's 34 million million bits. To put it another way, that's more than all the data transmitted by all previous JPL spacecraft put together!
Baltic
I'm at 1,1 GB MER data and 53 MB Cassini data so far. biggrin.gif

And of cause a couple of pictures from Mars Express and none from Beagle-2. sad.gif

Tom
djellison
I've only got 1.3 gig of MER stuff at the moment

With most of the panoramas and things that I've done - I downloaded a couple of dozen jpgs to a folder on my desktop - made the panorama - then deleted the folder as I knew I'd have to start over when it came to PDS releases.

When it comes to PDS releases, I tend to download only what I want, but when its >2meg an image - a 3 x 3 mosaic in RGB can end up being 54 meg before working on it smile.gif

Still none of the mpl and other calibrated image formats for sols 30-90 that we saw with sols 1-30 on the PDS yet sad.gif

Doug
Bjorn Jonsson
About 2 GB of Galileo images.

Currently less than 1 GB of MER stuff.

A little over 500 MB of Cassini images. This will increase to many gigabytes once they get released via the PDS (hopefully on DVDs too since downloading tens of thousands of images isn't very fun, even with a mass downloader).
djellison
I have the three Pathfinder IMP Cd's and the two Viking CD's as well - does anyone know if they're planning to release MER stuff similarly?

Doug
Bjorn Jonsson
Speaking of CDs I have most of the Voyager CDs (30+) - actually I'm in the process of copying them to DVDs and I also have lots of them on my HD.

Also 14 Viking disks (MDIM etc.).

I wonder whether people tend to assume that if someone has on the order of 100,000 images on his computer they must be porn biggrin.gif
azstrummer
Wouldn't someone that had 100,000 porn images on my computer be dead of exhaustion?
djellison
LOL - yeah

"Mr Ellison was arrested by the police for having over 100,000 explicit images of 'rat holes' 'trenching' 'L7 Mosaics' and 'idd work volumes' "

wink.gif

Doug
M_Welander
A couple of years ago I spent the entire summer downloading the Voyager image CDs, image by image. Oh, how I wish they'd put them up as .ISO files. smile.gif
OWW
QUOTE (djellison @ Dec 7 2004, 11:22 PM)
"Mr Ellison was arrested by the police for having over 100,000 explicit images of 'rat holes' 'trenching' 'L7 Mosaics' and 'idd work volumes' "

LOL biggrin.gif, that sounds nasty indeed. Make sure they don't find the 'microscopic images' and 'rear and front hazards', not to mention the 'night observations'!
djellison
Starting 'properly' on the book - I now have both complete RAD archives from both rovers downloaded via FTP. About 20 gig alltogether

Doug
Sunspot
20 GB?!!!!!!!!!!! blink.gif How long did that take to download, what connection speed do you have?
Sunspot
Tried making some colour images from RAW data dowlaoded from the MER Analysts website

first attempt came out ok I think - s shot of the pancam calibration target with L3,L5 and L7 filters (red green and blue)

Second attempt doesent look too good- what went wrong, did I use the wrong filers?
Sunspot
Second attempt
Sunspot
Second attempt
slinted
One of the good things about using the PDS is that you can get the images with the effects of differential exposure time, look up tables, flat fielding and many other sources of distortion. I assume from the name of your previously posted file that you downloaded the ESF versions of the images. This is the precalibration what-we-got-from-the-rovers version of the images. If you want the calibrated versions, look for the ones with RAD labels (Radiometrically-corrected RDR calibrated to absolute radiance units which are listed under science products, the most accurate calibration) or MRD (MIPLRAD Radiometrically-corrected under operations products, the JPL in-house product)

As I stated in another thread, if you are using the RAD files its necessary to go one step further than what is produced directly in the IMG file. There are offset co-efficients that tell you how many radiant units each particular image represents, so one has to apply these co-efficients in order to equalize the scale between frames.
djellison
20 gig's taken about 4 - 5 days. smile.gif

Dont use L3,5,7 - use L456 if you can, if not, L256, or L457

I've been downloading using the 1meg connection at work, and my 512K connection at home smile.gif

Doug
Sunspot
Thanks for the responses, its sounds quite complicated lol, think I might stick to admiring the work of other forum members smile.gif

It took nearly 10 minutes to download the three images for my second pic on my dialup connection lol - and I think they are subrames too blink.gif I opened them in NASA VIEW resaved as GIFs, opened them in Paint Shop pro 7 split them into RGB and then recombined each RGB frame.
slinted
I really didn't mean to make it sound overwhelming...actually, in terms of ease of use...Just download the MIPLRAD files, they are all scaled equally so you don't need to do any fiddling after you get them. Its not a matter of the data itself being hard to work with, its the matter of *finding* the data you want. Once you do, its not bad at all. Thats what I love about these missions...its all right there for us to play with.
Sunspot
Here's my second attempt using the RAD frames and L4,L5,L6

Looks a bit better, not as much blue
Nix
This is my first post here. Hi everybody I'm Nico.
I collected about 30 Gb with processed imagery included about 60 Gb.
Am I obsessed?
Sunspot
QUOTE (NIX @ Jan 5 2005, 03:41 PM)
This is my first post here. Hi everybody I'm Nico.
I collected about 30 Gb with processed imagery included about 60 Gb.
Am I obsessed?

YES biggrin.gif
djellison
I'm finding it harder to find interesting colour mosaics to put together for Opportunity ohmy.gif Nothing quite as interesting as the Spirit ones anyway.

Doug
remcook
NIX - do yu have some work to show? cool.gif
Nix
Yeah I might look into the folders and select some stuff for web-purposes.
It's good to be a Mars-addict.
Nix
Spirit Pancam SOL 88 anaglyph.
Nix
oops.
djellison
I've just finished downloading all the data up to sol 180 ready for archive onto DVD

31.8 gig smile.gif

Once I've made PNG's out of that lot it'll be 45 gig probably.

I'm going to break it down into 30-sol pieces for each rover with the PNG's pre-made.

Anyone know if the calibrated Navcam and Hazcam data is available on an FTP server anywhere - I couldnt find an appropriate folder from the place directed to from the Workbook.

Doug
slinted
QUOTE (djellison @ Mar 29 2005, 12:53 AM)
Anyone know if the calibrated Navcam and Hazcam data is available on an FTP server anywhere - I couldnt find an appropriate folder from the place directed to from the Workbook.
Doug
*

The PDS imaging node has the MRD (MIPL Rad, there aren't any Cornell RAD files for navcam or hazcam) files in an http interface. I don't believe there is an ftp interface to it, but you can get directly to the directories to bulk download a sols worth at a time.
ho for hazcam, no for navcam...they are in the rdr folders for each sol.

http://pdsimg.jpl.nasa.gov/data/mer/
djellison
Ahh - thanks for the heads up. I might ask the PDS if they're going to put them on their FTP server. I've just emailed them to ask about the next MGS MOC release( due any day now I believe )

Get this - I PNG'ed Opportunity Sols 1-17 before filling a DVD ohmy.gif (I'm backing up the IMG's, the Left Eye PNG's, Right Eye PNGs in their own folder and thumbnail PNG's in their own folder for every sol)

Luckily I got from 18-40 with the second disk ohmy.gif

It's worth doing - I want this stuff backed up and get-able without going back online to get it, but wow - there's a LOT of it.

I think Navcam and the other imagers will take up perhaps half as much space - maybe even less.

It'll make a nice graphic demonstration of the volume of data they've sent back at my next talk - as I get out about 40 DVD's...

"This is the uncompressed data volume sent back by the two rovers in their first 6 months. No - these are not CD's - these are DVD's biggrin.gif"


doug
Marcel
QUOTE (djellison @ Mar 29 2005, 11:30 AM)
Ahh - thanks for the heads up. I might ask the PDS if they're going to put them on their FTP server. I've just emailed them to ask about the next MGS MOC release( due any day now I believe )

Get this - I PNG'ed Opportunity Sols 1-17 before filling a DVD ohmy.gif (I'm backing up the IMG's, the Left Eye PNG's, Right Eye PNGs in their own folder and thumbnail PNG's in their own folder for every sol)

Luckily I got from 18-40 with the second disk ohmy.gif

It's worth doing - I want this stuff backed up and get-able without going back online to get it, but wow - there's a LOT of it.

I think Navcam and the other imagers will take up perhaps half as much space - maybe even less.

It'll make a nice graphic demonstration of the volume of data they've sent back at my next talk - as I get out about 40 DVD's...

"This is the uncompressed data volume sent back by the two rovers in their first 6 months.  No - these are not CD's - these are DVD's biggrin.gif"


doug
*


Doug, I can't wait to buy your "coffee-table" book. Any news on that great idea ?
djellison
None yet - got a few things to get out the way that'll make my submission to publishers a little more positive

Doug
Marcel
QUOTE (djellison @ Mar 29 2005, 01:18 PM)
None yet - got a few things to get out the way that'll make my submission to publishers a little more positive

Doug
*


Don't know exactly what "things" you're refering to, but i'd say your work, the datavolume and the (your) story behind it speaks for itself. What more positive do you need. I'm sure you'd make it to an adventure for the reader and this would surely get the real beauty of the missions to the public, which would give our MER's (and the people behind it) the attention they deserve.

If i was a publisher, i'd start right away !! I'm sure it would be a best-seller !!

Or is it you.....waiting until the last bit gets down in order to get a complete story...?
djellison
well - a book couldnt be finished until the rovers die really - i'd want to tell the full story

BUT - I have a speaking engagement in London on April 23rd - and something a little more exciting I'm trying out for on April 14th.

The former is just another feather in my cap ( Speaking at a BAA meeting is quite a big deal ) - and the later may give me some standing in the field of science and the media - which would give my 'pitch' to a publisher more clout

Doug
Marcel
QUOTE (djellison @ Mar 29 2005, 02:13 PM)
well - a book couldnt be finished until the rovers die really - i'd want to tell the full story

BUT - I have a speaking engagement in London on April 23rd - and something a little more exciting I'm trying out for on April 14th.

The former is just another feather in my cap ( Speaking at a BAA meeting is quite a big deal ) - and the later may give me some standing in the field of science and the media - which would give my 'pitch' to a publisher more clout

Doug
*


Ah, seems to me you're carefully planning the whole idea, including the network around it. Since you'll wait until the end......I suddently hope it will take another 1000 sols until it's published ! laugh.gif
djellison
I dont want to go to a publisher and look an idiot you see "Hello - I want to make a book" doesnt say much

but a renown speaker and so forth - makes more sense smile.gif

For those who've not heard of it - I'm going in for the London heats of www.famelab.org smile.gif


Doug
Sunspot
QUOTE (djellison @ Mar 29 2005, 02:13 PM)
The former is just another feather in my cap ( Speaking at a BAA meeting is quite a big deal ) - and the later may give me some standing in the field of science and the media - which would give my 'pitch' to a publisher more clout

Doug
*


Are you going to be on TV? biggrin.gif smile.gif
djellison
QUOTE (Sunspot @ Mar 29 2005, 03:44 PM)
Are you going to be on TV?  biggrin.gif  smile.gif
*


If I'm any good smile.gif

Doug
Sunspot
QUOTE (djellison @ Mar 29 2005, 03:44 PM)
QUOTE (Sunspot @ Mar 29 2005, 03:44 PM)

Are you going to be on TV?  biggrin.gif  smile.gif
*


If I'm any good smile.gif

Doug
*




The Sky at Night?
djellison
QUOTE (Sunspot @ Mar 29 2005, 03:46 PM)
QUOTE (djellison @ Mar 29 2005, 03:44 PM)
QUOTE (Sunspot @ Mar 29 2005, 03:44 PM)

Are you going to be on TV?  biggrin.gif  smile.gif
*


If I'm any good smile.gif

Doug
*




The Sky at Night?
*



Nope - www.famelab.org

Doug
lyford
Doug -

You should make a "pop up" book - comes in the shape of a tetrahedron and unfolds - the solar panels form the cover and the pages are in side.

On second thought, maybe not.... tongue.gif
wyogold
good luck doug.

I have less data than I want, and more than I need.

I have about 5-7 gig of images/papers/articles/pdf in my "space" folder. I have about another 1 gig from mmb after finding it a few weeks ago. I also have a working 500meg-2gig working folder from comet hunting with soho. All this has been dl'ed on dial up. blink.gif If i had dsl/cable i would have to buy several more hd's. Being on dial up keeps my want and needs in check. biggrin.gif

scott
Bubbinski
Man sakes alive, I've downloaded all the raw images from both rovers with Midnight Mars Browser, and I had to use 2 DVD's to get them all backed up. So that's, what, 8 gigs, and I've got another three gigs of NASA public release MER, Cassini, Voyager, Galileo, Magellan, HST, and other images. And almost 200 megs worth of space shuttle and Apollo/Gemini/Mercury images on top of that.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2024 Invision Power Services, Inc.