Because it has become a forum FAQ, I've created this sticky thread containing information on where to obtain new base images for Opportunity's traverse for Google Mars, and for discussion on creating new ones. I will continue to add links to new base image layers to this first post as they become available.

New users: Download this 4-MB kml file and open it in Google Earth
Then go to the last post in the Opportunity Route Map thread for the latest traverse map
You need to download the new KML file each time in order to follow Opportunity's peregrinations.

Google Mars comes with a color base image mosaic created from HRSC imagery. In addition, there is an inset full-resolution HiRISE image covering the area from the landing site at Eagle crater, through Victoria, up to the point between sol 2040 and 2041 (just west of Mackinac) where Opportunity drove off the map.

Unfortunately, this Victoria crater HiRISE layer included within Google Mars is not perfectly registered to the HRSC base map. As far as is known, there is nothing to be done about that. Both John Cody's image layers and Eduardo Tesheiner's traverse maps are aligned with the inset HiRISE layer included with Google Mars, NOT to the HRSC base map.

In June 2009 SFJCody posted a reduced-resolution mosaic of HiRISE tiles that cover the entire future traverse area including Endeavour's rim. I have made some small modifications to that map and have hosted it in a single file here (17 MB). Download the file, run Google Earth, select the Mars view, and File>Open the KMZ to view it.

In September 2009 SFJCody posted another HiRISE base image, this one at full resolution, covering the Western Route and reaching not quite all the way to Santa Maria. Here is a link to the kml file (4 MB). Here is a link to a zipped version if you'd prefer to have it locally (256 MB) and I also wrote a blog entry about it.

During the discussion below, in late 2010, I created a small tile that covers just the immediate area around Santa Maria. Here is a link to the kml file covering that region.

In February 2011 Eduardo Tesheiner provided another set of base images covering the area from Santa Maria to Endeavour's rim. Here is a link to the kml file. If you would like to work from a local copy, you can download this 75 MB zip file and unzip it to a folder on your drive, then open the file PSP_010341_1775_RED.kml within it.

All three base images can be loaded at once using this kml file (the same one that is linked to at the very top of this post).

OK, now that I've posted the existing maps, here is a call for help with future ones. At her current pace, Opportunity will run off of SFJCody's high-resolution base map in about two months. He can't do the next map, and I tried but couldn't figure out how to make any region file generators work within the amount of time I allotted myself to spend on the project, so someone else needs to try to figure out how to do this!

The first step is to get the images. I can provide those, thanks to James Canvin, who processed the small piece of PSP_005423_1780 that covers the area around Santa Maria plus the entirety of PSP_010341_1775 which covers the rest of the way to Endeavour plus the entire western rim into 8192x8192 JPEG format subimages, each of which overlaps by 10% with the next. They can be downloaded here:

Santa Maria (PSP_005423_1780, 33 MB)
Endeavour 1 of 3 (PSP_010341_1775, 183 MB)
Endeavour 2 of 3 (PSP_010341_1775, 199 MB)
Endeavour 3 of 3 (PSP_010341_1775, 148 MB)

Then, SFJCody had the following advice to me on how to get these into Google Mars:Note that, for the base map to be useful to Eduardo and all of us who follow his maps of Opportunity's progress, the images should NOT be registered to the color HRSC base map included within Google Mars, but instead to the low resolution HiRISE base map linked to in the first post in this thread.

Which is so frustrating, I really appreciate your endeavors ,
I hope Google Mars catches up soon...I mean, shouldn't they ?...

I'm wondering if it is really necessary to overlap the subimages? When I had the idea of making my GE version of route map I spent some time looking on how to make additional background pictures, specially on the topic of "region files". IIRC, GE splits their CTX and HiRISE overlays on subimages with no overlay so, in principle, there's no need for the 10% overhead.

Example: Load the CTX picture P01_001414_1780_XI_02S005W on GE by selecting first "Spacecraft Imagery > CTX Image Browser" then click on the yellow square corresponding to that image. Then click on "Load this image" on the pop-up window and have a look to the respective KML file structure.


PS: I'll have to include a link to this thread on future map updates.

The only reason I ever used the overlap was because of my crude method of matching the pieces by eye. There are bound to be better ways to do this.

Wow! I knew that was a *lot* of work put on creating the map layer but not that much!

Why can't Google do this work? Has anyone asked them?

They're too busy quietly taking over the world, one website, one byte, one pixel at a time...

Good idea tho, we should definitely try encouraging them to join in the adventure.

Is there any reason we should expect them to? Have YOU asked them?

Because it seems like an obvious benefit to all users of Google Mars. And I would not be so presumptuous as to be the lead voice on this when I have had nothing whatsoever to do with this imaging to this point. I'm just putting it out there as an idea for consideration, because I think Google has the resources to take this on.

I just tried "stitching" in the first image. GE's image overlay tool is very clunky, very difficult to make a perfect match with the previous detailed overlay that we've been enjoying. I wish I had more time to monkey with this. Any reason why they have to be applied in these broken up tiles? Why can't the original large file be overlayed in one fell sweep? Probably an obvious answer somewhere...

You answered yourself.



It's several hundreds of MB.

And back to the main topic everyone, here's a good reference on how to work with "region files" in GE. It's the key point to have an extended and high resolution map coverage on GE. It's not too complicated to implement but the process must be automated somehow since we are talking about dozens or perhaps a few hundreds of regions.

Working with Regions

I'm not familiar witht the images in question, but presumably the full hirise frame is much larger than we'd need to cover the proposed future route to Endeavour. Could it be cropped down to a single image that covers, say, a kilometre or a half kilometre on either side of the proposed route to Cape York? Would that be small enough to handle, without needing to piece together smaller subframes?

It would be nice to have the west rim of Endeavour as well, but that's still some time away for Oppy, and by then perhaps the folks at google will update GM...

So, we are talking more or less about nine square kilometers which, at 25cm/pix, corresponds to a picture 36000 x 4000 pixels big. Ouch! Too much for a single picture, I would say.

The region file generators that make images for Google Maps take a large original image, like our 8192 by 8192, and chop it and/or downsample it into different versions that get displayed at different zoom levels. When you are zoomed way out, Google Mars might display the photo in a single tile whose source file is only 256 pixels square. When you zoom in some, it'll display tiles from it that were chopped into, say, 1024 by 1024 chunks, then downsampled to 256 pixels square. When you zoom all the way in, you'll see the screen tiled with 256 by 256 images at the original resolution. This vastly improves performance and is what allows you to sweep around and zoom in and out so quickly. It's a technique that GIS applications have done for a long time.

Are subimages of the path between Oppy's current location and Santa Maria available? They work much better for me than using the overlays SFJCody provided for GoogleMars which slows to a crawl when the high-res overlays are enabled.

I still have them on my disk, in fact they were still on my web server until last week when I deleted them to make way for the new ones, doh! When I get the chance I'll dig them out.

James

I get it now, somewhat. I downloaded MapTiler on my Mac. It worked great, chopped up the large file with Santa Maria crater into the tiles-pyramid, but I can't figure out what coordinates to use in the "Georeference" box in MapTiler. It places the top image of the pyramid over the whole Earth, completely smothers it, when using the default coordinates. Anyone know the coordinates to use that would place the tiles near Santa Maria crater on Mars? If I could get the tiles to land near Santa Maria, I could then fine-tune their position, I'm assuming. Maybe it has to be done by tweaking the code in the KML file.

Wow, downloaded and tested MapTiler and I think that's *the* tool we need.
Regarding your question about coordinates: it uses a similar definition of bounding box as in KML. In this case you should input the north, south, east, and west coordinates separated by spaces. Just for testing I used the lat/long coordinates of a KML file I had here which corresponds more or less to the Meridiani landing site: -1.899331338 -2.44643146 -5.298233309 -5.552521575. You can do the same. The result is the picture of Santa Maria all over the landing site but, more important, it zooms in and out smoothly.

Now, about the real coordinates...
I would do the following (and will probably do later ondone):
1) Look for a feature near the right edge of SFJCody's map extension, annotate its lat / long coordinates, search for the same feature in the picture with Santa Maria and annotate its (x, y) coordinates.
Click to view attachment Click to view attachment

2) Calculate the angular size of the picture and of a single pixel. It has 8192 x 8192 pixels so it covers 2048 x 2048 meters. Given the mars circumference size (TBD) the picture covers X degrees and a single pixel covers Y degrees.

R = 3396.2km (equatorial), 3376.2km (polar)
X = 2.048 / (2*PI*R) * 360deg.
Y = X / 8192

3) Calculate the lat / long coordinates of all four borders using the results of 1) and 2).

north -2.160015615
south -2.194771186
west -5.45866131
east -5.424110413

Now, let's try it...

Bingo!

Click to view attachment Click to view attachment Click to view attachment

Thanks Walfy for finding the right tool! Map Tiler
Emily, I think the biggest problem to have the extended maps is solved but I still see an open point which is the transparency of the black areas outside the map-projected picture. The original 8k x 8k images should be available in PNG format with the transparency already set.

Nice progress! I had a feeling you would propel this forward. I was able to set the black part of the images to transparency by first opening the image in Photoshop, then going to "Save for web & devices." In there you can choose the transparency to the black when you set it to save as a GIF. Not sure if it's the best way, but SFJCody's images were done similarly because they have the same very small transparency artifacts along their edges when you zoom way in.

To match the lighting of the new images with the old, maybe you could open the image of SFJCody's in photoshop, the one nearest Santa Maria, then increase the canvas size of the image large enough to place the new and slightly darker image with Santa Maria in it. Once they are in there side-by-side, adjust the curves settings on the Santa Maria image until it matches SFJCody's image. Then crop out SFJCody's image, then you have a color-matched Santa Maria image.

But personally, I don't think it's necessary to change the brightness levels of all the images. If you're going to do it for the first one, then you're going to have to do it for all of them! It's not a big deal for me if the end result has 2 large image overlays of different brightness side-by-side. It will save a lot of moil and toil to skip the step of matching the brightness levels.

Those are my two bits. OK, back to my job. Otherwise, I'd be tinkering with all this... maybe tonight.

I have Serif Photoplus instead of Photoshop but AFAIK they have similar capabilities. GIF is probably not the best option because if you set black = transparency all dark areas *inside* the real image are also made transparent. Another option is to edit the picture, set/fill all the contour as transparent and save as PNG. Simple but ... it's too much for my computer; the image is too big.

It's not a problem, as there are no pixels in the actual image of the surface area that are as black as the background pixels outside the image. I tested this using the "Magic Wand" selection tool in Photoshop, which you can set to select only pixels of the same color of an image. It only chose those pure black pixels outside in the background, nothing on Mars' surface area. One drawback of GIF image format is that it reduces the number of pixel types to just 256. The GIF still look identical to the non-GIF image, at least to my eye, but the thought of the color gamut reducing to just 256 colors sounds bad to me!

Your PNG suggestion sounds great, if I have time tonight I'll try it . My Mac can handle the large files.

Presumably these are grayscale images originally, so that means 8 bits per pixel, or 256 shades of grey. (Even if they had 12 or more bits/pixel originally, you won't be able to take advantage of those on GE.) So you won't loose any information with a gif (gifs are lossless). Even if the images you're starting with are colour 24 bits/pixel, or 256^3 colours, all those millions of extra colours are completely superfluous. Depending on what format you chose for the new images, you may actually save some on the filesize by converting the original image to 8 b/p greyscale first.

For the sake of those downloading on slower connections, it may be worth checking which format would give the smallest final filesize. Jpegs would be much smaller than gif or png, but you won't get transparency with the jpegs. Is the transparency really necessary? Does it just mean getting those black edge regions like in Tesheiner's pics above? Can you just crop away all of the black regions, at the expense of loosing a small part of the new map? Or just live with the black regions?

Good to know that because I saw an option on Map Tiler to choose a transparent color. In that case there's no need to have a GIF or PNG image, just select black as transparent and voila!



That's correct but I think there's a misunderstanding here. The image files I'm talking about are those 8k x 8k which are the *source* for the tool. It then chops the image and the result is a quite big number of very small 256x256 images in JPEG and (that's what I read) PNG files with their corresponding region files. Therefore, once we finally get these map extensions working, the "end users" should not be concerned about filesize or type but just download the resulting KML and JPEG files.

My appologies if I'm missing something here! When I look at the files in the folder for the previous map extension, "MER-B Map extension v2.0" (which Emily linked to in her first post above), it contains close to 5000 individual image files. Almost all of them are png files (there are 6 jpegs). The big majority of those png files don't include any of the black regions near the edges, that you'd want to be transparent. So it seems to me that we could reduce the filesize and download times for the next extension by a large factor if these files were jpegs instead of pngs. (I see close to a factor of 4 reduction in filesize going from png to 80% quality jpeg.)

Presumably we could eliminate the need for pngs altogether if we cropped away the black as I mentioned above (or lived with it).

Again, sorry if there's some obvious reason those pngs couldn't be substituted with jpegs... Is it that people want a lossless format to avoid the compression artifacts of jpegs?

Oh, now I see our misunderstanding. You are talking about the previous map extension and that one was not made using Map Tiler.
The tool I'm talking about actually is quite optimized. Given a PGN input file it creates a big bunch of tiny JPEG files but only PNGs on those areas where some pixels are transparent. I did a little test with a 4k x 4k PGN file and the tool cuts it in 213 JPEGs and 54 PNGs spread on 36 folders. Not bad.

Another point would be to redo the previous map extension using Map Tiler. That would be great in terms of optimization but if it's not registered to the very same lat/long coordinates the route map would become completely displaced on top of the background map.

Google might not but Ross Beyer and his group at NASA Ames, who originally worked with Google to make Google Mars, is willing! See his post here!!!

(Meantime, Tesheiner has contacted me about a shorter-term solution that should work for Santa Maria, based on the use of MapTiler that you all figured out in this discussion, hopefully we'll sort that tomorrow)

Lots of activity in this thread. I guess my map is finally going to be supplanted with something better!

Great news! If anyone should do it, some folks at NASA!

OK, we are almost there, but not quite. I downloaded MapTiler and ran Eduardo's selected of the Santa Maria image through it. In a matter of two minutes MapTiler generated 1000 tiles and associated KMZ files with a single KML file in the top directory. When I opened the KML file in Google Mars, bingo! It worked just as Eduardo said. The numbers for the corner coordinates aligned the tile nearly perfectly with the existing map (it's offset to the northish by about 3 meters, give or take a meter). So far so good. That is a solution that will work for people -- as long as they are willing to download 20 MB worth of data per 8000-pixel-square image tile.

But Eduardo suggested that I save everybody a lot of hard drive space and hassle by hosting the image tiles at the Amazon S3 server (where the Planetary Society serves large files) so that people don't have to download the entire enormous map. Using MapTiler I specified the location of the S3 server (http://planetary.s3.amazonaws.com), and it regenerated the tiles, KMZ files, and KML files. The top-level KML file looks like it points to the right place:However, when I try to open the local copy of the KML file in Google Mars, it does not *quite* work. (Please try it for yourself: here's the KML file.) It's really puzzling, because it's not a matter of a broken link. It is clearly finding the georeferencing information within the KMZ files to correctly locate the corners of all the image tiles, but the images are not showing up:
Click to view attachment
What is really weird is that if I point my browser to one of the KMZ files, it finds the image data just fine. For instance, try http://planetary.s3.amazonaws.com/santamariatest/0/0/0.kmz (the very same link listed in the code block above, which is the lowest-res tile, the "apex" of the image pyramid) for yourself, and you should see this:
Click to view attachment
It works! So I don't understand why opening the KML file does not work.

Here is what I did with MapTiler. Help me figure out where I am going wrong and why the image tiles aren't showing up.
Tile Profile: Select Google Earth (KML SuperOverlay)
Source Data Files: Locate local file. I started with a PNG version of the image Eduardo selected, the tile that overlaps Santa Maria; I set the black pixels transparent using Photoshop. Click on its name and "Georeference" button. Put in 4 coordinates listed by Eduardo in green text in his post above, in 'north south east west' order as software specifies.
Spatial reference system: I selected WGS84. I wonder if we should be specifying some Mars-specific projection here -- but WGS84 seemed to work OK for the locally stored version of the map.
Tile Details: Accepted defaults (min zoom 0, max zoom 5; Hybrid JPEG + PNG)
Destination: Result directory: I specified the name of a local directory (the path to the image plus the folder name "santamariatest", so in the end it was "C:\Documents and Settings\Emily\My Documents\Google Mars\james canvin's images\santamariatest"); Destination URL: "http://planetary.s3.amazonaws.com"
Viewers: Google Earth (KML SuperOverlay)
Viewer Details: I specified a map Title (this is just metadata)

It's working perfectly for me.
It takes some time to download the first time but all LODs are working.
Click to view attachment

It worked for me, too, but took awhile to download, though not a problem with patience. I would love to work on it too, but unfortunately, do not have the time now. Let's hope NASA gets on it posthaste!

Emily, maybe you need to refresh the KML file within GoogleEarth, as it might have a cached version of the KML file that's still linking to your own computer for the images. Just a guess, not really sure.

Holy cow, suddenly, this morning, it worked! I think we have a solution that will work out for us!

My question now is: should we regenerate the first map extension from scratch? It would require someone (*cough* Tesheiner) to relocate the corners of the beginning image tiles. I would also think it would be better to regenerate the base images so that they are non-overlapping (which probably needs James' help). And if we are going to do that, and are doing all the map segments at once, I think we should try to match the levels among the different base images to make them appear seamless. This last step I can do very easily (by determining what levels adjustment is needed using the browse versions of the HiRISE images, then using a Photoshop macro to run it on the folder full of 8192 by 8192 chunks).

--Emily

Better maybe, but we simply would not be able to follow Opportunity's trek on Google Mars without your help.
For an "end user" such map overlay might look just line another picture but those who have worked (or tried to work) with multi-resolution images on GE should understand how much time it takes to get such map working without the help of an automated software. Thanks a LOT for the big effort you put on that map!


I like this proposal.
Just a remark about the non-overlapping tiles. Better test it first with e.g. two or three tiles, just in case the gap between them is visible on GE. If that happens, some overlap should be kept.

Good point, we should test. I do think think that if you set corners to be slightly *inside* each other -- adjusting this at like the 6th or 7th decimal place -- then it should work at all levels of detail.

This is reminding me of something that was deeply maddening to me when I was first doing planetary GIS work in grad school, going between ISIS and ArcView. There were different conventions for what the corner of a raster image actually meant. For one piece of software, the convention was that the specified corner coordinates were at the extreme upper left, upper right, lower left, and lower right corners of the image. For the other piece of software, the convention was that the corner coordinates were at the centers of the corner pixels in the image. I do not remember for sure which was which, but I think it was ArcView that called the corner coordinates the centers of the corner pixels. This would make sense because it would prevent the possibilities of image-to-image gaps with extreme zoom that you raised.

Let's test it but IMO I think the corners should be calculated in a way that the whole north border of the new map matches with the base map around Victoria. For the example with Santa Maria I calculated the coordinates using Mars radius as I found in a web page with Google; now we should do it slightly different.


For our case, I think it doesn't matter the convention IF the definition is kept the same from tile to tile.

My point is that if the convention is for it to be the centers of the corner pixels, then if you specify the same lat/lon coordinate for the corners of two adjacent tiles, then the two tiles will *always* overlap by one pixel no matter what zoom level you use.

Well, Tesheiner, you were right. I ran MapTiler on the other Santa Maria segment, using the same east and west coordinates as for the first one, using the first one's north coordinate for the new one's south edge, and doing simple arithmetic to arrive at the north coordinate. Obviously this is not the exact correct location for the "Santa Maria North" tile, because it is really 10% overlapping, but I wanted to test to see if it would really line up adjacent to the first tile.

It did not. There is a gap -- a rather big one, actually, more than 11 meters. So it looks like we will have to stick with the overlapping images.

Click to view attachment

Strange...
I was expecting a gap of maybe a pixel or two but 11 meters is a lot!

I just did a different test to compare with your results. Opening the map with Santa Maria, I manually activated the option which enables opening the map folders to see the whole map structure. Then I deactivated the picture for LOD 0 (the top one) and all other LODs except 1. LOD 1 is composed of four individual pictures and there's no gap between them.
Click to view attachment Click to view attachment

Then I checked the properties of those four images and the coordinates match each other exactly.

I'm puzzled.

Am I missing something here? I downloaded those HiRISE images ages ago when they were first made available to us, and they go right up to the rim of Endeavour. And they are still available, are they not? Why are we adding more? Or did the old files evaporate?

Phil

(EDIT - oh yeah - I guess I did miss something. Sorry Emily!)

Are you talking about the reduced-resolution version? As I mentioned in the first post, SFJCody's first product was a reduced-resolution version that did go all the way to Endeavour. We're now working on full-res products.

I see what's happening here. The borders' coordinates I posted before for the tile including SM were calculated using two different values (polar and equatorial) for the mars radius. But after generating the map overlays giving those coordinates as "georeference" I see that the tool changes the south latitude from the given -2.194771186 to -2.19456651200000; check it on the top level KML file. Perhaps that's a consequence of using WGS84.
Then, if you generate the second map layer north of the first one calculating the north/south coordinates based on the ones I provided on that post and not the ones calculated by the tool, the gap appears. But if you generate the layer based on the coordinates from the KML file, there's no gap.

So, perhaps we won't need overlapping tiles but will have to have a close look on how to properly calculate the tiles coordinates.

The northern border of the image of Santa Maria is at latitude 2° 9'36.06"S, according to GE. How do you turn that into the numerical convention you used above, such as the north -2.160015615 for 2° 9'36.06"S , as I'm attempting to fine-tune where the image is placed to match up more precisely with SJFCody's image.

Thanks for those numbers by the way! They are still very, very close!

You can select the way GE displays lat/long coordinates and among the options are: degrees with decimals or DDMMSS.
Or you can manually convert the minutes and seconds to the proper fraction value.

Wait a minute before matching the picture to the current extension map. If we finally redo SFJCody's map I think the new map may have slight differences in relation to the current one which would affect the placement of the tiles covering Santa Maria.

Back to this topic, I was playing working today on the task of finding the coordinates of the image tiles, and here's an Excel sheet with all of them and a little bit more.
Click to view attachment
The coordinates of the complete picture ESP_011765_1780 were calculated trying to register the image to the HiRISE picture embedded on GE. They don't match 100% but the difference is really small in the area I was concerned which is the point Opportunity's path moves from one image to the other one. From those coordinates to the ones corresponding to the individual non-overlapping 8k x 8k tiles the process was quite simple and straight forward.

I tested these data generating and processing the four tiles needed to cover the whole path and I'm quite happy with the result. GE zooms smoothly through all LODs.
Click to view attachment Click to view attachment

Wow. Great work, Eduardo.

The next step that needs to be done is for James to regenerate the tiles in PNG format, but I know he can't get to this for a few days at least. Then I have to do levels, run them through MapTiler, and upload, which will take some time and I won't be able to start for two weeks -- I am covering DPS this week, and will be on vacation without access to my main computer for the week after that. So there may be a little break here. But I think we'll get to the new map before Opportunity gets to the edge of the current one

Just to have this posted in one place in the thread, the three images for the South of Victoria, Santa Maria, and Endeavour segments are, respectively:
http://hirise.lpl.arizona.edu/ESP_011765_1780
http://hirise.lpl.arizona.edu/PSP_005423_1780
http://hirise.lpl.arizona.edu/PSP_010341_1775
(correct me if I'm wrong!)

--Emily

I may (or may not) have a chance to set my scripts running to make the image tiles for Emily tonight. Has the decision been made if we want to stick with the current overlap, have no overlap or a smaller one? Any of these just as easy for me.

All tests I did with non-overlapping tiles resulted on no gap at all. So I would say 8192 x 8192 tiles without overlap and starting at the top left corner.

Sorry for the long break -- DPS happened, then some necessary travel. In the meantime James got me the two tiles of PSP_005423_1780_RED that we've been working on that cover Santa Maria in PNG format. Unfortunately, MapTiler is not handling these correctly, but I have a suspicion that it's because my old computer is running out of memory. I should have a brand-new computer in two weeks with more than enough memory to handle this problem, and then I'll revisit it. I *think* Opportunity will not get to the edge of the current map in two weeks...it would be awesome if I were wrong about that, though!