I've been a heavy user of Midnight Mars Browser for several years and I'm incredibly grateful for the work that Michael has done.

Now I ask myself, should we create again an alternative way to look and search for images coming from Mars? In the last several months I developed a way to ingest InSight images in order to have a small database and a telegram bot that tells me when a new image is uploaded to the InSight web site and I can try to do the same with the MSL imagery.

But, I have several questions: What is the better way to show the data in the web page? Which is the best way to order the images?

I'd love to hear some suggestions for that.

Greetings.

I would suggest something that resembles the structure of MMB. I have found it very useful for years. Having access via a list of sols, and access to the previous and next sols, is great. And ordering images by time within a sol is very useful.

Phil

Not sure how far you want to take this, but Joe Knapp's MSL image browser also had some very useful features, of the many features the one I used the most was the pointing data which Joe got from the mission NAIF service.

Good luck if you choose to create a browser

In addition to ordering by sols, ideally a page would also somehow allow you to know which images you haven't looked at yet, ie which are newly downlinked images. This can be as simple as including ordinary text links to the full size images (presumably on the jpl server) which the viewer's web browser will change the colour of once visited (as Joe's site does). Ie, you can let the browser's own history handle it.

The option to order by downlink time is very nice when very old images are finally downlinked, otherwise they may be many sols back and you might miss them.

Otherwise displaying the LMST and UTC of image capture, and UTC of downlink are important. Also displaying the full-res image size in pixels is nice, so you can tell if the image is just a subsampled thumb or not.

(And it goes without saying that the page should only load thumbs, not full-res that are rescaled by the browser, until you open the full-sized image! )

Thanks a lot for considering this, and best of luck!

Would it be very complicated to implement a function that would debayer the Mastcam images when we download them?

That would be a great feature to have. It would be easy to do a quck-and-dirty job (showing some green artifacts) if you could run something like gmic (runs on linux). But you would need server space to host the deBayered images.

Thanks for all the suggestions!,

I'm thinking on creating two versions: a simple similar to Midnight Browser that was perfect for looking at the last downlinked imagery and a more complex one that allows searching (by Sol, by time of the day...).
I'm already doing some automatic debayering for the downlinked images. The program checks if the image is bayered and runs a debayering algorithm (there is still room for improvement in that point Click to view attachment). I know it will take some space on the server, but it is worth.

Thumbnails are quite useful for load times, right now I have this functionality implemented in the InSight image download code I'm using.

Of course, the viewer should allow to see the downlink time and when the image was taken in Mars and Earth time and if the image was taken day or night (might be useful for MAHLI night images and astronomical images). MAHLI images also will show the scale per pixel and the motor count.

The only problem I have is the pointing data. I have to study a bit because I never used the SPICE kernels for calcultations, but I hope I can achieve that.

Thanks for all the suggestions!

To reduce the need for server space, I was thinking that the debayering could be done on-the-fly during the downloading process, so that only the undebayered images would have to be archived.

It would still be nice to have at least deBayered thumbs stored on the server to ease browsing the images.

Thanks again for this, paraisosdelsistemasolar, and I'm sure you'd have a bunch of willing beta testers here if needed...

At this point I don't have any limitations regarding file space, so debayered files won't be any problem.
If more space is needed than my host allows, I can host it by myself at home. As soon as I have a beta version I'll show you all for feedback.

Greetings!

I was hoping Midnight Planets / MMB could be branched out to live into the future, so much work was perfected by Mike it would be great to continue that.
Per a recent conversation in the MER thread, he sounds too busy to continue with it, but didn't mention what he plans to do with his code base, so I was hoping for an opensource dump, but hard to say if that will happen.. In any event, however the plans manifest going forward, i'd be happy help out anyway i can.

Sorry for the delay,

We have had some rough weeks at home, but I'm now working again on the code and I hope to have a first version in a few days (I hope so).

The code and the database will be open source as soon as I release a first version. The scraping is being done in Python, the web will be in PHP and the database will be MySQL.

Looking forward to this! Thanks for putting time into it 'Para-doodle'

Hi to everybody,

Sorry again for the delay, but I'm a simple geologist and not a good programmer. I'm about to finish the robot that reads the MSL image archive. It's not only for showing the latest images, I plan, also, to be useful for searching images. That said, some of the avaiable fields will be:

- MTC, LTST and LMST
- Leds on/off
- Sun height.
- open/closed LID in MAHLI images
- Focus merge products
- Distance/resolution to object (MAHLI)

I hope to learn about spice kernels for driving/position purposes, but that will be in a later phase.

Hey all,

I wrote a basic Navcam viewer that reads the official json files and shows Curiosity’s panorama’s in a web browser.
It currently comes without interface and is pretty featureless in comparison to the great MidnightPlanets but it performs really well.
Check it here: https://captainvideo.nl/rob/mslview.html

To view panorama’s from a specific sol just add ?sol=2633 (or any other sol number) to the web adress.
For example: https://captainvideo.nl/rob/mslview.html?sol=2633

Hope you enjoy!
Rob

Wow Rob, I really liked your panorama viewer!

I'm still finishing the image viewer, but I'm pretty confident at least all the big data ingestion will be done and finished by the end of the week.

Yes, many thanks for this, its very nice! and fast!! just a few hundred lines of code and already its the best available way to get a quick pano view of the latest MSL adventures (not to downplay, i recognize this certainly represents a heck of lot of work on your part, just saying all these years later NASA has yet to match what you've accomplished!)

It looks like it loads all image data straight from mars.jpl.nasa.gov, rather than self-hosting like MP did. I can just plug in sols at random and many happen to have a large sweep of navcams so provide a pretty good view, so its already quite useful if you know what you want to see. For now, sol2595 looks to be hardcoded in the javascript, and i'd suspect a sol browser having navcam counts per sol is likely a big effort in itself. makes me wish i had chosen a career in software dev so i could get in the game not just cheer from sidelines, great work!

Rob!

This.

Is.

Awesome!!!

Thanks for sharing this. If you have plans to develop further I would love to make some suggestions but it is already an instant bookmark for me.

Congratulations Rob, well done! This is a really nice way to "transport" us there, and it realy is fast!
Thank you for sharing it.

Fernando

So I went sol-searching for the intent of coding self-improvement...
And it seems as if the NASA index Rob is using is all jacked up: https://mars.jpl.nasa.gov/msl-raw-images/im...e_manifest.json
The data seems fine until you get to sol "191" which for some reason has "sol192" inside, then it just snowballs from there, by the time its gets to sol '2382' it has 'sol2639' inside! or maybe i just dont understand what NASA is doing..? anyways, i didn't look into it any further yet and so remain thoroughly flummoxed and perplexed! ..of course i had to check again, it goes nuts at 'sol' 200 implying no MSL downlink for 15 days -and yes thats what i see in Midnight Planets "Memory anomaly", so Rob probably didnt use those numbers for anything and just looks at the catalog_url, so NASA seems to have incorrectly labeled their index 'sols' but its really just a plain old enumerated sequence unrelated to actual sols.. its nice to know there is this wealth of data indexed for each recorded sol

Paraisos, if I may formulate another request for your MSL image browser: it would be very useful if we could select a range of images and download them all simultaneously, instead of having to do it one by one.

Hey everyone, you are welcome to visit the latest version of the Mars rover navcam viewer.
It is available at marslife.org and should work on every device that supports WebGL.

New features are:
- display of the rover traverse path.
- clickable sol/site positions for easy navigation.
- menu option to select and view image products from pds (when available).
- site-updates occur automatically when new mission data becomes available.
- initial support for viewing the MER missions.

Usage:
- Tab key (or upper right circle) for menu/sol list.
- Left/Right arrow keys (or the lower right buttons) to view the previous or next site.
- L key to show/hide site locations.
- P key to toggle path display.
- S key to toggle the simulated sky on/off.
- C key to toggle image brightness corrections.

The interface needs work and the path accuracy is flawed (especially with MER) but the
clickable sol/site buttons make it real easy to get acquainted to the Martian territory.
Feel free to post comments and suggestions. Have fun wandering on Mars!

Rob

Rob,
This is excellent! I haven't yet tried the MER paths but why do you say MSL's path is flawed?
Congratulations.
Fernando

Thanks Fernando! I meant to say that the paths are not very precise and do not cover every individual rover movement.
And while the path matches nicely in most panorama’s there are certain areas earlier in the mission where the path is way off.

Very nice work! your project is really coming along great, Congrats! i like the method of subtlety toggling the ui elements and options and how well the correction algorithm is working.

Since i'm sure you are not already nearly busy enough with all this haha, I will pile on extra features, the first being a separately toggleable pan/zoom overview map overlay with path and current location dot, and yes of course it has to be clickable so you can go to the sol nearest to the click location.

Fantastic work Rob, this is a super useful resource.

Thank you all!

@atomoid, There’s no algorithm... the brightness adjustments are done manually.

oh boy somebody's got their work cut out for them!! now that explains why if i go back to old sols that "correction algorithm" doesn't seem to function!

Hey all, two extra features are added to Marslife.org!

- The update script will now automatically generate an anaglyph set of images for each new panorama.
Set the 'image product' menu option to "STEREO" to view the anaglyph version of the panorama.

- The script will also post a tweet at the marslife.org twitter account when a new panorama is added.
You can follow this twitter account to keep track of Curiosity’s latest movements on Mars.

It’s all highly experimental and bound to break at some point but I hope you will enjoy anyway!

Rob

Click to view attachment

(edited to add twitter link)

Can't seem to find your twitter @handle

Apologies, I'm fairly new to Twitter. This is the link https://twitter.com/MarslifeOrg.

Thank you！

This is a great way to view pictures from MSL & MER. I'm all for modern HTML5 tools.

But since the official Raw images webpage is crap, and since Midnight Planets don't work anymore, is it possible to have the pictures displayed also as a grid, just a simple HTML webpage, with a simple sol by sol classification ? Just like Midnight Planets did ? A simple access to the jpegs, without that "responsive design" thing, and without that JS messing around. A good old plain HTML page.

Agreed. Something with the look and feel of Midnight Planets, the same functionality, and capable of being extended to Perseverance...! However, I am getting used to working with the Raw image pages for InSight and Curiosity and they are actually quite good, better than I thought at first. You can select any sol and any instrument if you need to.

Phil

Like Phil I'm slowly getting used to the new page, but I'm far from happy with it. I just wish they left as it was until the end of mission, or at least had a revert option on it so you could use the old or new interfaces. Even the old interface wasn't perfect, but back then we had MP and Joe's page to use if we wanted the various whistles and bells they provided. Sadly no more, so we're stuck with the new interface which doesn't quite match the InSight raw image inferface for whatever reason. So let's see what we get for 2020's 23 cameras next year 😁

From my point of view, I don't. I tried. I really tried. But this page is heavy. The pictures are not that easy to load, every time I right click on it, it slow my navigator. I used the Web Inspector to have a list of the resources but it's not complete. But, I agree with one point : the filters are quite useful. Although, I would have prefer a simple webpage, with miniatures and have the full version right after a click, not that pop-up window. This is time consuming.

Maybe it would be possible to have a similar gallery like MP, with some filters in it (instruments selection, the site number (location), the type of datas (full frame, downsampled…), sol by sol list… I don't know. A way to simplify the access to the pictures, with the less clicks possible.

@Paul : You can be sure that the Perseverance raw images webpage would be the exact same as Curiosity and Phoenix.

Hi everyone!
Here is a prototype for a Curiosity raw images viewer. It is not intended for smart phones.

Not so quick but still somewhat dirty: the code does not have much robustness. It consists of a single HTML file and does use CSS and JavaScript (how else?)
I'm releasing it in order to get some feedback: Am I going in the right direction? Does it have your favourite features? What is wrong and what else would you like to have it do? Just post here or PM me.
The crucial thing about it is the JSON file(s). If NASA changes those then everything will need reviewing.

About installing it:
Because I do not have a web site in which to host the file I am attaching it here. Simply download it to your favourite location. Click (or double click) on it to open it on your default browser.
However, this creates a problem as the page is local for the browsers and makes cross-domain requests (the famous CORS problem). The solution is to install a browser add-on. I am using Firefox so I installed CORS Everywhere . There is one for Chrome as well Allow CORS. This is quite an easy process.

About using it:
Open the file. It will automatically be positioned on the most recent sol and display all its images, in small format. The images are displayed by sol. Any sol can be selected by typing its number on the appropriate entry field (top right) or using the scroller. Press the "GO !" button to load the images.The images can be filtered by instrument. A row of buttons, one for each instrument, is dynamically built when the images are loaded. Press a button to see the corresponding images. Clicking on an image opens it in large format on an independent (resizeable) window which also shows the available information for image. Any number of images can be opened (depending on your computer's resources).
Note: the code is fast but some sols have hundreds, even thousands of images. This takes time, and I do have a fast connection!

Future:

1. Simultaneously selecting images from several instruments;
2. Selecting images by type (full, subframe, etc);
3. Downloading images;
4. Should image loading be automatic, or after filters are applied? Should it be by small batches (50 images?)
5. ... ?

Enjoy and let me have your feedback. Thanks

Fernando
Click to view attachment

Nice, Fernando! I'm not sure what you meant by CORS - my browsers showed the images fine without any plugins.

One suggestion, to help those without a fast connection: load only the thumbnail images on the pages showing multiple images from one sol, and then load the full image when you click on the thumbnail. Chemcam images should be courtesy NASA/JPL-Caltech/LANL.

Moderators: Is there any chance this thread could be moved to MSL? It only talks about viewing MSL images and doesn't relate to "Image Processing Techniques". It's really a sequel to the old thread Curiosity Image Retrieval Tools, scripts and software.

Thank you so much fredk ! I like the suggestion, although I'll have to think about what to do when there are no thumbnails (not every image has one, I believe). Thanks also for the correction to ChemCam credits.
Fernando

That's awesome!

So - I'd actually be cautious about using the thumbnails.... all thumbnails are full-frame, but all dust devil movies or upper tier images or other sub-framed pictures are not full frame, so the thumbnail might be missleading? I've got a couple of ideas / requests

1) Have a thumbnail on/off button ( to stop them showing in the list at all to avoid duplication)
2) Have a text list at the bottom of any page with a listing of links direct to the images so people can do a bulk download of data for any given sol.
3) What some people are beginning to figure out is how useful the Sequence name is.... ( in a file like NRB_655824982EDR_S0822188NCAM00595M_.JPG the SeqID is the NCAM00595 part ) - being able to filter by SeqID for a given Sol would also be really good as that would let someone ( combined with 2, above) download, say, all the images in a single dust devil movie or MastCam mosaic on a given sol.

One bug to report - it's not a constant sol number - but sols before about 2785-2775 or so, it will return an image count, but not return any actual images.

If you don't mind me asking - how long did this take you?

Many thanks Doug, for your comments and encouragement.
I decided to do this after reading Damia's post (a few posts back) but I didn't work full time on it. I don't clock in (after all I'm retired) My estimate is it took more than 10 hours over two days. I know a bit of HTML, CSS and JavaScript ...
And now I have a bug to hunt down. Mnham!
Thanks again for the great feedback
Fernando

It may depend on what you mean by a thumbnail. The downlinked thumbnails are indeed full frame - eg, for the navcam frame https://mars.nasa.gov/msl-raw-images/proj/m...NCAM00595M_.JPG the image https://mars.nasa.gov/msl-raw-images/proj/m...NCAM00595M_.JPG is a full-frame thumb.

On the other hand, I believe all images hosted at mars.nasa.gov/msl-raw-images are paired with thumbs generated on the ground - for that navcam the thumb is https://mars.nasa.gov/msl-raw-images/proj/m...00595M_-thm.jpg. It's these latter thumbs that should be used to speed loading of sites like this.

That IS really nice ! Already handier than the official raw page.

If I may suggest : having a "hard" link when hover a thumbnail, permitting the use of the link to be displayed at the bottom of any webbrowser, and maybe don't display the full frame in a pop-up window, but in the same window (user can control that behavior later, by "click+command/control").

MP also had something really nice and simple, a navigation "tool" like this (example) : "... > Curiosity > Sols > ←2538 / 2540→" on top of a page. And when in a specific image page (displaying the full frame), we could also navigate previous / next picture.

Just express my feeling on it But this is already a nice job !

PS : If you want some place to host it, I can do it. I can also decompose your page in source files (separate the CSS from the HTML and the JS).

Merci beaucoup Damia!

For your encouragement and great suggestions. I'm compiling a list and will work on them as time allows, now that I know I'm heading in the right direction (but I'll be mostly off next week, COVID allowing).
Thank you also for your offer to host the tool. Right now it is evolving, so it is not the right time, but I'll get back to you on that. Isolating the JS and CSS in separate files is good practice, this was just the prototype.

To all:I may need to create several html pages (in addition to the above mentioned JS and CSS files) so I may be distributing a small zip file in the future.
Thank you for the information, nice reviews and ideas. Keep'em coming!

Those thumbnails are not part of the JSON feeds that drive Nogal's tool - they're part of the Mars web team's implementation and thus are probably an order of magnitude more likely to get changed, moved, broken as a result.

Another challenge! By the way, I couldn't reproduce the bug you've mentioned. It worked for all the days you pointed out and a few extra on either end of the interval. Perhaps it is related to the slowness of the loading? I'll make it more obvious in the tool when it is loading. Thanks
Fernando

I've been parsing the MSL json files into html and thought I'd make the pages public in case anyone finds them useful. My pages are static html so there's no selecting cameras, changing sort orders, autoload scrolling, etc. It's inspired more by the linear style of the old curiosityrover.com than MMB, Fernando's page, or www.mars-browser.co.uk/curiosity.

The info and pics are straight from the json files and the mars.nasa.gov/msl-raw-images server with two exceptions. First I calculate the pointing according to mcaplinger's suggestion in this post. I haven't tested this much so let me know if you notice any problems.

The second is that I deBayer the Bayered mastcam frames automatically as they arrive. Due to the chroma subsampling I do you shouldn't take the colours too literally, and there are lots of crosshatch artifacts, so these images are only meant to give a rough sense of the scene.

On my site there's the "latest images" page located here: http://lcdm.ca/msl/. This checks for updates every half hour and adds any new images to the top, chronologically by time taken. After updates I use this by scrolling and paging down until I see image links change to the visited colour, which makes it very easy to keep up and not miss any new images, even if they're from old sols.

There are also individual sol pages, eg: http://lcdm.ca/msl/2924/ (going back to sol 2714). These are always ordered by time taken.

More details on this page.

If anyone has any suggestions hopefully I'll have time to implement them (within the limitations of static html), so let me know.

Viewed on my Mobile phone and impressed Looking forward to seeing it in action on my home PC when my covid travel restriction order is supposed to be lifted in 11 days 😊

Thanks, Paul. Sorry my site's not small-screen friendly. Maybe now that we're moving again on Mars that'll relieve your earthbound status a bit?

Thank you for sharing this. Fast, straight and simple! Definitely useful. Cheers
Fernando