Page Link:

An Examination Of NASA's Color Methods

Getting the proper calibration data is tricky, but here is how to do it

BACK

    I decided to go to a later Sol and pick out some calibration images that were being directly lit by raw sunlight.  I also chose images that matched closely the times that other images of the ground were taken.  Then, I should be able to generate proper calibration values by balancing the white and gray reference levels at the outset.

    In theory, this will yield the proper values to show the ground color, and that will be that.  Then, we should be able to get from there to the sky color, once we have known colors and illumination levels.  Is this a good method?  Here we shall see.

   This calibration image is from Opportunity, Sol 027.  This is "rule of thumb" corrected and I have labeled the image with the RGB levels and the filters used to create the image.

It looks pretty good except for the slightly purple cast to the blue reference.  I suspect that the L3 filter is leaking some IR and that is causing the problem.  It may be that the filter has changed somewhat during its 7 months in space, perhaps due to exposure to vacuum or temperatures that it was not meant for.

   Or, there could be some other cause.  I will go into that later on.

   The original images are here at the NASA web site.

   For comparison, I assembled this uncorrected, full color version that uses the red, green, and blue data at 100% intensity across the board.  It really looks a bit odd and washed out.

   I noticed the pale grayish cast to the soil right away, and the fact that even though the colors are less "fruity", the image is flat or lacks something.

   Then I realized that it was the anodized aluminum base plate.  I have the exact same materials in my lab here and I know what color they should be.  Gold anodized aluminum has a characteristic and very recognizable look to it.  This was washed out and did not look right at all.

   Therefore, the soil also must be wrong in color.  The reality is somewhere between the last image and this one.

   Having a piece of gold anodized aluminum on hand, as well as the same blue heat shrink tubing and other materials that are sometimes seen in the calibration images, I could compare them directly to the image results and see how they should look.

   Also, since the image has known intensity white and gray circles, I could perform a standard color densitometry test and determine what the starting values should be to arrive at the proper "pure white" figures.  This image is the result of a few simple calculations and comparisons to known hardware that I own.

   And sure enough, this is a very realistic rendition and the color has body and matches the expected white and anodized colors.  So except for the violet cast to the blues, it is a good image.

   Now let's assemble a ground image that uses these derived calibration values.  If things go as we expect, then the ground picture should be accurate and the colors and intensities should show the soil exactly as it is behind the calibration fixture.

   But instead, we see a horrible mixture of blue-green and pale rock.  What went wrong?

   We used the exact same filters, and the images were taken only minutes from the calibration images.  The sky surely did not cloud over, and the lighting conditions should be exactly the same.  But something is radically different.  What could it be?

   Original Sol 027 images are located here at the NASA website.

   If we step back and try our old rule of thumb method, we can then get some sort of idea where things went wrong.  This image is assembled using that method, and it looks better, although it looks nothing like the ground in the background of the calibration fixture image.

   While this image is more realistic looking, the problem is simple- it has no bright reference object in the image field, and therefore, the camera calibrated itself now to the much dimmer ground, and the colors that make the image are at very different percentages from those colors of the reference fixture.

   This means that 100:80:60 is wrong for dark fields, and when you have no white reference in the field, the camera will set each according to the overall scene but not based on a bright white reference object.

   This means that our carefully calibrated figures will no longer work, even though everything else is apparently the same.  So how can we know exactly what we should see?

 

Getting the true calibration from known references

    To summarize the problem, in the reference image, we know exactly how things should look.  So whatever the camera settings are, we can figure out how to compensate and get the image to look as it should.  But when we do not have that calibration image, and the objects in the picture are of very different color and brightness, then we have no references and we don't know how to compensate.

    The fact is, the camera is choosing its own settings and we have no idea what they should be.  So using the rule of thumb gets us close, but still does not tell us what the picture truly should look like.  We have to rely on "common sense" or opinion, and that is a treacherous means of choosing things.

    We need either relative intensity data, or another reference.  We do have one, if we choose to use it.  We just have to make the soil look like the soil in the background of the calibration fixture.  I chose to take the soil in the calibration image, read various elements in it, and use those numbers to calibrate the soil and rock images.

   I chose the nearly perfect calibration image above and performed an average function on it to eliminate wide variations in chroma and composition.  Then I examined the RGB codes for selected areas of pixels to determine the true content of each, as they should appear in a properly tuned image.  A patch of soil read about 74:33:19 for RGB, a bright rock read 135:79:54, and a "blueberry" read 66:47:37.  If I accept these values for the rock and soil image, it should be properly lit and calibrated.  That means reading them in each raw frame, developing a light curve, and finding the proper brightness ratios for each.  This should lead to the proper illumination and color balance in the final image.

    To do this, a proper understanding of the brightness and contrast functions must be had.  Brightness is like a mathematical "offset" value, shifting all the data up or down by a specific amount.  Contrast, on the other hand, is like a scaling value, literally stretching or shrinking the range of values.  So brightness is like addition, and contrast is like multiplication.  Once that is made clear, the rest is just mathematics.

    Since the mid-scale value is 127 (out of a possible 0 to 255 for any pixel value for any given color), the strategy is to brighten a frame until its center pixel value is 127.  Then the contrast must be adjusted until the values read off the soil and stones and features "stretch" to their proper locations.

   Now that we know how to make a true correction, let's apply that knowledge to the soil and rock image and see where it takes us.

   This image is made to match as closely as possible the color values that are shown with the calibration reference and are the proper brightness and chroma if the calibration reference is in the image.

   But our organic eyes adjust automatically to varying lighting conditions, and just like going outside after being in a house for a while, the true brightness levels are incredibly different from what we expect.

   As a result, we tend to adjust the images until we can see the most detail in a manner that is comfortable to us.  And that is one reason why the images look very different- people make them look best.

   This very clearly explains why the ground images from the previous pages are so dark- when your eyes adjust to the brightness of the sky, they lose the ability to see the details of dark objects well.  Based on NASA's own calibration reference, this is the true color of the ground and rocks in this picture.

   This is backed up by the incredibly dark interior of Endurance Crater in visible light, as well as the calibration image shown above.

Compared to the sky and the color calibration reference, the surface of Meridiani Planum is pretty dark.  This is very close to the way it would look to human eyes, if a white object was placed in the field of view.  Otherwise, our irises would expand and make the scene much lighter.

   Taking this data into account, we can now see the actual sky color.  Due to the fact that it is brightly lit and that the ground colors are now known, we can set the actual ratios of colors- the true mix is 100:80:60 or very close to it for pictures that include sky.

   This is because the greater brightness of the sky sets the upper limit for the brightness in all frames of color data.

   By adjusting the contrast and brightness levels to match the average rock and soil densitometry, this process has produced the truest image that is possible using the L2, L5, and L6 filter data from Opportunity on Sol 001.

   Since these values are attuned to the calibration from NASA's own color references, it is therefore the true and natural color of Mars' sky as it would appear to a human observer on the surface of the planet.

   But take note of the fact that this image still has a bit too much red in the ground, due to the use of L2, which is (as stated before) infrared.

    Is there a leak?

    I used the latest Spirit calibration images from Sol 159 so I could see how dusty the rovers and their solar panels are getting.  I assembled the typical L3-L5-L6 image, using the 100:80:60 calibration for bright objects and the results showed that pesky violet look to the blue calibration chip.

   The brightness and contrast on these images was set to +65% and the saturation was fine without adjustments.

   The dust is getting thick and we can expect degraded performance from the solar panels due to this accumulation.  It is slowly blocking out more and more sunlight and making the panels unable to collect the full amount of power they are capable of.

   Just out of curiosity, I decided to test the IR leakage idea and I used the L4 filter (601 nm, really an orange instead of red).  See the next image.

   Here is the resulting image for L4-L5-L6.  This is a very convincing assembly of the three and it produces the most realistic looking image that I have yet seen.

   So although the orange is not a very good red, the images it makes are very clean and have an excellent purity and color balance.  And, it works for us because the "red" pigment in the human eye is most sensitive to light very close to 601 nm anyway.

   There is no visible degradation of the red, yellow, or green in this image, and on top of that, the blue is perfect.  Dusty, sure, but the chroma is exactly as we would expect.

   Once again, I used the RGB ratio of 100:80:60 for this brightly lit white reference object.  Other darker objects will have different white balance levels and will require something more like the rule of thumb, which is 100:60:30 for RGB.

   The original images are here at the NASA web site.

    Summary and recommendations

    Pretty clearly, for whatever reason, we can see that significant amounts of infrared light are leaking through the L3 filter, and the L2 filter is totally useless for anything that has blue in it.  It overrides blue pigmented objects and ends up making things look redder.  This can totally skew the true color images and lead you to believe that you are seeing something that is not there.  The sky is one case in point.  If any IR skyglow is present, it will drown out some portion of the blue and make the sky look redder than it should.  For both the L2 and L3 filter, this is a real problem.

    True color images typically should use L4, L5, and L6 for the best results.  Images that are being assembled to show the fossils in sharper contrast should use the L7 instead of the L6 because it will enhance the blue portion of the spectrum.  I sometimes use L7 instead of L6 because of two factors- it makes a far more convincing blue, and it appears from recent testing that our visual systems are more attuned to that shorter wavelength due to a little recognized mechanism shown at this link.  Oddly, L7 is close to the peak response of our eye's blue pigment.

    This could all have been avoided through two simple means.  One, don't throw away your relative intensity data!  I am sure that more than a few NASA scientists are cursing the choice to do that.  It was a stupid move, and one that hindsight makes very clear as one of the true blunders in the project.  This one thing has caused (in my opinion) more trouble for Martian geology alone than any other three problems put together.

    Two, be certain that a color and gray scale reference is in every single picture.  If I could change one thing on this mission that weighs nothing at all, it would be the inclusion of many color dots and white/gray dots on each and every visible surface of these rovers.  Then, there would be no controversy.  The colors would be known absolutely.  A simple light intensity meter and the sight of the color dots would end any arguments over the true colors in a flash.

    it would even be prudent to make a small rod that drops into view of the camera at all times, at the edge of every image, that has color and gray scale dots on it and would be photographed every single time an image is made.  You cannot wreck that sort of data- it is either right or wrong.

    As it stands, some news organizations have even accused NASA of trying to hide the true colors of sky and ground on Mars, or to make it look more "alien" to the general public.  Whether this is through intent or through error, I cannot say.  What I can say is that this is one more black eye that they could have avoided.

UPDATE:  NASA has indeed colored the skies intentionally.  Check this page for the information.

BACK        DONE