|
An Examination Of NASA's
Color Methods
Getting the proper
calibration data is tricky, but here is how to do it
BACK
I decided to
go to a later Sol and pick out some calibration images that were being directly
lit by raw sunlight. I also chose images that matched closely the times
that other images of the ground were taken. Then, I should be able to
generate proper calibration values by balancing the white and gray reference
levels at the outset.
In theory, this
will yield the proper values to show the ground color, and that will be
that. Then, we should be able to get from there to the sky color, once we
have known colors and illumination levels. Is this a good method?
Here we shall see.
| This calibration image is
from Opportunity, Sol 027. This is "rule of thumb"
corrected and I have labeled the image with the RGB levels and the filters
used to create the image.
It looks pretty good except for the slightly
purple cast to the blue reference. I suspect that the L3 filter is
leaking some IR and that is causing the problem. It may be that the
filter has changed somewhat during its 7 months in space, perhaps due to
exposure to vacuum or temperatures that it was not meant for.
Or, there could be some other
cause. I will go into that later on.
The original images are here
at the NASA web site. |
|
 |
| For comparison, I
assembled this uncorrected, full color version that uses the red, green,
and blue data at 100% intensity across the board. It really looks a
bit odd and washed out.
I noticed the pale grayish cast to
the soil right away, and the fact that even though the colors are less
"fruity", the image is flat or lacks something.
Then I realized that it was the
anodized aluminum base plate. I have the exact same materials in my
lab here and I know what color they should be. Gold anodized
aluminum has a characteristic and very recognizable look to it. This
was washed out and did not look right at all.
Therefore, the soil also must be
wrong in color. The reality is somewhere between the last image and
this one. |
|
 |
| Having a piece of gold
anodized aluminum on hand, as well as the same blue heat shrink tubing and
other materials that are sometimes seen in the calibration images, I could
compare them directly to the image results and see how they should look.
Also, since the image has known
intensity white and gray circles, I could perform a standard color
densitometry test and determine what the starting values should be to
arrive at the proper "pure white" figures. This image is
the result of a few simple calculations and comparisons to known hardware
that I own.
And sure enough, this is a very
realistic rendition and the color has body and matches the expected white
and anodized colors. So except for the violet cast to the blues, it
is a good image. |
|
 |
| Now let's assemble a
ground image that uses these derived calibration values. If things
go as we expect, then the ground picture should be accurate and the colors
and intensities should show the soil exactly as it is behind the
calibration fixture.
But instead, we see a horrible
mixture of blue-green and pale rock. What went wrong?
We used the exact same filters, and
the images were taken only minutes from the calibration images. The
sky surely did not cloud over, and the lighting conditions should be
exactly the same. But something is radically different. What
could it be? Original Sol 027
images are located here
at the NASA website. |
|
 |
| If we step back and try
our old rule of thumb method, we can then get some sort of idea where
things went wrong. This image is assembled using that method, and it
looks better, although it looks nothing like the ground in the background
of the calibration fixture image.
While this image is more realistic
looking, the problem is simple- it has no bright reference object in the
image field, and therefore, the camera calibrated itself now to the much
dimmer ground, and the colors that make the image are at very different
percentages from those colors of the reference fixture.
This means that 100:80:60 is wrong
for dark fields, and when you have no white reference in the field, the
camera will set each according to the overall scene but not based on a
bright white reference object.
This means that our carefully
calibrated figures will no longer work, even though everything else is
apparently the same. So how can we know exactly what we should see? |
|
 |
Getting the true calibration from known references
To summarize the problem, in the
reference image, we know exactly how things should look. So whatever the
camera settings are, we can figure out how to compensate and get the image to
look as it should. But when we do not have that calibration image,
and the objects in the picture are of very different color and brightness, then
we have no references and we don't know how to compensate.
The fact is, the camera is choosing
its own settings and we have no idea what they should be. So using the
rule of thumb gets us close, but still does not tell us what the picture truly
should look like. We have to rely on "common sense" or opinion,
and that is a treacherous means of choosing things.
We need either relative intensity
data, or another reference. We do have one, if we choose to use it.
We just have to make the soil look like the soil in the background of the
calibration fixture. I chose to take the soil in the calibration image,
read various elements in it, and use those numbers to calibrate the soil and
rock images.
I chose the nearly perfect calibration
image above and performed an average function on it to eliminate wide variations
in chroma and composition. Then I examined the RGB codes for selected
areas of pixels to determine the true content of each, as they should appear in
a properly tuned image. A patch of soil read about 74:33:19 for RGB,
a bright rock read 135:79:54, and a "blueberry" read 66:47:37.
If I accept these values for the rock and soil image, it should be properly lit and
calibrated. That means reading them in each raw frame, developing a light
curve, and finding the proper brightness ratios for each. This should lead
to the proper illumination and color balance in the final image.
To do this, a proper understanding of
the brightness and contrast functions must be had. Brightness is like a
mathematical "offset" value, shifting all the data up or down by a
specific amount. Contrast, on the other hand, is like a scaling value,
literally stretching or shrinking the range of values. So brightness is
like addition, and contrast is like multiplication. Once that is made
clear, the rest is just mathematics.
Since the mid-scale value is 127 (out
of a possible 0 to 255 for any pixel value for any given color), the strategy is
to brighten a frame until its center pixel value is 127. Then the contrast
must be adjusted until the values read off the soil and stones and features
"stretch" to their proper locations.
| Now that we know how to
make a true correction, let's apply that knowledge to the soil and rock
image and see where it takes us.
This image is made to match as
closely as possible the color values that are shown with the calibration
reference and are the proper brightness and chroma if the calibration
reference is in the image.
But our organic eyes adjust
automatically to varying lighting conditions, and just like going outside
after being in a house for a while, the true brightness levels are
incredibly different from what we expect.
As a result, we tend to adjust the
images until we can see the most detail in a manner that is comfortable to
us. And that is one reason why the images look very different-
people make them look best.
This very clearly explains why the
ground images from the previous pages are so dark- when your eyes adjust
to the brightness of the sky, they lose the ability to see the details of
dark objects well. Based on NASA's own calibration reference, this
is the true color of the ground and rocks in this picture.
This is backed up by the incredibly
dark interior of Endurance Crater in visible light, as well as the
calibration image shown above.
|
|
Compared to the sky and the color calibration
reference, the surface of Meridiani Planum is pretty dark. This is
very close to the way it would look to human eyes, if a white object was
placed in the field of view. Otherwise, our irises would expand and
make the scene much lighter.
|
| Taking this data into
account, we can now see the actual sky color. Due to the fact that
it is brightly lit and that the ground colors are now known, we can set
the actual ratios of colors- the true mix is 100:80:60 or very close to it
for pictures that include sky.
This is because the greater
brightness of the sky sets the upper limit for the brightness in all
frames of color data.
By adjusting the contrast and
brightness levels to match the average rock and soil densitometry, this
process has produced the truest image that is possible using the L2, L5,
and L6 filter data from Opportunity on Sol 001.
Since these values are attuned to the calibration from NASA's own color
references, it is therefore the true and natural color of Mars' sky as it
would appear to a human observer on the surface of the planet.
But take note of the fact that this image still has a bit too much red in
the ground, due to the use of L2, which is (as stated before) infrared. |
|
 |
Is there a leak?
| I used the
latest Spirit calibration images from Sol 159 so I could see how dusty the
rovers and their solar panels are getting. I assembled the typical
L3-L5-L6 image, using the 100:80:60 calibration for bright objects and the
results showed that pesky violet look to the blue calibration chip.
The brightness and contrast
on these images was set to +65% and the saturation was fine without
adjustments.
The dust is getting thick
and we can expect degraded performance from the solar panels due to this
accumulation. It is slowly blocking out more and more sunlight and
making the panels unable to collect the full amount of power they are
capable of.
Just out of curiosity, I
decided to test the IR leakage idea and I used the L4 filter (601 nm,
really an orange instead of red). See the next image. |
|
 |
| Here is the resulting
image for L4-L5-L6. This is a very convincing assembly of the three
and it produces the most realistic looking image that I have yet seen.
So although the orange is not a very
good red, the images it makes are very clean and have an excellent purity
and color balance. And, it works for us because the "red"
pigment in the human eye is most sensitive to light very close to 601 nm
anyway.
There is no visible degradation of
the red, yellow, or green in this image, and on top of that, the blue is
perfect. Dusty, sure, but the chroma is exactly as we would expect.
Once again, I used the RGB ratio of
100:80:60 for this brightly lit white reference object. Other darker
objects will have different white balance levels and will require
something more like the rule of thumb, which is 100:60:30 for RGB.
The original images are here
at the NASA web site. |
|
 |
Summary and recommendations
Pretty clearly, for whatever
reason, we can see that significant amounts of infrared light are leaking
through the L3 filter, and the L2 filter is totally useless for anything that
has blue in it. It overrides blue pigmented objects and ends up making
things look redder. This can totally skew the true color images and lead
you to believe that you are seeing something that is not there. The sky is
one case in point. If any IR skyglow is present, it will drown out some
portion of the blue and make the sky look redder than it should. For both
the L2 and L3 filter, this is a real problem.
True color images typically should use
L4, L5, and L6 for the best results. Images that are being assembled to
show the fossils in sharper contrast should use the L7 instead of the L6 because
it will enhance the blue portion of the spectrum. I sometimes use L7
instead of L6 because of two factors- it makes a far more convincing blue, and
it appears from recent testing that our visual systems are more attuned to that
shorter wavelength due to a little recognized mechanism shown
at this link. Oddly, L7 is close to the peak response of our
eye's blue pigment.
This could all have been avoided
through two simple means. One, don't throw away your relative intensity
data! I am sure that more than a few NASA scientists are cursing the
choice to do that. It was a stupid move, and one that hindsight makes very
clear as one of the true blunders in the project. This one thing has
caused (in my opinion) more trouble for Martian geology alone than any other
three problems put together.
Two, be certain that a color and
gray scale reference is in every single picture. If I could change one
thing on this mission that weighs nothing at all, it would be the inclusion of
many color dots and white/gray dots on each and every visible surface of these
rovers. Then, there would be no controversy. The colors would
be known absolutely. A simple light intensity meter and the sight of the
color dots would end any arguments over the true colors in a flash.
it would even be prudent to make a
small rod that drops into view of the camera at all times, at the edge of every
image, that has color and gray scale dots on it and would be photographed every
single time an image is made. You cannot wreck that sort of data- it is
either right or wrong.
As it stands, some news organizations
have even accused NASA of trying to hide the true colors of sky and ground on
Mars, or to make it look more "alien" to the general public.
Whether this is through intent or through error, I cannot say. What I can
say is that this is one more black eye that they could have avoided.
UPDATE: NASA has indeed
colored the skies intentionally. Check this
page for the information.
BACK
DONE
|