Support: 16/Oct/2007, Rachel Gaulton/Tim Malthus, GB06/05

[mggr]

Rachel contacted us with a problem relating to radiometric calibration.

I'm not sure who specifically is the best person to ask about this,
but I am having some problems with the AISA Eagle data recently
delivered for my sites (Clocaenog and Glasfynydd, acquired July 06).
The radiance values differ considerably to those of the CASI and ATM
data acquired at the same time, specifically they are much lower
(about 1/2 to 2/3) in the Eagle data (see attached spectra of a paved
area in Glasfynydd for an example).  Following attempts at atmospheric
correction, this results in significantly lower reflectance than is
expected based on ground spectra.  I  may be missing something obvious
(e.g. different units), or is this down to errors in the radiometric
calibration of the Eagle sensor?

[mggr]

(16/Oct) Replied to Rachel and said Andrew was the most appropriate person to look at this.

(18/Oct) Andrew spoke to Gary, who spoke to us, recommending that we investigate this on a known dataset with all sensors (suggested boresight flight)

[mggr]

18/Oct: told Rachel what was happening

~22/Oct: examined boresight flight and confirmed that we're also seeing a difference between CASI and Eagle spectra - holding onto this until Gary comes down to look on the 25th (then moved to 29th).

29/Oct: Gary looked and we agree there appears to be a problem

[mggr]

Spoke to Andrew, who recommends:

1. Chris from FSF is visiting in early Dec to calibrate some instruments for 2008. While there, get him to do a quick and dirty calibration on the Specim too, so we have a calibration made in the same conditions as the CASI one to compare against (giving us a ground truth). This will establish if there's a problem with Specim's calibration procedure.
2. As soon as Bill is back from holiday (~10th Nov?), ask him to check on how he handles the Specim calibration in azspec, particularly with non 1-1 binnings (all 2007 Eagle flights are 2-1 or higher and the calibration is 1-1).

[mggr]

Other things to pursue:

• compare level 0 vs level 1b data to ensure the calibration is being applied
  • done - the level0 looks a bit like a CCD response curve and the level 1b looks like a ground-spectrum and matches the CASI spectra, so this appears to be working, albeit with an offset/factor
• check the 2007 flights for various binnings to compare against CASI
  • done - no 2007 flights are 1/1, they vary from 2/1 to 8/1 and 8/4
• compare CASI and Eagle spectra in multiple flights to see if there's a common factor, bearing in mind the binning rates
  • oop, only a single flight has CASI and Eagle! (the boresight)
• contact Rachel and let her know what's going on

[mggr]

IPY 2007 flights have CASI/Eagle data - these will be harder to process, but..

Steve says to test a few of the CASI/Eagle matches before contacting Rachel so that we can give her more definite information.

[mggr]

Tested on L3 data (1m) to make sure this discrepancy holds true (it did).

[mggr]

Original L0 and L1b tests

[mggr]

L3 tests

[mggr]

L3 tests (big graphs)

[mggr]

Confirmed the L3 & L1b numbers are comparable (ie. safe to use L1b)

[mggr]

For purposes of 'easy' viewing:

[mggr]

Creating a new ticket to track the problem (#81) so this ticket remains about supporting Rachel rather than tracking the calibration investigation.

[mggr]

#81 shows tentative hints it may be related to spatial binning - Rachel's flight is 195c/2006, which was 2 spectral, 2 spatial. Going to contact Rachel so she knows we're working on this and have some initial results.

[mggr]

Got a reply from Rachel confirming she received the update on progress so far.

[mggr]

Email from Rachel asking about estimates on timeframes for fixes:

re. the Eagle radiometric calibration issues (or software bug), I've
been keeping an eye on your internal tracking of the issue and your
clearly making great progress with it! However, I was just wondering
if there was any chance you would be able to give me some sort of
indication (even very approximate) of when you think it is likely that
you will manage to resolve the issue and over what approximate
time-scale, if at all, I could expect to receive corrected data? I
appreciate you may well not be able to answer this, I just thought it
was worth asking as I am now in the 4th year of my PhD and really need
to be able to produce a final plan of my thesis and re-think sections
if necessary (the Eagle/Hawk data would have comprised the basis for
at least a chapter).

Thanks for any help - best guesses are fine!

Explained the current situation - likely timeframe would be several months, even if the fix was immediately available now.

We've offered to try processing her data via the Caligeo route, if she can help us verify it's workable.

[mggr]

Begun downloads of 195a and 195c eagle/hawk data pre-emptively.

[mggr]

Rachel agreed to try this - selected flightline is 195a, eagle, line 6? ("e195061b"). Contacting AKW to see if he has the previous script / parameters.

Many thanks for your reply,  I appreciate its hard to predict how long
this will take, I was just keen to get some idea.  If you are willing,
I would be very grateful if you could try processing a strip with
Caligeo - the most suitable would be one of the Glasfynydd strips -
e195061b as this has some areas I have field spectra for.  Don't worry
if it takes a few weeks until you can do this - I am still writing up
my lidar work anyway so have plenty to keep me going.

 If this works it may not be necessary to process all of the remaining
data immediately, my research plots cover a relatively small area so a
smaller number of flight lines would be sufficient (for the time being
at least).

[mggr]

Downloads complete. Data stored at ~arsf/arsf_data/in_progress/2006/195A_GB06_05/

[mggr]

Ben sent off a sample file processed via AISA Tools (CaliGeo blew up) - we'll see if this meets Rachel's needs, then reprocess the rest of her data via this route if needed.

Handed ticket to Ben since he's looking after this one now :)

[benj]

Rachel confirmed that the AISA tools output was of acceptable quality so I'm processing the remainder of her eagle flight lines. I'm using Bill's new azspec version to do it with rather than AISA tools because that's faster and appears to give the same output on checking.

I have sent the Glasfynyyd data to Rachel via FTP (and told her the above and suggested she have a quick check that it's what she's expecting) and am currently processing the Clocaenog data.

[benj]

Clocaenog data delivered to Rachel. However, the Eagle data for Clocaenog do not line up with each other or with the previously-processed ATM and CASI - boresight numbers are believed to be slightly wrong (previous flight lines using these boresight numbers showed what is appeared to be slight inaccuracy with the pitch offset). Inaccuracy relative to ATM and CASI is quite large though, plus the CASI image seems to be stretched across the line of flight compared to the Eagle data, so it may be there is some other source of inaccuracy as well.

Rachel says that she needs the flight lines to align correctly if at all possible (not necessarily exactly in the correct position geographically, but aligned relative to each other), so need to get vectors for the area to check correct positioning.

[benj]

Got vectors from AKW, adjusted boresight numbers and reprocessed Eagle images to match vectors. New Eagle data for both Glasfynydd and Clocaenog delivered to Rachel via FTP, who says she thinks she's got everything she needs now. Marking ticket fixed until/unless Rachel comes back saying she needs something else.