Opened 8 years ago

Last modified 3 months ago

#260 accepted flight processing

IPY09/10, flight day 217/2009, Russell

Reported by: knpa Owned by: knpa
Priority: alpha 5 Milestone:
Component: Processing: general Keywords:
Cc: Other processors: benj

Description (last modified by knpa)

Data location: ~arsf/arsf_data/2009/flight_data/ipy/IPY09_10-2009_217_Russell

Data arrived from ARSF via SATA disk Daphne03 on 24/08/09.

Scientific objective: Supra-glacial lake survey and glacial discharge measurements to investigate link between the two.

Priority: alpha-5

PI: A. Hubbard

Sensors:

  • Eagle (22/03/10)
  • Leica LIDAR (27/01/10)
  • RCD (27/01/10)

Change History (36)

comment:1 Changed 8 years ago by knpa

  • Description modified (diff)

comment:2 Changed 8 years ago by emca

  • Description modified (diff)

PI changed to Alun Hubbard (Aberystwyth)

comment:3 Changed 8 years ago by knpa

Corrected logsheet project code from GB09_20

comment:4 Changed 8 years ago by knpa

Seems to be an extra line of lidar data at 13:11

comment:5 Changed 8 years ago by benj

  • Owner set to benj

comment:6 Changed 8 years ago by benj

Eagle lines 13 and 14 have no sync and there is no Hawk data to compare to. Will need to try and compare to Lidar to try and fix sync (not really possible to do them on their own - it's a glacier, there aren't any regular patterns to try and straighten up).

comment:7 Changed 8 years ago by benj

Base station data received from PI and put into ~arsf/arsf_data/2009/flight_data/ipy/IPY09_10-2009_217_Russell/rinex/from_PI (previous base station data received with original dataset and downloaded from web at ~/arsf_data/2009/flight_data/ipy/IPY09_10-2009_217_Russell/rinex/with_data).

Accompanying email from PI below. Note that three of the five base stations were on a glacier and will be moving.

Dear Ben (& Gary/Carl/Dan),

After much delay we managed to decompose the rather large file from the Leica 1200 that was logging at 1 second all summer...  the final file was 2.5GIG & neither Alan Hobbs nor Leica new how to break down to manageable chunks.  Thankfully the most recent version of Tecq came to the rescue...

So I have placed a zip file with records from 6 stations (including Kelyville) which cover the 5th August 2009 when you carried out the Russell Glacier (IPY09) survey.  It can be found on:

ftp://fserver.natgeo.su.se/alh/Rus/5viii09-ARSF.zip

Please note that - 'kely' & 'base' are bedrock fixed monument sites.  The others are 'on ice' stations & will likely have moved in a westerly direction between ~3 - 6 cm over the 3 hours of the survey.

I apologise about the delay in getting this data to you - I have been working on it - trying to break out the Leica 1200 files - for quite a while.  I am happy that we got there in the end.

Please confirm receipt of this email & download of the data.

Many thanks - good luck with your many other projects.  I look forward to getting into our results.

Best wishes,


Alun Hubbard

comment:8 Changed 8 years ago by knpa

  • Owner changed from benj to knpa
  • Status changed from new to accepted

comment:9 Changed 8 years ago by knpa

Navigation done and all lines initially processed.

Some lines appear to have problems:

A18 - Develops an issue two thirds of the way through in which there is a data gap under the nadir of the plane (but still some data from the swath edge).

A16 - Has the above issue all the way through (very little data)

A15 - Has the above issue all the way through (very little data)

A10 - Has the above issue for part of flightline

In addition, many of the flightlines appear quite short.

comment:10 Changed 8 years ago by knpa

Roll corrected (values in aux_files).

Started QC stage. Currently waiting to get on t-scan to tun macro.

comment:11 Changed 8 years ago by knpa

Project delayed over issue with splitting and merging las files (in order to put through macro).
We think we've resolved it now.

comment:12 Changed 8 years ago by knpa

Two more flightlines with little or no data (seems like same issue as described three comments above):
A14 and A13.


comment:13 Changed 8 years ago by knpa

In addition to above issue on affected flightlines, most flightlines exhibit a higher amount of noise than usual, classification macro seems to have picked up most of these.

comment:14 Changed 8 years ago by knpa

Ignore comment about A13 and A14 - these were reflown

comment:15 Changed 8 years ago by knpa

Preparing lidar delivery

comment:16 Changed 8 years ago by knpa

delivery completed, awaiting check

comment:17 Changed 8 years ago by benj

  • Other processors set to benj

Doing delivery check

comment:18 Changed 8 years ago by benj

Delivery looks fine, can be rsynced back and delivered.

comment:19 Changed 8 years ago by knpa

rsynced back to ~arsf/arsf_data/2009/flight_data/ipy/IPY09_10-2009_217_Russell.

Exempted most eagle/hawk and photography stuff as these are still in progress (and bring total to 1.3TB).

Command:

rsync -av /users/rsg/knpa/workspace/IPY09_10-2009_217_Russell/ ./ --exclude 'eagle' --exclude 'hawk' --exclude 'admin' --exclude 'leica/raw_laser' --exclude 'leica/webcam' --exclude 'lev1' --exclude 'lev3' --exclude 'delivery/20100115/' --exclude 'leica/rcd' --exclude 'rune/logs' --exclude 'leica/ipas/raw' --exclude 'calibration' --exclude 'applanix/Raw' --exclude 'leica/ipas/base'

comment:20 Changed 8 years ago by knpa

Ready for delivery. Currently waiting for photos to include on same disk

comment:21 Changed 7 years ago by knpa

LiDAR and Photography delivered today (27/01/10)

comment:22 Changed 7 years ago by knpa

  • Description modified (diff)

comment:23 Changed 7 years ago by knpa

HYPERSPECTRAL

Eagle line 1 has broken up into three sections and was recorded as three separate flightlines.
This line therefore has gaps in data and every subsequent line is two out of sync with the logsheet (i.e. line 4 is 6th entry on logsheet).

comment:24 Changed 7 years ago by knpa

The DEM for each flightline were to small to cover the all hyperspectral data, project had to wait until I had writted a script to extend each of the DEMs.

DEM's stretched and processed for scts -0.08 to +0.08

comment:25 Changed 7 years ago by knpa

Eagle SCTs found for all flightlines apart from-

4: Insufficient LIDAR data for which to check against.
5: Insufficient LIDAR data for which to check against.
13: Completely featureless ice

comment:26 Changed 7 years ago by knpa

Whilst finding SCTs noticed an across-track shift to the left of flight direction. May be a correctable boresight error.

comment:27 Changed 7 years ago by emca

Delilvery checked hyperspectral data at
/users/rsg/airborne/workspace/IPY09_10-2009_217_Russell/delivery/20100316

Ran dimfixer.py on the BIL files as some of them were affected by the extra lines problem
Renamed the screenshots (removed _gptsct from the filenames)
Converted mosaic screenshot from png to jpg
Minor edits to readme

DEM has not been generated for all of the flightlines - need to add something in the readme to state this. Also, DEM line numbers do not match with the lev1 file line numbers (e.g need to use DEM 18 for lev1 file 20).

comment:28 Changed 7 years ago by knpa

Put note in Readme explaining the lack of DEMs for some flightlines.
Renamed DEM lines to "A17" formate so they match the hyperspectral files.

Ready to deliver.

comment:29 Changed 7 years ago by knpa

  • Description modified (diff)

Hyperspectral delivered today (22/03/10)

comment:30 Changed 7 years ago by knpa

  • Component changed from Processing: general to Archiving

Ready for archiving

comment:31 Changed 7 years ago by knpa

  • Component changed from Archiving to Processing: general

EDIT: Do not archive until rsynced (respository currently full).

comment:32 Changed 7 years ago by knpa

SCTs

E

1 0.01
2 -0.05
3 0.00
4 x (No lidar)
5 x (No lidar)
6 -0.01
7 -0.07
8 -0.04
9 -0.07
10 -0.05
11 -0.06
12 -0.04
13 1.58 (SYNC)
14 0.95 (SYNC)
15 -0.03
16 0.01
17 0.0
18 -0.06
19 0.02
20 -0.01
21 -0.06
22 0.02
23 0.04
24 0.01
25 -0.01
26 0.4
27 -0.02
28 -0.03
29 0.04
30 0.00
31 -0.05
32 -0.06

comment:33 Changed 7 years ago by knpa

The lidar for this project was processed with the wrong calibration file, have reprocessed the lines. Corrected lines Will be sent out once have looked into inquiry by Andrew Fitzpatrick.

comment:34 Changed 9 months ago by dac

Archiving

There were a lot of files in the main directory which looked like tests. I've moved these to processing/tests. No reprocessed hyperspectral but LiDAR, Camera and az processed hyperspectral ready to be uploaded to NEODC.

comment:35 Changed 6 months ago by asm

Archiving

Started. This project also needs raw data to be archived. Have cleansed the test files.

comment:36 Changed 3 months ago by asm

Archiving

Archived and it is available from CEDA. Processed data at http://browse.ceda.ac.uk/browse/neodc/arsf/2009/IPY09_10/IPY09_10-2009_217_Russell

Plymouth 12th May 2017

Note: See TracTickets for help on using tickets.