Opened 3 years ago

Last modified 18 months ago

#551 new flight processing

RG13/06, flight day 295/2014, SAFE Area

Reported by: dap Owned by:
Priority: alpha 5 Milestone:
Component: Processing: general Keywords:
Cc: Other processors: tec, dap

Description (last modified by dap)

Data location: /users/rsg/arsf/arsf_data/2014/flight_data/malaysia/RG13_06-2014_295_Danum_Valley

Data arrived from network transfer on 17/12/2014.

Scientific objective: Investigating the impact of humans on tropical forests.

Priority: alpha 5

PI: David Coomes

No owl data present

Sensors:

  • Fenix (requested)
  • Leica FW LIDAR (delivered 22/05/2015)
  • Leica LIDAR (delivered 22/05/2015)
  • RCD (requested)

Duplicate lines

Process only one for Fenix and RCD - check for which has least cloud cover, failing that choose closest to local noon

  • 1159-1161 (may be 3? Check vs. day 290)
  • 1250-1252
  • 1185-1189 (day 294)
  • 1188-1200 (day 298, 1188-1189 repeated 3 times - also day 294)

Change History (52)

comment:1 Changed 3 years ago by tec

Navigation Processing

SAFE_11th Basestation Data
Latitude4 43 30.56057
Longitude117 36 19.18859
El Height479.791m

comment:2 Changed 3 years ago by tec

  • Other processors set to tec

comment:3 Changed 3 years ago by tec

Processed in IPAS Pro
C/A 1.20
ell 12.5
max freq distance 80km

after 1:28:20 the lat/long positional separation goes from between 0±0.05 to 0±0.1

Nav done

comment:4 Changed 3 years ago by benj

  • Description modified (diff)

Added duplicate line details

comment:5 Changed 3 years ago by lah

Fenix duplicates of 290
1159 - line 26 is best
1160 - line 24 is best
1161 - line 25 is best

These have not been processed as part of 290, so need to be processed as 295.

comment:6 Changed 3 years ago by tec

Nav
Data in ~/arsf_data/2014/flight_data/malaysia/RG13_06-Basestation_data/RINEX DATA SABAH/Rinex/temporary_gps_dataz/295, move out into project folder once processed.

comment:7 Changed 3 years ago by dap

RCD Processing started

comment:8 Changed 3 years ago by tec

Navigation Processing
Antenna Height 1.31
Lat 4 43 30.56119
Lon 117 36 19.18667
Ell 480.179

comment:9 Changed 3 years ago by tec

Navigation Processing
From first flight line until CROSS_004 navigation data is horrible, nothing I can do about that.

comment:10 Changed 3 years ago by tec

RCD Processing
Starting

comment:11 Changed 3 years ago by tec

RCD Processing
Removing images 352, 386, 387, 406, 425.

comment:12 Changed 3 years ago by tec

RCD Processing
Ready for DC

comment:13 Changed 3 years ago by dap

RCD Delivery Check

Complete. Looks fine, just had to change the data quality remarks to make them read a little better.

Marking as ready for delivery.

comment:14 Changed 3 years ago by tec

Delivery
Rsyncing to FTP:arsf1 DONE

Last edited 3 years ago by tec (previous) (diff)

comment:15 Changed 3 years ago by lah

PI notified of RCD delivery 05/05/2015 (RCD only)

comment:16 Changed 3 years ago by dap

LiDAR Processing

Roll and pitch offsets found. The values seem to alternate between two numbers for the majority of the flight lines.

Roll and pitch values as follows:

Flight lineRoll errorPitch error
230824-0.00163180-0.006375
231103+0.00063180-0.005975
231339-0.00163180-0.006375
231654+0.00063180-0.005975
232347-0.00163180-0.006375
232655+0.00073180-0.005975
233006-0.00163180-0.006375
233313+0.00073180-0.005975
233616-0.00163180-0.006375
233934+0.00073180-0.005975
234230-0.00163180-0.006375
234547+0.00073180-0.005975
234914-0.00163180-0.006375
235208+0.00073180-0.005975
235520-0.00163180-0.006375
235840+0.00073180-0.005975
000142-0.00163180-0.006375
000451+0.00073180-0.005975
000815-0.00163180-0.006375
001128+0.00073180-0.006675
001455-0.00163180-0.006375
002256-0.00163180-0.006375
002553+0.00073180-0.005975
002828-0.00163180-0.006375
003138+0.00073180-0.005975
003527-0.00163180-0.006375
003904-0.00163180-0.006375
004150+0.00073180-0.005975
004444-0.00163180-0.006375
004918+0.00073180-0.005975
005220-0.00163180-0.006375
005452+0.00073180-0.005975
010247-0.00163180-0.006375
010555+0.00073180-0.005975
010826-0.00163180-0.006375
011115+0.00073180-0.005975
011350-0.00163180-0.006375
011623+0.00073180-0.005975
011903-0.00163180-0.006375
012330+0.00073180-0.005975
012628-0.00163180-0.006375
012937+0.00073180-0.005975
013220-0.00163180-0.006375
013529+0.00073180-0.005975
013824-0.00163180-0.006375
014130+0.00073180-0.005975
014441-0.00163180-0.006375
014732+0.00073180-0.005975
015038-0.00163180-0.006375
015605-0.00163180-0.006375

comment:17 Changed 3 years ago by dap

LiDAR Processing

Corrected files (incl. full waveform) have been produced and moved LAS1.2 files to las/corrected and LAS1.3 files to als50/las-fw. Now classifying LAS1.2 files automatically using the isolated 10 5 algorithm (outputting to als50/las-classified).

comment:18 Changed 3 years ago by dap

LiDAR Processing

All files have been classified automatically and manually and classifications have been copied to the LAS1.3 files. Now creating delivery.

comment:19 Changed 3 years ago by lah

Fenix duplicates
Please process 1185-1189 as these are not in 294. 1188-1200 are also not in 294.

comment:20 Changed 3 years ago by tec

Fenix
Starting processing

comment:21 Changed 3 years ago by dap

LiDAR Processing

Delivery and read me have been created. Now ready for delivery check.

comment:22 Changed 3 years ago by lah

Fenix Duplicates
Lines 42-51 (1191-1200) do not need to be processed as day 298 has less cloudy duplicates.

comment:23 Changed 2 years ago by dap

Fenix Duplicates

Lines 6-20 (1205-1219) do not need to be processed as day 297 (ticket:552) has less cloudy duplicates.

Last edited 2 years ago by dap (previous) (diff)

comment:24 follow-up: Changed 2 years ago by lah

Lidar DC

  • Pitch & roll look ok.
  • Classification:
    • Most lines ok with occasional lone points not classified as noise.
    • Lines 17, 28, 29 & 41 have missed some rain. Worth reclassifying these?
  • Any idea why the elevation offsets are so consistently high?
  • Changed las1.0 folder to las1.2
  • Changed text files to crlf
  • Test delivery folder needs to be deleted
  • Looks like AGC value has not been saved
  • demcompare: masked mean = 17.3114
  • no errors with laszip on fw
  • fws look fine in waveviewer
  • removed an odd flie from logsheet directory (.nfs000...)

comment:25 in reply to: ↑ 24 Changed 2 years ago by dap

Replying to lah:

Lidar DC

  • Pitch & roll look ok.
  • Classification:
    • Most lines ok with occasional lone points not classified as noise.
    • Lines 17, 28, 29 & 41 have missed some rain. Worth reclassifying these?

I've had another look at the classifications and reclassified the missed rain. The number of missed points was quite small so I haven't regenerated the DEMs but have copied the new classifications to the full waveform files.

  • Any idea why the elevation offsets are so consistently high?

I did notice this but wasn't sure why. dac is investigating this.

  • Changed las1.0 folder to las1.2
  • Changed text files to crlf
  • Test delivery folder needs to be deleted

Moving this to my network scratch space (was using it for testing script changes)

  • Looks like AGC value has not been saved

I've ran lasinfo on some of the LAS 1.2 files, the AGC values are in 'point_source_ID' field in them and the AGC values are present in both the 'user_data' and 'point_source_ID' fields of the LAS1.3 files. dac said this is OK.

  • demcompare: masked mean = 17.3114

dac said this is fine.

  • no errors with laszip on fw
  • fws look fine in waveviewer
  • removed an odd flie from logsheet directory (.nfs000...)

comment:26 Changed 2 years ago by dap

LiDAR

dac has had a look at the elevation differences and we suspect this is due to the poor quality of the navigation data (as mentioned in comment:9), but there is nothing we can do about it.

comment:27 Changed 2 years ago by dap

LiDAR

I've added a sentence in the read me about the high elevation offsets.

LiDAR dataset ready for a recheck if necessary or delivery if not.

comment:28 follow-up: Changed 2 years ago by lah

LiDAR
Just noticed the screenshot names are 2014-295 rather than 2014_295, but we have delivered both forms, so probably nothing to worry about.

comment:29 in reply to: ↑ 28 Changed 2 years ago by lah

Replying to lah:

LiDAR
Just noticed the screenshot names are 2014-295 rather than 2014_295, but we have delivered both forms, so probably nothing to worry about.

actually they should be like that, so never mind. reaady to deliver

comment:30 Changed 2 years ago by dap

rsyncing LiDAR delivery to FTP server (arsf1)

LiDAR delivery rsync complete and PI has been notified of LiDAR delivery (22/05/2015).

Last edited 2 years ago by dap (previous) (diff)

comment:31 Changed 2 years ago by dap

  • Description modified (diff)
  • Summary changed from RG13/06, flight day 295/2014, Danum Valley to RG13/06, flight day 295/2014, SAFE Area

comment:32 Changed 2 years ago by tec

  • Description modified (diff)
  • Summary changed from RG13/06, flight day 295/2014, SAFE Area to RG13/06, flight day 295/2014, Danum Valley

Fenix Processing
Fenix ready for delivery check.

comment:33 Changed 2 years ago by tec

  • Description modified (diff)
  • Other processors changed from tec to tec, dap
  • Summary changed from RG13/06, flight day 295/2014, Danum Valley to RG13/06, flight day 295/2014, SAFE Area to RG13/06

comment:34 Changed 2 years ago by dap

  • Description modified (diff)

comment:35 Changed 2 years ago by dap

  • Summary changed from RG13/06, flight day 295/2014, SAFE Area to RG13/06 to RG13/06, flight day 295/2014, SAFE Area

comment:36 Changed 2 years ago by dap

Starting Fenix Delivery Check

comment:37 Changed 2 years ago by dap

Fenix Delivery Check

Just a couple of issues, which I've fixed:

  • Removed fodis directory
  • Renamed the screenshots
  • Copied logsheet to delivery

I've also zipped the mapped files up, so ready for delivery.

comment:38 Changed 2 years ago by lah

Fenix uploaded to ftp and email sent 12/06/15.

comment:39 Changed 2 years ago by stgo

Starting Fenix reprocessing

comment:40 Changed 2 years ago by stgo

Fenix reprocessing

Ready for delivery check.

This delivery needs to be thoroughly checked for:

  • correct lines included
  • correct linenames from logsheet
  • data quality comments
  • spectra shifts

comment:41 follow-up: Changed 2 years ago by asm

Fenix reprocessing DC

Started
-Removed fodis directory.
-Added logsheet.
-All flightlines has correct name compared with logsheet.
-Please, add some extra information on Readme about data quility for flightline 52 which has an inverse 'C' shape.
-Some extra information about flightlines not included should be added (both readme and ticket). The ticket only mentions duplicated lines from number 6 to 20 and 42-51. However number 6 is included and number 5 is not. Also missing flightline 23.
-Need to change data quality report to the latest one.

Last edited 2 years ago by asm (previous) (diff)

comment:42 Changed 2 years ago by asm

Fenix reprocessing DC

Checked green vegetation spectra with Py6S. Good match for all flightlines.

comment:43 Changed 2 years ago by asm

Fenix reprocessing DC
Tested all tif files created from apl commands.

comment:44 Changed 2 years ago by lah

Fenix DC - spectra
Checked spectra of a few lines and no major problems. There is a persistent spike in the mask region for this flight, not unlike many other flights. The spectral shift is within 5 nm compared to previous data (2014 169).

comment:45 in reply to: ↑ 41 Changed 2 years ago by stgo

Replying to asm:

Fenix reprocessing DC
-Please, add some extra information on Readme about data quility for flightline 52 which has an inverse 'C' shape.

Will say something similar to 298, I'm assuming it is the same "F2" error in that the sensor doesn't terminate a flightline.

-Some extra information about flightlines not included should be added (both readme and ticket). The ticket only mentions duplicated lines from number 6 to 20 and 42-51. However number 6 is included and number 5 is not. Also missing flightline 23.

Missing flightlines weren't included in the original delivery, no explanation was given for the reasoning. I'll put a comment that flightlines are missing but tec hasn't left me a great deal of information to go on, I won't include flightlines that weren't in the original delivery though.

-Need to change data quality report to the latest one.

True of all flights

comment:46 Changed 2 years ago by stgo

Fenix reprocessing

Made changes outlined above, new readme in the delivery folder.

comment:47 Changed 2 years ago by asm

Fenix DC
-Need to change flightline 006a (wrong name being 005a) on readme to have the same flightlines than the last delivery.

comment:48 Changed 2 years ago by stgo

fenix dc
Fixed readme flightline numbering

comment:49 Changed 2 years ago by asm

Fenix DC:

All comments addressed and solved. Creating zipped files now. Would be ready to go once they have been created.

comment:50 Changed 2 years ago by asm

Fenix delivery

Delivery placed on FTP server (arsf1). Notification mail to PI still pending remaining deliveries within the same project.

comment:51 Changed 2 years ago by asm

Fenix delivery

Notification sent to PI on 3rd November 2015. Delivery finalised.

comment:52 Changed 18 months ago by dac

Archiving

Raw and archived data available from NEODC: http://browse.ceda.ac.uk/browse/neodc/arsf/2014/RG13_06/RG13_06-2014_295_Danum_Valley

Marking as closed.

Note: See TracTickets for help on using tickets.