Data presented with the kind permission of Green Diamond Resource Company
This project sought to better understand TSI's ability to accurately predict the species of individual trees. For our test set, we used 29 100% stem-mapped plots, each 0.75 acres in size. The plots were distributed throughout California and Oregon, ranging across 100 miles from the north to south. The LiDAR was flown in both 2007 and 2008 by WSI with a Leica ALS50. Scan angle was restricted to +/- 8 degrees from nadir.
We will begin by exploring TSI's stem accuracy results on a small sample of our 29 plots. From there we will take a deeper look into the various calibration tests that were performed during this study, and conclude with a quick glance into intensity issues that were discovered during the process.
Calibration tests are used to determine the accuracy with which TSI can determine the species of the trees within its own training set. During our study, we tested how various configurations of our training sets (Ground Truth), led to varying results.
Due to the nature of Redwoods (RW) and deciduous species often diverging into multiple peaks on a single tree, TSI can have difficulties accurately segmenting these species into a single tree. Due to this, we ran two separate calibration tests in order to see where TSI performed best. In the first, species which often have multiple peaks were each granted two species slots, one to account for multi-peak trees (clusters), and one for single peak (singles). In the second, rather than separating out the species by peak, both clusters and singles were clumped together under their natural species name. In both cases, a high degree of accuracy was seen, with the separated clusters and singles test resulting in a 94.2% stem accuracy, and the clusters as singles test resulting in a 95.4% accuracy.
In the second set of calibration tests, additional validation was performed by dividing each plot into multiple sample zones (In example chart: 1, 1E, 2). Each zone type was then combined into a new training set in order to test whether TSI's calibration accuracy held consistent results. This turned out to be largely the case, with both sample zones 1 and 2 resulting in an accuracy in the mid-to-high nineties.
Each plot below has been broken into three sections for analysis. The first section is a chart with a comparision between stem-mapped data (labelled "cruise"), and TSI results. The second section is a 3D view of the plot's LAS point cloud with each tree coloured by TSI's species prediction. The third is an overhead view of the plot, showing TSI's individual tree segmentations and the stem-mapped trees that made up the test group.
In the Cruise-to-TSI comparison chart, Hit Rate % represents a direct comparison between TSI's results and the cruise (in other words, did TSI predict the correct species?). Precision % tests the reliability of the Hit Rate. A high precision indicates TSI is finding the correct species in the correct places.
Note: A species abbreviation table is provided at right.
This first plot is divided between two species, with two-thirds of it belonging to RW (Redwood), and one-third DF (Douglas Fir). TSI's stem accuracy results in this area are strong, with both an average hit rate and precision rate of 93%.
This plot highlights TSI's ability to differentiate between different broadleaved / deciduous species, as the majority of the area is Tanoak (TO) and Red Alder (RA), and the remaining third is Douglas Fir (DF). Both the Tanoak and Red Alder have a 100% hit rate and precision. The overall average falls to 85% due to half the Douglas Fir incorrectly being identified as Redwood (RW). The latter is an issue that is discussed further in the "Intensity Investigation" section of the study.
This plot once more highlights the two points seen consistently throughout our study. The first is the success of TSI is correctly identifiying and differentiating different deciduous / broadleaved species, as can be seen with the 100% accuracy of the Red Alder in this block. The second is TSI's noted difficulty during this study in differentiating between Douglas Fir and Redwood, which receives more detailed discussion in the Intensity Investigation section below.
This study was interesting for a number of reasons. First, it was the first TSI test against a significant number of stem-mapped trees; approx 3500 in all. Second, this was the first TSI analysis that attempted to identify species for trees under 10m in height. And finally, the project goals included a rigorous test to determine multiple deciduous species.
The test was structured in two parts. In part 1, ground truth trees solely from zone 1 were used in the TSI model to determine the species of the trees in zone 2. In the table below we can see the results of that test broken out for each plot. TSI scored a stem accuracy of 78.9%. In part 2, sample trees from both zones 1 and 2 were loowed in the model to determine the species in zone 3. Tested on 3000 zone 3 trees, GDRCo reported an accuracy of 79.6% for TSI.
Throughout this study, intensity values were challenging to use due to inconsistent calibration across the different flight years (2007/2008). As a result, it was discovered (see chart) that the intensity calibration differences almost certainly impacted TSI's accuracy when differentiating between Douglas Fir and Redwoods. GDRCo concluded that the final Sample 3 stem accuracy percentage of 79.6% was likely closer to 85% had there been time & resources to conduct an intensity recalibration.