Data presented with the kind permission of Canfor
We underwent a study in order to evaluate TSI's accuracy in tree segmentation, species identification, and DBH prediction. A test set of 6 stem mapped plots were used, spanning a variety of Biogeoclimatic Zones. Each plot was 11.28m, or 400m² in size. Below is a sampling of our test results.
Each sample Includes an image of the LiDAR of the plot (Point Cloud View, the color of each tree is dictated by TSI's calculation of which species it is), a segmentation image (which shows the points / species calls from the Cruise, as well as the shapes that were segmented by TSI, colored by species), and a Plot Summary (which includes a breakdown of the plot, and TSI's corresponding analysis).
From the data gathered from the cruise, we know that half the trees in this plot should be BL (Balsam), with the other half split between FD (Fir) and SX (Spruce Hybrid). Of the 29 trees counted from the ground, TSI segments 20 trees (accuracy of 69%), and of those 20 segmented, correctly identifies the species of 15 (accuracy of 75%).
Comparing TSI to Ground results in the chart, dominants and co-dominants look reasonable. Some of the BL was short (under 20m) and had a smaller diameter, which contributes to why it could have been missed. TSI's accuracy in DBH approximation holds up nicely with the plot area having an average DBH of 25.4cm, and TSI just slightly overcalculating the average DBH to be 26.4cm.
This plot is split between two species. Two-thirds of it being SX (Spruce Hybrid), and one-third going to BL (Balsam). Of the 16 trees collected from the cruise, TSI successfully segmented 13 (accuracy of 81%), and of those 13, correctly identifies the species of 11 (accuracy of 85%).
As can be seen in the segmentation image, the three trees TSI failed to segment were all neighboring taller, dominant trees. This resulted in these smaller trees being occluded from the segmentation. Similar to the previous plot, TSI again has a high degree of accuracy when calculation the DBH of trees in the plot, with the plot area having an average DBH of 35.3cm, and TSI coming in just under at 34.4cm.
This plot is made up of approximately half PL (Lodgepole), with the remainder being split between BL (Balsam), and SX (Spruce Hybrid). The Cruise found a total of 18 merchantable trees, TSI segmented 11, with the shorter trees missing (accuracy of 61%). Of these 11 segmented trees, matching them by height and proximity, TSI correctly predicts the species for 10 of them (accuracy of 91%).
The one tree TSI failed to predict the species of was an LW (Western Larch), likely due to the fact that there were two field collect trees under that one segmentation. TSI's average DBH score is respectable but 34.7 cm is clearly high in comparison to the cruise's 30.6 cm. This is likely due to the smaller trees not being segmented by TSI.
A summary of all plots used in the study give us an overview of the TSI accuracy. On average, TSI had a species accuracy of 78%, and a segmentation accuracy of 71%. With this data, we can conclude that TSI can accuractely segment the larger, more important stems in terms of volume, and that it's species identification is largely correct.
During our study, we also looked into how well TSI's volume scores compared to known Cruise and Scale volumes.
A quick note about our methodology when comparing TSI/Cruise Data to scale data:
As a sample of our analysis, two areas will be looked at. The first area is made up of 3 blocks, totalling 15 ha, and will hereby be refered to as the GRA CP Blocks. The second area is a single block, which will be refered to as Dunn 0033.
The species mix corresponds well between the cruise data and TSI's findings. TSI does find additional PL (Lodgepole) not called for by the cruise. The real concern, however, was that TSI's volume was close to half of the cruise volume. Sometime later, scale data became available and cast some doubt on the veracity of the cruise.
The species mix (above) corresponds well between both the cruise and scale. Although the cruise volume is higher than what TSI calls for, it is only a slight deviance when compared to the GRA CP Blocks. The scale volume, however, matches TSI's calculation very well.
Plot data is likely the best way to test TSI's accuracy, however, too few plots were examined to draw statistically significant conclusions. Although comparisons with cruise and scale are imperfect, they do show solid correlations with TSI results. When comparing TSI volume to scale volume, a high degree of accuracy was seen.