Deep Learning for Point Clouds

You reach this chapter with a pipeline whose every quantity you chose yourself. You chose the neighbourhood, the eigenvalue features and the FPFH descriptors of chapter 10, and in chapter 12 you handed those features to a random forest and tuned it against a labelled tile. That classifier learned how to weight what you gave it and nothing else. Whatever you did not think to compute could not affect its predictions. The expert knowledge behind the choice was yours to supply on every new site. This chapter gives the construction of the features to the optimisation instead.

Which brings a question with it, and it is not which architecture is best. Here is why. Every network in this chapter is trained on one collection of scans and used on another. And what it loses in that move is larger than what the published rankings put between one architecture and the next. Why? Because those rankings are measured inside a single collection. And figure 19.11 carries both halves of that statement. A single set of weights classifies 89.4 per cent of the points correctly on the national survey it was trained on. It classifies 70.9 per cent on a second national survey, over the three classes the two class schemes share. Now how large are the margins that rank the backbones? Of another order. The strongest entries regularly lie closer to one another than the variation between training seeds of any one of them. This variation is what Qian et al. (2022) measured at 0.3 points of mIoU over three runs of a single architecture. These are the gains architectures published between PointNet in 2017 and the Point Transformer line of the 2020s reported on Semantic3D, S3DIS or ScanNet. Now section 19.12 works through what one of them does and does not predict.

So I order the two questions of this chapter against the volume of the literature rather than with it. Your project turns on where the training signal comes from and on how far your deployment data lies from your training data. And the choice among architectures matters less. If labels are scarce, which is the usual condition of survey work, fine-tuning a pre-trained backbone on the labels that exist outperforms training the same backbone from scratch. The advantage grows as the labelled set shrinks (Xie et al., 2020; Pang et al., 2022). So three questions follow, and the chapter takes them in this order. What does a point cloud lack that an image provides a network, and what has been built to work around that? Where does your training signal come from when nobody has labelled your site? And how far will the accuracy you measured fall when the data changes?