Neural Rendering and Gaussian Splatting

For eighteen chapters the point cloud has been the primary representation of three-dimensional space, a recorded set of measurements with a sensor, a timestamp, and an evidence trail behind every coordinate. Since 2020 the visual computing literature has been dominated by an alternative in which the scene is encoded as the weights of a coordinate network or as a collection of learned primitives. Neural Radiance Fields (NeRF) asks for dozens of calibrated photographs and roughly twelve to forty-eight GPU-hours, and returns photorealistic novel views at a quality the classical mesh-and-texture pipeline cannot approach. 3D Gaussian Splatting, three years later, returns a comparable or better result in fifteen to forty-five minutes of training and renders at over a hundred frames per second on consumer hardware, at the price of a model file approaching one gigabyte before compression.

Neither, in its current form, replaces the point cloud for surveying or heritage measurement. The heritage façade is where neural rendering and laser scanning meet most productively, and where the temptation to treat a neural reconstruction as a measurement-grade record is most dangerous.

This closing chapter consumes nearly everything the book has built. The photographs it learns from are posed by the structure-from-motion workflows met alongside LiDAR in Chapter 3, the point clouds that seed and constrain its hybrid reconstructions arrive registered by the methods of Chapter 8, and the mesh-and-texture pipeline it collapses is the surface reconstruction of Chapter 15. Its learned representations extend the deep architectures of Chapter 18 from per-point prediction to whole-scene appearance, and the digital twins they enable serve the application domains of Chapter 17. The chapter develops NeRF first, then 3D Gaussian Splatting, weighs the two against each other, and closes with what neural rendering does and does not yet offer measurement-grade geomatics.

From point clouds to neural scene representations

Traditional point cloud processing follows a linear pipeline, drawing on the methods of the preceding chapters:

Acquire → Register → Filter → Classify → Mesh (Chapter 15) → Texture → Render

Each step introduces errors and requires parameter tuning. Neural scene representations collapse much of this pipeline into a single optimisation solving geometry and appearance jointly:

• Input: multi-view images with known camera poses (and optionally point clouds or depth maps).
• Output: a continuous 3D representation that can be queried at any location.
• Rendering: new views are synthesised by querying the representation, with no meshing or texturing required.

The figure below contrasts the two pipelines side by side.