Accelerating Neural Point-Based Graphics

An alternative project for neural rendering: creating a 3D object from multiple photos to produce another image with the desired perspective. This is an alternative to mainstream neural rendering, where an implicit representation of density or surface is learned. In this method, new images are rendered from Point Cloud. The authors pumped the previous paper, where point descriptors were learned for each scene separately. Here descriptors for each point in the cloud are predicted by neuron for one run, i.e. the same neuron can work on many scenes. The method is short: we run COLMAP, get Point Cloud + camera positions, then run the input images through CNN, get descriptors for each point in the cloud. Then we rasterize the 3D points with their descriptors and run it all through U-Net, which renders the final image.