An artificial intelligence algorithm transforms still images into an explorable 3D world. This could have implications for film effects or virtual reality.
The neural network can accurately visualize what the scene would look from any perspective by feeding it images of a scene as well as a rough 3D model created using off-the shelf software called COLMAP.
Darius Rckert and his colleagues from the University of Erlangen–Nuremberg, Germany, created the neural network. It is capable of extracting physical properties from still photographs.
He says that we can alter the camera position to get a different view of an object.
Advertisement
Although the system could theoretically create a 3D explorable world from two images, it would not be very accurate. Rckert says that the higher the number of images, the better quality it can create. Rckert says that a model can't create things it hasn't seen.
The generated environments that are smoothest use 300 to 350 images taken from various angles. Rckert plans to improve the system by having the software simulate how light bounces off objects within the scene to reach it. This would reduce the number of still images required for precise 3D rendering.
Tim Field, founder and CEO of Abound Labs in New York, states that until now, 3D reconstructions were not fully automated. He also noted that the process of creating photorealistic images was slow and had obvious flaws.
You can turn still images into a 3D world Darius Ruckert and co.
Field acknowledges that the system requires 3D data to be accurate and does not yet support moving objects. However, Field says the rendering quality is exceptional. It is proof that automated photorealism can be achieved.
Field predicts that the technology can be used to create visual effects in films as well as virtual reality walksthroughs of places. He believes it will accelerate machine learning-based rendering of computer-generated imagery.
Reference: arXiv.org/abs/2110.06635
