OpenAI Releases Point-E, an AI For 3D Modeling

If you were to input a text prompt, say, "A cat eating a burrito," Point-E will first generate a synthetic view 3D rendering of said burrito-eating cat. It will then run that generated image through a series of diffusion models to create the 3D, RGB point cloud of the initial image -- first producing a coarse 1,024-point cloud model, then a finer 4,096-point. "In practice, we assume that the image contains the relevant information from the text, and do not explicitly condition the point clouds on the text," the research team points out. These diffusion models were each trained on "millions" of 3d models, all converted into a standardized format. "While our method performs worse on this evaluation than state-of-the-art techniques," the team concedes, "it produces samples in a small fraction of the time."