The 3D industry has been buzzing about Nvidia’s Instant NeRF (which stands for neural radiance fields) ever since they published their first demo video.
This technology has since been in rapid development and we now have a free iPhone app called Luma AI that anyone can use to capture NeRFs.
I tested the app with a toy robot, here’s a automatically generated video of the result:
The amazing thing about NeRF renders is that they can handle light reflections and bounces in a very realistic manner. This makes it a good fit for VFX work. We can also export a textured 3D-model from the Luma AI app, but it’s not as impressive as rendering with NeRFs. It’s still quite good compared to a basic photogrammetry process, especially considering that the surface of our object was quite reflective. Here’s a screenshot from Blender:
Here is how the mesh looks like (very dense):
Here’s another cool shot from a Robotime Ball Parkour toy:
I have been testing out a lot of 3d-scanning apps for my iPhone 12 Pro recently. The Lidar sensor on the phone was one of the major reasons why I decided to make at least a temporary switch from the Android ecosystem to the iOS world. I’m especially interested in creating 3d-versions (or digital twins) of different rooms and spaces. Here are some of the apps I have already tried:
DotProduct ios app
3D Scanner app
Scandy Pro
Qlone
Kiri Engine
Widar
RTAB-Map
3D Scanner
Scaniverse
Modelar
Polycam
RoomScan
Pix4DCatch
Metascan
Heges
Trinio
ItSeez3D (iPad only)
Most of the apps in the list use the freemium model, where you can do a basic scan for free, but in-app purchases are often required at the export stage. There were two exceptions that were free from start to finish:
3D Scanner App and RTAB-Map of which the latter is open source.
I did some scans of the same space with both apps and the process was quite enjoyable with both. The quality was however somewhat more limited with the 3D Scanner App and in the end I was able to to get best results with RTAB-Map. So out of these scanner apps that I tested I would recommend RTAB-Map. It has versions not only for iPhone, but turns out it’s a very versatile app used for many purposes and it has also desktop releases for the major operating systems, including Windows, Mac and Linux.
I did experience some problems when I tried to do a really detailed scan. RTAB-map probably ran out of memory on my iPhone and crashed quite frequently when post-processing the scans. Luckily it turns out that you can simply export the database from the iPhone in order to do the processing on the desktop version instead. It might not be obvious at first how the RTAB-Map database can be accessed or exported on the iPhone, but you simply need to long-press on a scan in the library and choose “share”.
After you haved shared your database and downloaded it to your desktop computer, you can simply import it to the desktop version of RTAB-Map. I must say I found the interface to be rather confusing. For example, when I was looking for the option to export a textured mesh, the way to do it was to choose the “export 3d clouds” option, although one might thing that a mesh is not exactly a “cloud”. And there wasn’t a typical option to just “export a textured mesh” but you have to check the “meshing” checkbox and then the “texturing” checkbox for the texture options. And after that I actually had to close and the re-open the export to be able to see the “save” button. So quite quirky, but it worked out well in the end.
One more piece of information I want to share about using the RTAB-map on the iPhone: one time I had it crash right after a scan and at first I thought the scan had disappeared completely but after some searches in the “issues” page of their Github, I found out that some kind of recovery might be possible in the latest version. The problem was that I just couldn’t find any recovery option in the app interface. Finally I gave up and decided to do a rescan and when I chose to do a new scan, that’s when the recovery dialogue for the previous scan finally appeared and I was able to recover my crashed scan.
