Anyone can try NeRF now with a free app!

The 3D industry has been buzzing about Nvidia’s Instant NeRF (which stands for neural radiance fields) ever since they published their first demo video.

This technology has since been in rapid development and we now have a free iPhone app called Luma AI that anyone can use to capture NeRFs.

I tested the app with a toy robot, here’s a automatically generated video of the result:

The amazing thing about NeRF renders is that they can handle light reflections and bounces in a very realistic manner. This makes it a good fit for VFX work. We can also export a textured 3D-model from the Luma AI app, but it’s not as impressive as rendering with NeRFs. It’s still quite good compared to a basic photogrammetry process, especially considering that the surface of our object was quite reflective. Here’s a screenshot from Blender:

Here is how the mesh looks like (very dense):

Here’s another cool shot from a Robotime Ball Parkour toy:

@oneminutevideotutorial

Wow! NeRFs are going to be great for displaying products like this!

♬ original sound – user635578337916 –

Best free 3D-scanning app for iPhone in 2023

I have been testing out a lot of 3d-scanning apps for my iPhone 12 Pro recently. The Lidar sensor on the phone was one of the major reasons why I decided to make at least a temporary switch from the Android ecosystem to the iOS world. I’m especially interested in creating 3d-versions (or digital twins) of different rooms and spaces. Here are some of the apps I have already tried:

DotProduct ios app

3D Scanner app

Scandy Pro

Qlone

Kiri Engine

Widar

RTAB-Map

3D Scanner

Scaniverse

Modelar

Polycam

RoomScan

Pix4DCatch

Metascan

Heges

Trinio

ItSeez3D (iPad only)

Most of the apps in the list use the freemium model, where you can do a basic scan for free, but in-app purchases are often required at the export stage. There were two exceptions that were free from start to finish:

3D Scanner App and RTAB-Map of which the latter is open source.

I did some scans of the same space with both apps and the process was quite enjoyable with both. The quality was however somewhat more limited with the 3D Scanner App and in the end I was able to to get best results with RTAB-Map. So out of these scanner apps that I tested I would recommend RTAB-Map. It has versions not only for iPhone, but turns out it’s a very versatile app used for many purposes and it has also desktop releases for the major operating systems, including Windows, Mac and Linux.

I did experience some problems when I tried to do a really detailed scan. RTAB-map probably ran out of memory on my iPhone and crashed quite frequently when post-processing the scans. Luckily it turns out that you can simply export the database from the iPhone in order to do the processing on the desktop version instead. It might not be obvious at first how the RTAB-Map database can be accessed or exported on the iPhone, but you simply need to long-press on a scan in the library and choose “share”.

After you haved shared your database and downloaded it to your desktop computer, you can simply import it to the desktop version of RTAB-Map. I must say I found the interface to be rather confusing. For example, when I was looking for the option to export a textured mesh, the way to do it was to choose the “export 3d clouds” option, although one might thing that a mesh is not exactly a “cloud”. And there wasn’t a typical option to just “export a textured mesh” but you have to check the “meshing” checkbox and then the “texturing” checkbox for the texture options. And after that I actually had to close and the re-open the export to be able to see the “save” button. So quite quirky, but it worked out well in the end.

One more piece of information I want to share about using the RTAB-map on the iPhone: one time I had it crash right after a scan and at first I thought the scan had disappeared completely but after some searches in the “issues” page of their Github, I found out that some kind of recovery might be possible in the latest version. The problem was that I just couldn’t find any recovery option in the app interface. Finally I gave up and decided to do a rescan and when I chose to do a new scan, that’s when the recovery dialogue for the previous scan finally appeared and I was able to recover my crashed scan.

Two story house scanned with 3D Scanner App

Sauna scanned with RTAB-Map

Open source tool for viewing e57 files

If you are working with laser scans and lidar data, the chances are you have bumped into e57-files, which are often used for point clouds.

Finding a good tool for opening the e57-files can be a challenge, since many of the solutions are proprietary and quite expensive.

Luckily, there is an open source tool that can import many different data formats, including e57-files. It’s called Meshlab and it has builds for Windows, Mac and Linux. The ability to open e57-files was added to the program quite recently, but it works well.

Check out Meshlab here:

https://www.meshlab.net/

Blender keyboard shortcuts for working with nodes

Being able to work efficiently with different kinds of nodes in Blender is becoming increasingly important. These basic node related keyboard shortcuts should work in all of the Node editors in Blender including material nodes, the compositor and geometry nodes.

Add new node: shift+A

Just like you can add new objects with shift+a, the same shortcut also works for adding new nodes. If you want to immediately enter the search mode, read about the next shortcut below.

Add new node and search immediately: shift+A, then S

If you want to search for the name of the node instead of browsing through the categories, hit s immediately after hitting shift+a and it will take you to the search bar.

Disable a node: M

Sometimes you just want to see the results of a node tree without a certain node. In that case you can easily mute a node by selecting it and hitting M on your keyboard.

Delete a node: X

Just like you would expect, hitting X on the keyboard will delete the selected node(s).

Delete with reconnect: ctrl+X

If you want to delete a node in the middle of a node tree and automatically connect the node that came before the deleted node to the node where the deleted node was connected to, simply hit ctrl+X to delete the node from between.

Backdrop Zoom: V and alt+V

If you need to zoom out of the backdrop image to see it completely, just hit V on the keyboard. Zoom in by pressing alt+V.

Duplicate a node: shift+D

To make a duplicate of the selected node(s), just hit shift+D on the keyboard.

Connect a node to the viewer: ctrl+shift+leftclick

If you want to quickly connect a node’s output to the viewer node to see the result of the node, just hold down shift and control and left-click on the node you want to connect.

Cut a node connection: ctrl+rightclick-drag

If you want to cut a node connection, hold down control and the right mouse button and drag over the connection line.

Collapse node: H

Sometimes you want to collapse a node so that it takes less space. To do that, simply select the node and hit H on your keyboard and the node inputs will no longer be visible. If you want to reopen a collapsed node, simply hit H again.

Create node group: ctrl+G

Grouping nodes together allows you to simplify your node trees. By grouping several nodes together it is like combining them so that they appear to be just a single node. Simply select the nodes that you want to group together and hit ctrl+G. Note that all node types cannot be grouped together. For example you can’t put the “render layers” node in a node group.

Enter or exit node group: tab

After grouping nodes, Blender will automatically take you inside the group. To exit the node group, press tab on the keyboard. Similarly, to enter a node group, select the group and press tab.

Basics of Using Cryptomatte in Blender 2.93

In the buttons area (on the right side of the interface), select the “View layer properties” tab.

Under “cryptomatte” turn on “object”, “material” or “asset”. I like “asset” since it let’s me select entire rigs that consist of several parts.

Go to the “compositing” workspace.

Turn on “use nodes”.

Add a viewer node with shift+a –> output –> viewer.

Add the cryptomatte node from Matte –> Cryptomatte.

Connect the image output from the render layers node to the image input of the Cryptomatte node. Connect the “pick” output from the Cryptomatte node to the image input of the viewer node. Render the scene (keyboard shortcut F12).

You should now see different matte colors that identify different assets in your render layer. Use the + button to access the eyedropper tool and select as many assets as you need for the matte you are building.

To see the actual matte, you can plug the “matte” output from the Cryptomatte node to the viewer.

Now you have a matte that you can use in various way when you are compositing. As a simple example, you could color correct the matted area by combining two copies of the input image with the “AlphaOver” node while using the matte as the factor. Then simply drop a color correction node like RGB curves between the bottom image connection.