Home Unity Tutorials

Using Meshroom to Insert Real Life Objects in Unity

In this article you’ll learn how to use Photogrammetry to photoscan and insert real life objects into your Unity projects with Meshroom.

5/5 2 Ratings

Version

  • C# 7.3, Unity 2019.3, Unity

In the context of 3d models, photogrammetry is the process of making 3D models from objects you’ve scanned.

Have you ever needed some realistic 3D models for a game idea on a budget and felt helpless? Do you have an ultra-realistic game environment to create? If so, then you’ll want to read on!

With photogrammetry, you take a bunch of inputs, your photos, and run them through a pipeline, or series of methods that process them. Data from these processes, or outputs, inform the creation of the final 3D model.

data model for a typical photogrammetry process

In this tutorial, you’ll use several photographs of a dragon statue to create a detailed 3D model, clean it up in Blender and import the result to a dark and moody Unity Viking village scene.

Along the way you’ll learn how to:

  • Choose a good object to photoscan.
  • Take suitable photographs.
  • Decide when and where to take the photographs.
  • Cull bad photographs.
  • Use Meshroom to create 3D scans.
  • Import models to Blender for cleaning and touching up.
  • Reduce polygon count.
  • Prepare and import 3D models to Unity.

preview of the final scanned dragon

Despair no longer! Soon you’ll know the secrets of creating ultra-realistic objects for your games! :]

the final dragon model in Unity

Getting Started

Note: The Meshroom photoscan part of this tutorial requires a fairly powerful CPU to churn through the work in a reasonable amount of time. If you plan to grab a hot drink while you tackle this tutorial, save it for the Meshroom part! :]

Download the project materials by clicking the Download Materials button at the top or bottom of this tutorial. Next, extract the zip file to a convenient location.

The download contains the Starter and Final project files, which you’ll get to later.

Note: There’s one more file to download a little further on containing a collection of photographs. More on that later.

To create a good photoscan you need to capture your model from many angles. In this tutorial, you already have all the photographs you need at your disposal.

Before you get to those photos, it’s important to learn a few tricks for when you go out into the wild to do your own captures.

Finding Good Objects to Capture

Of course you’ll want to capture objects that are relevant to your environment or game idea. After this obvious priority, there are some traits that make an object easier to photoscan:

  • Flat bases.
  • Decent contrast in colors or tones.
  • Not too dark, or difficult to see detail.
  • Not too big, making it difficult to move around and capture from different angles.

You should also consider doing photoscans of objects that would be difficult to model. For example, items in nature are challenging to model because it can be difficult to create organic materials.

These rocks from The Tales Factory on the Unity Asset Store are a good example.

The Tales Factory photoscanned rocks PBR

When to Take Photographs

As with photography, there are certain conditions and times of the day that make for better photoscans.

If you have an indoor studio and lighting, you could model these conditions yourself with a bit of careful planning. If you don’t, you’ll need to work outside.

When working outside, keep the following in mind:

  • Capture photos in daylight.
  • Overcast skies are great. The clouds diffuse the light which results in more even light distribution on your model.
  • Dry conditions are good and wet conditions are bad! In wet conditions, you’re likely to get more reflections on your final result. In some cases, you might also find your 3D model has a shiny gloss that detracts from the realism.

Choose a neutral looking model that could suit a variety of potential conditions. Then you could apply effects depending on the conditions in your game.

Is it raining? Throw on a shader that adds a glossy, wet effect. Or, is your game set in some post-apocalyptic world where everything is old and worn out? Add an aging or corrosion effect instead.

Capturing Photos

Download the photo collection you’ll use for this tutorial. It’s not in the Download Materials link because of its size. Click this link to begin the download:

Download dragon sculpture photos (zip file, 695 MB)

Now, extract the zip file to a convenient location and open the folder. You’ll see about 100 photos of the dragon sculpture, taken from various angles and heights.

the extracted dragon photographs

This is what you should aim for when capturing photos. A 360 degree covering of photos from various angles and heights ensures you get the details you need.

When taking pictures, move around the object to get views from various angles. Try to move in the smallest possible increments and take photos both closeup and from a bit further away. You’re aiming to get close up details and photos that show the entire model.

Don’t move the model or turn it during your capture session as this tends to throw some software off. Instead, move around the subject, while leaving it stationary.

Camera Settings

To get a uniform looking set of photos, set your camera to manual or pro mode. Most smartphone cameras let you enter this mode, while hand-held DSLR cameras always have a manual or M mode.

  • First, set a Fixed ISO mode. This changes how sensitive your digital camera’s sensor is to light. Higher numbers make it easier to capture photos in darker conditions but result in noisier images. Outdoors in good lighting, you can usually keep this down to 100 or 200. For photoscan images, always choose a fixed number.
  • If you’re new to photography, use a fixed shutter or fixed aperture mode on your camera. These are sometimes called Priority modes. Doing so helps you pin down that setting and lets the camera control the other settings to maintain an even exposure. Again, this is all in the name of getting well-lit photos.
  • When taking your pictures, always make sure they’re in focus. You don’t want any blurry images.
  • Keep your pictures in one orientation, either landscape or portrait, throughout.

setting manual ISO mode on your camera

Setting the ISO to a fixed 200 worked well for middle of the day, overcast lighting in this situation.

Creating Photoscans With Meshroom

Meshroom is an open-source photogrammetry app you can use to create 3D models from photographs. You’ll use it to create a photoscan of the dragon sculpture.

Download the latest version here and install it. You need either Windows or Linux to run the software.

Upon running Meshroom, you’ll see the main interface. You can do everything you need to create your photoscan in this view.

Here’s a quick run-through of each section of the main interface:

the main meshroom UI view

  1. This is where you drag and drop the photos you want to recreate in 3D.
  2. The Image Viewer is simply a preview pane for individual photos.
  3. Once the process begins, the 3D Viewer shows you the results of the reconstruction of your object. It also shows you the different angles your photos were taken at based on metadata the program gets from your images.
  4. Graph Editor shows the pipeline of processes the app runs through to reconstruct your object in 3D. Each stage is represented by a Node. These are a series of inputs and outputs with processing that occurs at each stage. Your photos run through this pipeline, with the result of each stage feeding into the next.
  5. In the Node pane you can access details about individual nodes in the Graph Editor. You can tweak each node’s settings, but for this tutorial, the default settings are enough.

Importing Photos to Meshroom

Now, locate the downloaded dragon sculpture photos. Then drag and drop them into the Images pane in Meshroom.

The photos appear in the Images pane in filename order.

Import photos into Meshroom

Next, click File ► Save As… and choose a new folder location somewhere on your drive and save the project as Dragon.mg.

Saving the meshroom project

Now, close all non-essential apps in preparation for the Meshroom reconstruction process. The scan won’t start if your computer is low on resources.

Beginning the Meshroom Reconstruction Process

At this point, you’re ready to begin reconstruction. As Captain Picard would say, make it so! Click the Start button to begin the process.

Beginning the Meshroom re-construction process

While the process is running, you can track its progress in the Graph Editor view. Notice a small colored progress bar on each node. Your CPU will also most likely max out for 15 to 30 minutes, depending on its speed.

CPU intensive process

On completion, you’ll see the resulting scan in the 3D Viewer pane.

the scanned dragon result in Meshroom

Now Left-click and drag your mouse cursor to rotate, or Middle-click and drag to pan the 3D view around. Bask in the 3D glory your CPU cycles have helped create!

3D viewer in Meshroom

Next, open the folder where you saved your Meshroom project file. You’ll notice a folder called MeshroomCache. This is where the scanned object files were output.

locating the important meshroom exported files

Importing to Blender

Next, you’ll use Blender, a popular 3D computer graphics application, to import the dragon 3D model scan and clean it up in preparation for import to Unity.

If you don’t have Blender version 2.80 or higher installed, download and install it now. You can download it here.

After the installation has finished, launch Blender.

If you’re new to Blender, take a deep breath in and relax! While the interface and controls can be confusing for beginners, you’ll find many key combinations and screenshots to help you here.

relax and breathe

With the Blender launch screen open, click General under New File. The new file contains a cube object by default. Click the Cube to select it, then press Delete to get rid of it. Now you have a clean space to work in:

The default new blender project view

Next, click File ► Import ► Wavefront (.obj). Navigate to the MeshroomCache folder, where Meshroom saved your scanned object model files. Under this folder, find the model in the sub-folder MeshroomCache/Texturing/GUID. GUID is a folder name with a random ID assigned when your scan finished.

Then, double click texturedMesh.obj to import the scan to Blender.

Importing the meshroom scanned model into Blender

The import might take a while because the photoscanned model file is quite large.

View of the dragon scanned model in Blender by default

Start by getting a feel for how the photoscanned dragon looks by default. Click the Viewport Shading button in the main Blender viewport to enable full Viewport Shading and texturing.

enable viewport shading

Next, hold Shift + Middle Mouse and drag the viewport to pan the main dragon model to the middle of your screen. Then hold Middle Mouse and drag to rotate the viewport. You can zoom in and out by scrolling the mouse.

Look at how good the scan looks by default. Great job! :]

The viewport shaded dragon model in Blender

Touching Up With Blender

The model is complex with high vertex count and detail. Next, you’ll reduce the polygon count with a quick and effective Blender modifier called Decimate.

For commercially released games that need more optimization, you might want to spend more time reducing polygon count. For now, Decimate does a good enough job.

Using the Scene Collection panel in Blender, click texturedMesh ► Modifiers ► Add Modifier.

adding a modifier in Blender

From the selection, choose Decimate.

decimate the mesh in Blender

A small configuration area appears with the Decimate options.

Now, choose Collapse with a Ratio of 0.1, and Factor of 2.0. Then click Apply.

This reduces the detail in the entire model and reduces the vertex count by a factor of about 10! You shouldn’t notice too much visual loss in detail, and your Unity profiler and game performance will love you for this!

your CPU and profiler will love you for optimizing the mesh

Removing the Grass

Now to tackle the elephant in the room.

If you’re planning to create a scene where a dragon sits in the middle of a playground or pristine section of green grass, then you’re already set. But you going to place this particular dragon in a Viking village setting, so this grass won’t do.

Now, you’ll focus on removing the grass from the mesh.

In Blender, change from Object Mode to Edit Mode. You can use the UI to do this, or press Tab on the keyboard.

Notice your viewport changes to show all the dragon’s vertices selected.

mesh vertices selected

Selecting the Vertices

Next, you’ll select only the vertices for the area of grass around the dragon’s base and then delete them.

Be warned: The next step requires a fair bit of patience.

First, select the Select Lasso tool by pressing Shift + Space, L. That’s Shift and Space together and then L.

Or, use the UI tool bar to choose the Select Lasso tool.

the lasso selection tool

Next, use a combination of Middle Mouse click and drag to rotate and Middle Mouse + Shift click and drag to pan t around to view the model from all angles.

From each angle, Lasoo selects vertices of the areas of grass and carefully around the base of the dragon. Between each selection, press Shift when selecting new areas to ensure they’re added to your existing vertex selection.

Here’s a GIF that illustrates this process:

Selected vertices around the dragon base to delete the grass and unwanted areas

You need to rotate the view occsasionally to also select vertices that are behind bumps and ridges in the grassy mesh.

If you select some vertices that you don’t want to delete, Shift + Left Click them individually to remove them from your selection.

Once you have all grass and surrounding vertices selected, press Delete ► Vertices to delete them.

all the unwanted vertices selected

delete the vertices

Have a good look around the remaining mesh, zooming in and out and rotating the view. Repeat the select and delete process again for any vertices that you missed.

You’ll get something resembling this menacing fellow once done:

after removing the unwanted vertices from the dragon model

Closing the Base of the Mesh

Now, go into Edit Mode, if you are not already there. Selecting Tab switches modes. Then, select the entire bottom edge of vertices on the dragon mesh using the Select tool and holding down Shift. This takes some patience.

the select tool in blender

Or, you can use the Select Lasoo tool for this.

bottom edge of vertices selected

With the vertices selected, click Vertex Smooth Vertices.

smooth out the vertices

This smooths the ring of vertices you’ve selected a bit.

Now, select the 3D Cursor tool by clicking Shift + Space, Space and position it in line with the lowest vertex in the bottom edge selection of vertices using Left Click. You can pan and rotate your view to get a side-on view of the dragon to determine the lowest vertex.

Next, press the period, ., key to the Pivot Point and then choose 3D Cursor to use the 3D Cursor’s position as the pivot point.

Then, press S, Z, 0, Enter. That’s a zero, not an O. This series of commands scales all the selected vertices to the same height as the pivot point you created with the 3D Cursor.

Now you’ll have a mesh that resembles this:

scaling the vertices down to create a base

Finally, click Vertex ► New Edge/Face from Vertices.

This completes the new base for your dragon by capping the selected vertices and creating a new face.

new edge face from vertices in blender

side view of the scaled vertices creating the new base for the dragon

Repositioning the Dragon

Next, change back to Object Mode by clicking Tab and Select the texturedMesh object. Position the dragon using the Move tool by clicking Shift + Space, G to the origin point of the scene, where the green and red lines cross. Then use the Rotate tool by clicking Shift + Space Bar, R to rotate it into an upright position.

Now, select the 3D Cursor tool and click it approximately in the center of the dragon on its back. Press F3, then type Set Origin and press Enter. Next choose Origin to 3D Cursor from the new list of commands that appears.

A yellow dot will appear in the middle of the 3D cursor to signify the new origin point of the model.

repositioning and setting the dragon origin in blender

You’re ready to export!

Next, click File ► Export ► Wavefront (.obj).

Make sure to select Path Mode ► Copy and check the Selection Only option in the Export Settings. Save the file as Dragon.obj in a convenient location.

blender export options

A Path Mode of Copy for the export ensures the textures are copied and referenced from the same folder the .obj model file is. This helps prevent missing texture issues once imported into Unity.

Bringing it all Into Unity

Now, open the Starter project in Unity, then open the Assets/RW/Scenes/PhotoscanDragon scene. Locate the empty pedestal between the rock formations in the village. This is where you’ll position your photoscanned dragon model.

the dragon village unity scene

Now, drag and drop the exported model and associated export files into the Assets/RW/Models/Dragon project folder.

import the dragon model into Unity

Then, use the Unity Hierarchy window to locate the FocalPoint GameObject. Drag and drop the Assets/RW/Models/Dragon/Dragon.obj model into the Scene, and child it to the FocalPoint GameObject.

adding the dragon to the Unity scene

Now,change the position and rotation of the Dragon Transform Component as follows:

  • Position: (X:-33.32, Y:0.53, Z:5.08)
  • Rotation: (X:-2.229, Y:90, Z:-0.6)

The positioned and rotated Dragon model

Note: Your position and rotation may be slightly different, depending on where you clicked when setting the pivot point in blender.

Then, click Play and marvel at your very own photoscanned dragon, taken from the real world, and re-constructed in full 3D. :]

the final dragon model in Unity next to the torches and rocks

Where to Go From Here?

You can download the completed project using the Download Materials button at the top or bottom of this tutorial.

You’ve covered many steps to get here, and in the process, learned about Photogrammetry, editing 3D meshes in Blender, Vertex operations and bringing that all into Unity!

I’d like to add a special mention and thanks to The Tales Factory on the Unity Asset Store for letting me use their Photoscanned MountainsRocks PBR asset in the Dragon Viking Village scene in the included project files here.

Go check out their excellent photoscanned assets.

The next step is up to you. What would you like to re-construct and bring into your Unity projects? Here are some resources and further reading that might help you along the way:

Please share anything you capture and create on the forums for this article with the link below! :]

Average Rating

5/5

Add a rating for this content

2 ratings

More like this

Contributors

Comments