Written Guide
Last updated: Mar 2024
The FaceBuilder add-on for Blender lets you create photorealistic 3D portraits using only photos or images right inside the popular 3D creation suite. The combination of AI-powered mesh alignment and advanced settings make FaceBuilder a professional image-to-3D tool for 3D artists and CG enthusiasts requiring no special 3D modelling skills. The resulting animatable 3D heads and faces can be used for digital makeup, 3D tracking, creating 3D characters or avatars in Blender or exported to any other 3D software.
Installation
Download KeenTools Blender Pack from our download page. Open Blender, go to Edit > Preferences > Add-ons, press the Install… button, navigate to the downloaded .ZIP file (for Mac users: double click on the DMG file and find the zip inside) and hit Install Add-on.
Select the Interface: KeenTools FaceBuilder… checkbox to enable the add-on, unfold it, accept the licence agreement, and finally press Install online to download and install KeenTools Core Library necessary for the add-on performance.
For offline installation, you’ll need to have our Core Library previously downloaded from this page. Simply click on the Install from disk button and navigate to the Core Library zip file. Note that the versions of the add-on and the core library should match.
Like other KeenTools add-ons for Blender, FaceBuilder can be accessed through the N-panel (sidebar) by pressing Nor clicking on a tiny little triangle in the upper right corner of the viewport.
Once you press Create new Head, FaceBuilder adds a default 3D head to the scene. This is a neutral head that you will be morphing using reference images.
Load Images
Start by pressingAdd Images and load reference photos. New cameras will be added automatically to your scene according to the number of views. You can load multiple images at once.
When building a 3D head from a single photo, consider using a ¾ view as it contains more of the relevant three-dimensional information compared to full face or profile views.
Though you may use photos of different sizes and shot with different cameras, painted or AI-generated as well as video snapshots, we recommend using a set of images showing the head from different angles and sharing the same settings and lighting in order to create a perfectly accurate 3D head with good-looking texture. The preferred views are:
– Full face
– Left profile
– Right profile
– ¾ Left
– ¾ Right
– Half-face back side left
– Half-face back side right
– Low-angle view
– High-angle view
Note that when solving, lens distortion is not taken into account. In other words, FaceBuilder treats images as if there’s no lens distortion in them at all. So if you build a 3D head from wide-angle closeup shots, consider undistorting images first, using the Undistortion node or other preferred method.
Camera settings and EXIF
FaceBuilder sets up cameras automatically using the data stored in the image EXIFs. In case image metadata is empty, FaceBuilder will attempt to estimate focal length automatically. You can adjust the focal length manually in case you know the right value. For manual adjustment go to the Views tab > Options, deactivate Estimate focal Length, and type in the new value below. Make sure you use a 35mm equivalent focal length value.
Facial expressions
Before you start aligning mesh with the images, it’s important to decide whether or not you want FaceBuilder to capture facial expressions. By default, FaceBuilder assumes the person’s face on the images has a neutral expression, and so the resulting 3D head will also be neutral. If there’s a smile on the face, raised eyebrow, neck turn, etc., consider turning Allow facial expressions on, which makes the blue parts of the mesh and neck more flexible and easier to align. Note that once activated, this option applies to all views. There may be different emotions in different photos, and you can specify which view you want to use as the expression source for the resulting 3D model or keep it neutral in the Model tab.
Once you check Allow facial expressions, there will appear a couple of new options below: Lock eyelids and Lock neck. When these options are selected, FaceBuilder expects that the eyes are completely open and the neck is not tilted or turned.
Align mesh to face
The loaded images appear as view buttons with filenames on them in the Views tab. When you click on a view you’ll see the image and the face mesh on top of it in the viewport. If it’s the only view you loaded, FaceBuilder will attempt to detect the face automatically and make rough estimation of its 3D position and shape.
Start with pressing the Auto Align button. The built-in AI will match the facial features of the mesh with the face in the image and place the related camera in the 3D space based on those estimations.
In case there are multiple faces in the frame, you’ll be offered to select the one you want to turn into 3D.
You will notice red dots on the mesh. They are called Pins. You can adjust the parts of the mesh that are off by dragging them to the corresponding positions. However, the pins that were set automatically are not mandatory. Those are just suggested reference points and you are highly encouraged to delete (right click) those pins whose position is not immediately apparent. You can also create new pins by left-clicking anywhere on the mesh to make it match more accurately with the image. Use Tab to turn mesh on and off for better control over the alignment process.
Don’t try to match the mesh with the face perfectly in a single view. Keep only those pins whose position you are sure of.
When FaceBuilder AI can’t detect enough facial features for auto-alignment (e.g. back views) you’ll need to locate the mesh manually. Rotate the head left or right with the 45-degree buttons to get close to the needed angle. Then create pins and drag them to match the most prominent parts (ears, top of the head, nose tip, if visible). Note that the first 3 pins set the model’s position and rotation in 3D. From the 4th pin and on you already start morphing the mesh.
If you need to undo all your mesh adjustments in the current view and start everything from scratch, just press the Reset View button. This will remove all pins and reset the camera position. You can also rotate the image, remove it from the stack or replace it with a different image by clicking the hamburger icon next to it.
Appearance
If the mesh wireframe is too bright or fades with the background, you can adjust that in the Appearance tab. You can change the wireframe opacity, switch between different colour schemes or customise the colour of base mesh, facial features and midlines. Alternatively, you can adjust exposure and gamma of the background image in the current view. Here you can also adjust pins size and sensitivity.
How to get it right
In order to build a precise 3D head we suggest working your mesh in a number of rounds. Make rough alignment in the first view, then do the same in the other views. Keep the number of pins as low as possible in the first round. Then go back to the first view and adjust all main facial features more precisely by creating and dragging new pins. Repeat that in the other views. Go for a third round to focus on tiny curvatures and bumps on the face to fine-tune your 3D head.
When you create pins and drag them in one of the views, the mesh gets automatically adjusted in other views. Think of it as if you’re actually sculpting your 3D model using pins instead of brush tools.
Model settings
Go to the Model tab to turn different parts of the 3D head on and off or instantly switch between Highd, Midand Low topology. Here you can also change the absolute size of your 3D model by changing its Scale. The default Scale value is 1, which makes the head 2.18 m high. If you need your 3D head to be close to the real-life size, set the head scale to 0.1, which will make its hight around 21 cm.
You can also adjust Mesh rigidity, i.e. the flexibility of your model as you sculpt it. Go to Views > Options > Mesh rigidity and change the Shape value to adjust overall shape flexibility: 0 = most flexible and relaxed, 10 = most rigid and firm, 1 = default. When Facial expressions are on, you can also adjust the flexibility of Expressions in general and Eyelids and Neck in particular.
Texturing
FaceBuilder lets you create head texture with just one click. Simply press the Create Texture button in the Texture tab. FaceBuilder creates texture blended from the pinned views using the selected UV layout, and bakes it onto the 3D model. You can customise texture settings by pressing the gear icon next to it: select and deselect source images and set output texture size.
Advanced texture settings may help you fill the gaps in the texture if the source views don’t provide enough coverage. Angle strictness lets you adjust the blending of colours between different cameras: 0 = average colour from all views, 100 = colour from 90 degree views only. The best result is usually achieved within the range 10–20. Expand Edges lets you extend texture on the edges using neighbouring colour. Equalise brigtness and Equilize colour are experimental options that may help get more consistent texture colour in case of lighting issues in different photos. Autofill is another experimental option that automatically fills the gaps with the generated texture.
Available UV maps are: butterfly, legacy, spherical, maxface, and mh. Butterfly is aimed at reducing distortions with the fewest possible number of seams. Legacy UV has even less distortions but contains more seams. Maxface gives you the highest resolution possible. Spherical is a slightly improved version of the popular ‘cylindrical’ UV. MH creates texture with the MetaHuman-compatible UV map.
After the texture is created, you get access to Export andClear buttons that let you process it outside Blender or remove it from the scene.
FACS blendshapes
You can animate your 3D head with 51 built-in ARKit-compatible FACS blendshapes. Press the Create button in the Blendshapes tab to generate shape keys. New buttons will appear below to give you more control over the created blendshapes. Delete removes the shape keys and unlinks the animation. Reset value zeros out all shape key values in the current frame. Note that this will not alter the animation nor will it create new keyframes. If you want to save the state in the current frame, you’ll need to do it manually.
You can now animate the blendshapes from the Shape keys tab in Object Data Properties. The shape key editor in the Animate panel gives you control over the keyframes.
FaceBuilder also lets you import pre-recorded animation as a CSV file containing ARKit-compatible values. Just press Load CSV and navigate to the file. The loaded animation will be added starting from the current frame. By loading more CSV files you can extend your animation sequence.
One of the facial animation software supporting this format is Live Link Face app available only for iOS devices equipped with the TrueDepth camera like iPhone X and newer.
You can also export facial animation as a CSV file from our FaceTracker node for Nuke.
Project management, Save & Export
You can save your project anytime in the middle of the face-building process and come back to it at a later time.
FaceBuilder does not store images in the project file or folder. So if you plan to transfer your project to a different computer, make sure you add the photos. It’s also wise to keep the photos in the project folder as Blender uses relevant file paths.
Once you create a texture, it’s stored inside of the project file, which may make it quite heavy depending on the texture output size. You can delete it from the project file by pressing the Clear button in the Texture tab. You can also export it as a separate file with the Export button.
In order to export your 3D head, you need to make sure it’s selected in the viewport or Scene Collection, then go to File > Export, choose the file type, and save the model. We recommend using Wavefront (.obj) or Alembic (.abc) to prevent important data loss. Both formats let you export texture along with the model. If you choose the Wavefront format, the texture files (.mtl and .png) will be saved next to the model file, while the Collada file will have everything embedded in the model.
Use the FBX for games button in the Export tab to save your 3D model along with all the blendshapes and animation. It is pre-configured for importing into Unreal Engine and Unity, though you may select different export options in the pop-up export window.
Press Export to CC4 (Windows only) to launch Reallusion Character Creator 4 and transfer your head mesh and texture to CC4 in order to create look-alike 3D characters. This workflow also requires the Headshot 2 plugin installed to your computer. Read the full guide here.
Links
Download FaceBuilder for Blender from our site.
Watch the official FaceBuilder video tutorial.
Follow us: Facebook, Twitter, Instagram, YouTube
