How to Replace a Face in a Photograph using Neural Networks

Updated: June 6, 2024 Author: Vitaly

Replacing a face in a photo is a fun and creative process that brings a lot of joy. There are a lot of possibilities, from creating funny memes to restoring damaged photos. In this article, we will find out how to easily replace a face on a photo using the tools available and programs.

Deepfake is a neural network technology that can easily replace a person’s face in a photo making it barely impossible to distinguish it from a real photograph. Just a few years ago, this technical solution was mainly used by attackers to compromise a person, but now this opportunity is available to everyone.

With simple manipulations you can create such images for fun or use them to create better photographs of yourself. For example, if you want to create a business social network profile, but you don’t have time to take a high-quality photo in a suit, so you can take a suitable picture from the Internet and insert your face into it.

Replacing a face in a photo is a powerful tool that can be used for both creative and practical purposes. However, to obtain a high-quality result, you need the appropriate tools and the ability to work with them.

What tools exist to replace a person's face in a photo?

With the development of neural network technologies, high-quality replacement of a face in a photograph is no longer difficult. Today, graphic editors and neural networks are available to users.

Until recently, programs such as Photoshop or GIMP were the only way to replace a person's face in a photo. Often, by the term “Photoshop” people mean a photograph artificially modified in the editor, but not the program itself You've probably heard “it's Photoshop!” However, for a person without design skills, these tools are too complex, and the process of replacing a face is very labor-intensive. Therefore, they are worth mentioning as an option, but in most cases, image editors are not suitable for basic users. Of course, the introduction of neural network technologies by Adobe in Photoshop and OpenVino from Intel in GIMP makes working with a graphics editor easier, but using these tools fully requires special skills.

Neural networks offer the most advanced way to change faces in photos. They can create incredibly realistic images and even generate new faces. However, access to these neural networks may be limited or paid, and using local versions requires significant computing power.

Each of these tools provides different solutions with different functionality. To make your choice easier, in this article we have collected the best options for programs, mobile applications and services based on AI.

Midjourney

Midjourney is a popular neural network model designed for generating and editing images. This service has enormous capabilities to create graphics, photographs and designs professionally. At the same time, you do not need to study Photoshop or other graphic editors for months; you just need to correctly formulate a request to the neural network through a bot in Discord.

The presented neural network replaces a person’s face in a photograph so well that it seems as a real photograph. The program n retains all the lighting, shadows and other details from the original image. But to perform such manipulations you need to have a good understanding of Midjourney commands and scripts.

To make the neural network replace people's faces in photographs, you need to connect an additional bot InsightFaceSwap. It's free, but you must have a Midjourney subscription to use it.

Then you need to upload your photo and assign it a unique ID name. To do this, enter the /saveid command in the InsightFaceSwap message line and enter any memorable name for the sample. Then, an area where you can add a photo will show up.

Now you can use the uploaded photo of a person's face both as a replacement in the generated images and in other photographs. For the last option, enter the command /swapid in the message line of the InsightFaceSwap bot and add an advising snapshot. The ID name contains the name of the identifier that you provided when uploading a photo of a person’s face.

To make the InsightFaceSwap and Midjourney work conjunction more clear, let’s look at how the neural network coped with replacing the face of Jim Hendrix with Barack Obama.

Of course, there is no limit to perfection, but for personal use the neural network provides good results.

Stable Diffusion

Stable Diffusion is also a neural network model, but open source. Although anyone can edit the code, this does not mean that using Stable Diffusion is free. To gain access to the latest versions of the model, you need to pay.

The minimum top-up amount is $10, which provides 1000 credits. It turns out that for this money you can generate about 150 images using the latest version of SD3.

As an alternative, you can use the services of third-party developers who use the Stable Diffusion neural network as the basis for their services, for example, stablediffusionweb.com. For the same $10, you can generate or edit up to 2,000 images. Recently, the company developing the stablediffusionweb.com project launched a service for changing a person’s face in a photo together with the Stable Diffusion neural network at https://faceswap.so.

The service provides 10 free attempts to change a face in a photo, so you don’t need to register.

The user first uploads the photo where the person needs to be replaced, and then the character itself. For example, let's try to replace the face of Leonid Brezhnev with Barack Obama.

After a short wait, the service produced the following result.

Here's a replacement of the baby's face with Obama's. As in the previous version, it is noticeable that the head is out of place.

Despite some errors, the developers of faceswap.so managed to get good results using the Stable Diffusion neural network.

Adobe Firefly Photoshop

Despite Adobe's efforts, their Firefly neural network, which is built into Photoshop, is still inferior to Midjourney and Stable Diffusion in terms of the realism due to their longer development. However, it can already boast of some results. Note, that you should have basic skills to use the Lasso tool, so there is some manual work you have to do.
There are two options for replacing a face in Firefly: inserting a person into the generated space and changing it on an existing photo. In the first case, you need to select the person, and then invert it to the entire photo. After this, just enter the prompt in the dialog line.

To replace a person in an existing photo, you need to take a face and superimpose it on the photo. Next, use the Lasso tool to indicate the areas the neural network should work with and start combining. In this case, there is no need to register any additional information.

The presented option for replacing a face in Photoshop is more suitable for users with at least basic skills in graphics or design. It is not suitable for one-time use, because you need to register on Adobe Cloud, subscribe or use the trial period and install the software to get started.

Artguru.ai

Another online service for replacing a person’s face in a photograph. It is difficult to understand which neural network model Artguru.ai uses, but the way you work with the program is similar to the previous example.

Artguru.ai is a shareware service. The user is given only 8 attempts to replace a person's face in a photo.

Faceswap

Faceswap is open source software based on Google's Tensorflow neural network model. The program was developed by enthusiasts for creating deepfake videos. However, the presented solution can also be used to replace a person’s face in photographs.

A distinctive feature of Faceswap is that the user will have to independently train the neural network to search and identify faces. Here you can’t take two photographs and replace the face on one of them. To make the neural network perform correctly, you should upload at least 25 variants of the donor’s photo and the same number of images as the basis fordeepfake. The process of identifying, training and converting images requires significant computer resources and takes a lot of time.

However, the ability to batch process photos and a large number of options make Faceswap a good choice when you need to process a large number of photos.

How to find the perfect face to replace in a photo

Although neural networks have facilitated the process of replacing a face in a photograph, it is still difficult to find the very donor photo. To make it easier, all photographs stored on the computer must be well cataloged. However, it is difficult to maintain order when the photo archive contains pictures from different phones, cameras, and photos sent by friends via social networks and instant messengers.

The solution is the Tonfotos program. This is a special software that collects all the photos on your computer into a single database and sorts them in chronological order, by location or pre-created albums. There is a function for identifying and recognizing faces, which can help you quickly find a suitable photo to create deepfakes. To do this, you only need to specify the person’s name in the photo once, and then the program will automatically find the rest of the photos on the computer with this person.

Summary

Neural networks have reached a high level in creating and processing images, including replacing people's faces in photographs. Today you can get a truly high-quality and realistic result that will satisfy the majority of household consumers. Besides, professionals can make the work easier and faster, because all that remains is to slightly correct the resulting image.