Surely in the last few weeks you have seen portraits of both celebrities and contacts you have on Facebook and WhatsApp, with different styles, some imitating lions and others comic characters, but all with a very successful resemblance. The surprise is that these images are not painted or made by human beings, but rather an artificial intelligence (AI) that, in the style of Diego Velázquez, has immortalized these people.
There are different applications that, with more or less skill, are capable of converting a descriptive text into an image, just as dalle-2, Open AI either midjourney. At first it can be a little difficult and requires some practice to achieve decent results, however, that of portraits is a genre that gets stuck.
However, there is a way to train these artificial intelligences to recognize a person’s traits so that they can be applied to different artistic styles, settings, or colors. This requires a good number of steps, however, from PIXEL we have prepared a tutorial to follow step by step, with which you can master this technique.
It should be noted that the entire process could be carried out locally, that is, with the graphic performance of our own computer, but taking into account that there is a 100% online option, which is accessible for any type of computer, Whatever your specifications, we will guide you from that option.
First of all, collect between 19 and 25 photographs of the person you want to generate portraits of, yourself, a friend, family member, someone famous. You can also make them from a pet. Try to take photos from different angles, showing the face from the side, profile, tilted, some up close, others from the hips up, and some full-length.
These photographs will serve to train the AI, the more varied they are, the better to identify the features. Crop each photo to fit 512×512 pixels. If you don’t have a photo editor, you can do it from this website. Press crop and then select Freely, in the first two boxes write 512. If the crop is too small and does not fit the face, go to resize first and in the blue circle where it says 100, slide to the left to make the photo smaller . Then repeat the step of cropping to 512 size again.
Then access to this Google Colab linka Google Research tool designed to execute code, already programmed, automatically that, in addition, It will work with a remote virtual machine (with its own graphics), provided by Google for free. That is why it does not matter from what type of computer you access it and its graphic capacity. This particular Google Colab has been adapted by the Dot CSV content creator, which has added some very useful and intuitive features to it.
Check that in Execution environment (top, next to File and Edit), pressing Change execution environment type, GPU is checked where it says Hardware acceleration. Before where it says Step 1, under the red siren icon, where it says !nvidia-smi, press play and press Run anyway. Just below it will appear the type of GPU that has been assigned to you.
1. Go to Step 1 and press the Play button again to execute and a message will be output for connect with google drive, since Google Colab needs your storage in the company’s cloud. Press Connect with Google Drive, select your Google account and grant it the permissions it asks for, which is basically to use 4gb of storage
available. This is where the artificial intelligence will be stored once you have trained it.
2. Go to Step 2, display the cell clicking on the box and installing (pressing play) each one of the libraries necessary to make the AI work; now wait about a minute for them to finish installing. You will know when each of the two bookcases has a green check in front of it.
3. Go to Step 3, which is to download the Stable Diffusion checkpoint model (the AI engine), which you will retrain with the photographs. The first thing you have to do is access the link that comes in step 3 itself, that of hugginfacea website that contains a large number of AI models that can be downloaded.
First, create an account by clicking Sign Up at the top right (it’s completely free). When you have already logged in with the user you have registered, go back to the main page, the one with the hugginface link and then a menu will appear that you must accept by clicking Agree, thus accepting the terms of use of the model.
Then press the circle at the top right and select Settings and click Access Tokens and click New Token. Enter an identifying name, for example ‘Image Creator’ and under Role select Write. In the box that will have appeared on the screen you will see some points as if it were a long password, hit copy (by selecting or pressing the two overlapping squares to the right of the box itself) and return to the Google Colab tab.
In Step 3 where it says Huggingface_Token: paste the code that you just copied from the web and press Play. In this way you will have linked Google with the web from where Stable Diffusion is automatically downloaded. Wait until at the end of Step 3, green says Done.
4. Step 4 is one of the most important as it is where you are going to set up the AI training. In Training_Subject: select what you want to train on. That is, select what appears in your photographs, a character (Character) or an object (Object). Since we have focused on portraits, select Character.
Where it says With_Prior_Preservation: leave it set to Yes. This setting is telling the AI what face we want to fit on the models, but at the same time we are telling it not to forget the features of those same models. What will give better results, mixing styles and themes.
In SUBJECT_TYPE: Write the character that appears in the photos with which you are going to train the AI. Man if it is a man, Woman if it is a woman, Dog if it is a dog. You can also leave it in person to leave it in a human being in general, regardless if he is a man, woman, girl, boy, etc.
In INSTANCE_NAME: you are going to write the name that identifies the concept with which we want to train. Remember not to use words or names of concepts that exist, that is, do not put Raquel, Juan, plate, car or Otto (you can mess with the AI), rather use acronyms or an anagram, a combination of letters that is short, identifying and above all unique. Leave the rest of the options as they are and press Play in Step 4.
When you press Play, a menu will appear to choose files. This is the step in which you will upload the photos with which the AI will train. Press choose File and select the 15/25 images that you have prepared.
5. Now we go to Step 5. This consists of a parallel training of the same concept to which the photos belong. That is, if we are going to make portraits of a person, thanks to this cell, the AI train in parallel with photos of other people to be able to polish the portraits. For them where it says Dataset: select the same thing you wrote in step 4 (person/woman/man/dog, whatever you put) and hit Play.
6. Go to Step 6, leave the options as they are and hit Play. Then some letters will appear in which TRAINING can be read. This training process can last between 40 minutes and an hour. When the training is finished, the following message will appear at the end of the cell: DONE, the CKPT model is in your Gdrive.
7. In Step 7, click play to run an interface to use Stable Diffusion on Google Colab, to be able to make portraits with artificial intelligence. When you have loaded, a link appears in Running on local URLpress it and Click to Continue to enter the interface.
In the interface, the box at the top is where you must enter the descriptive text to tell the AI what type of portrait you want. For example: a photograph of token name. The token name is the anagram you wrote in step 4, in INSTANCE_NAME:. Then hit Generate.
To refine the results, the AI will require you to modify and try different parameters. For example, where it says Sampling Steps, raise the bar up to 50 to achieve a good level of detail. To save the generated photo, under the photo, press Save and in the box below, click Download.
Enter to this website to find combinations/text descriptions to generate portraits. Click on the photo you like the most and copy the text, paste it in the AI box and where it puts concept of/portrait of or the name of a celebrity, write the name of the token. In this way you will achieve portraits of all kinds of styles, baroque, futuristic, comic, realistic, there are millions of combinations.