Month: November 2022

Based on my experience with the last test project, I became more comfortable with the final production, but I still encountered a lot of difficult problems.I won’t go into detail about the parts that have been described before, but here I will only describe the parts that are different from the previous part.

Because the patterns to be made this time is not just an even division of the face into four parts like in the test project, I I need to have the uv of the face to know exactly where my patterns are distributed on the face to help me draw the patterns more easily. So I went to the SparkAR website and downloaded this face uv.

I then put this uv image into Photoshop and drew the pattern on it according to the size of the face to get these masks.

The next steps are the same as in the test project, the only difference being that I have replaced the black NO.0 material with a texture.

I used Midjourney AI to create this texture, a portrait of Cyborg.

Then, again, I put this picture into Photoshop to make it the same size as the UV map.

Finally, the most exciting moment of all, I added these images to SparkAR and created the effect of a flipped face by moving and rotating the face meshes.

But before that, I ran into a few more tricky problems.

I found that when one face mesh was covered over another, the face mesh underneath would be blocked, and while I was thinking about how to make this face mesh only non-transparent part mode, a feature came to my rescue, the Alph test(A contril that discards pixels with opecity less than the cutoff threshold), which solved this problem very well.

After that when I turned to the back of the flipped face it became transparent, then I found that when I turned on double-sided rendering it copied the texture from the front to the back. This is a good solution to the problem of transparency on the back.

After solving these problems, here is the final result

Before making the final AR avatar, I made a test version. Because I wanted to try out the effect I envisaged first to see if it would work. With what I learned in class on how to make the 3d glasses model track to the face and some tutorials I learned on the YouTube I made the following test version of the split face.

First, I create a face tracker, then add five face meshes 01234 underneath them, at the same time create five materials 01234, then assign the materials to each face meshes by number, then set the material map for 1234 to face tracking and the material for 0 to black because I like to create a sense that the original face is missing.

After that I drew four png layers in photoshop that divided a canvas into four parts（The black parts are transparent layers）

Then I put each of the four png images into the alpha setting（a control that masks the alpha chennel of texture） of the 1234 material, so that the face is divided equally into four parts.

After that, I moved the four face meshes to the position I wanted, by moving and rotating them (the rotation in the SparkAR is very difficult to use I don’t know how to set the coordinate system to the position I want instead of the world or object coordinate,and in blender there is this very good function for moving cursors)

Final result