I started this blog with the idea of showing people some of the tricks and processes I use in Smoke and Flame. I just learned how to do this Auto Stabilize trick and it is quite the game change and is one of the coolest workflows I have ever done. I had a 650 frame shot of two people standing in front of a monitor. The camera was handheld and moved all over. Up down, back, perspective change, size change, jitter in short it was a real nightmare and I had to track stuff in the monitor. To make matters worse, the talent blocked the lower left and the lower right hand side of the frame and I could not get a track. I tried in Mocha for half a day and it would not track. I tried in Flame to track it for the rest of the day and it would not track. I used Auto Stabalize and got my footage locked onto the monitor in about 15 minutes! Of course I can not get the rights to the real shot, so my buddy Scott shot me in one of our finish rooms in front of a monitor. This shot replicates what I had to do and I can use it without getting sued. Here it goes. First, lets look at the shot. I have put a white border around the shot because it is so dark and we will need to see the edges to demonstrate this technique. It will make sense a bit later. Here goes. Lets look at the footage
This is a shot of your truly in our finish room. Note the hi tech Lustre Panel. It was shot on a 5d and is a pretty good match of the shot that I had to track.
Lets look at our batch setup.
I have drawn a blue box around the first part of the setup. All this is doing is drawing a white frame around the video. The reason is the video is quite dark and to understand this tip, you really need to see where the edges of the frame are. This will be apparent in a few steps.
The above still is the action setup with the white frame around the edge of the video.
The first thing we need to do is cut a matte out of what we want the auto stabalize node to track. After much trial and error, I discovered that my big noodle was throwing off the track. What I needed was to make a matte of the tracker crosses on the screen. I did this in a modular keyer. Below is a video of the matte that I cut.
As you can see from this video, I have drawn rough (real rough) gmasks around the area where the crosses are. I will now take this matte into the auto stabalize node. Lets look at the schematic. I am going to skip the Modular Keyer setup because this post is long enough. All I did was draw circles and track them to the crosses.
In the above schematic, I have taken the matte output of my modular keyer and stuck it into the Auto Stabalize mode. The fill is the output of action. Remember, this action is only slapping a white box onto the video to make it easier to see the edges. YOU DO NOT HAVE TO DO THIS. It will be clear in a few moments why I did this.
Now were are getting into the meat of this here tip. The object here is to use the Auto Stabalize mode to make the plate as smooth as possible. What we are trying to accomplish is getting the monitor to “lock” in the center of the frame. Lets look at the stabalize node.
Above we can see the auto stabalize node. I have selected the Use Matte option. (see video above to see what the matte looked like). I have also selected frame From to be 0. I have found that if you leave this at the default of 1, the first frame does not stabalize, so I always put it on zero. Here is one of the problems I have with this node. It tends to work a little too “automagically” for me and it is hard to see what is going on. I put Scene Flexability at 90% (after much trial and error) and then hit the Analyze button. The Flame then “does its thing” and you select the Result tab to see how it worked. In this case, I left all the Stabilization boxes in Lock and got a good result. However, during my trial and error process, I was having a real big problem with rotation going crazy. I turned the Lock box off on rotation to see if I could get it better. Lets take a few minutes to look at this alternative to Lock on because it may help you.
You can see with Lock Off the effect % comes on and so does smoothness. I found that I got a good result with smoothness on, and jacked way up to 10000. Beware, this attribute IS keyframeable. However, what really helped me out was making the masks of the crosses. This really smoothed out the plate. At the end of the post I will detail what I tried to get this to work, which I am sure is what most people want to know about this node. For now, lets get through the tip. Now I noticed that no matter how much I tweeked, there were a few frames that had a rotation jerk. I discovered that if I went to the offending frame I could go into the KF editor and move that one offending KF around so it would not jerk for one frame. Lets look
The KF that is hilighted is the offending KF. This is where my track shook rather badly. Remember, we are not looking for perfection here folks. Just get it close. I was able to isolate the KF and move it more “inline” with the rest of the KF. This really smoothed over my shake. Notice it is in the Analysis menu NOT the offset menu.
OK we have done all of this work and you are asking yourself, “WHY?” Lets have a look at the Auto Stabalize node again.
Here is the cool part. I promise. Notice what I have done to the Canvas resolution. I have made it huge. The image will now stabilize itself around the tv monitor on a huge canvas. Lets look at the output of this auto stabalize node. The clip below has a resolution of 4096×1521. The video of me is 1920×1080.
In the above video, you can see why I put a white frame around the video. What is going on here? The video has a resolution of 4096×1521 and there is a white box around it. There is also a white box around the 1920×1080 video clip of yours truly in the edit room. Notice how the entire HD clip seems to “move” around the monitor. What I have done is “Locked Down” the video monitor and the entire shot is moving around the monitor. Again, it is not a perfect track. It doesn’t have to be. Don’t you think this shot would be a lot easier to track something into that monitor now? Take a look at the original shot.
That monitor is jittering around everywhere. The tracked one is really locked down. Lets look at the auto stabalize node again.
In the above still, note the 2d transform. After I did the auto stabalzie analysis I then made the Canvas Size huge. The last thing I did was adjust the X and Y 2D transform tabs so my video raster stayed “locked” in the center better. I had a lot of trouble locking down the lower left hand corner of the TV so I maually set KF using position X and Y so the lower left hand side of the box was locked to the center of the frame. Watch the video of the output of the stabalizer again. The lower left corner is locked down quite nicely. Now lets look at my batch schematic.
I have taken the output of my auto stabalize node and stuck it into the background of action. I put a 1920×1080 fifty dollar bill into layer 1. We will be sticking this in the monitor. Finally I have also stuck the output of Auto Stabilize into layer 2 of action which is where I will do the chroma key. Lets look at action
Above we can see action. We want to make our action the same size as our oversized plate. In this case it is 4096×1521. However, who wants to type all that crap. Just select Node Setup and hit background resolution all the way at the top. This copies all the background info into this mode. Then hit apply and scale. Now your action is setup to output this huge frame. I next took the $50 dollar bill and did a 4 point bilinear track. This track was a LOT easier because my background was locked down. I was able to get a nice track pretty quickly. I then keyed out the green screen on another layer. Lets look.
Above is the action node. I have autostabalize32 in the background, layer 1 is the bilinear 50 dollar bill that is being tracked to the background and layer 2 is the autostabalize32 which is where I have done my key. BTW since the plate is so smooth now, it was a piece of cake to mask out the crosses. They barley moved. Now you are most likely thinking how are we going to get this shot back into 1920×1080 space. Here is how.
In batch, I simply copied the stabalize node and placed it at the end of the batch tree.
Lets look at the final Stabalize node
In the still above, I have simply selected Negate Stabilization. This will do the exact opposite of the first node, INCLUDING the offsets. I then selected Custom on the canvas and made set the output back to 1920×1080. I rendered it out and this is what I got.
PS As promised, these are the things I “discovered” during my tracking process.
1. Always start with frame 0, NOT frame 1 which is the default.
2. There is no way to invert a matte in this module. You have to do that in the keyer.
3. The first thing I play with is the Scene Flexibility percentage and then push analyze. 0 means it will lock onto one solid object. 100 means it will lock onto all moving objects. I would start at 50 and then go up and down by increments of 20 and see what worked the best. Then I would adjust the matte. Then I went in and adjusted KF.
4. Use mattes when ever possible, especially in a case like this. When I started, I simply did a chroma key of the monitor. The crosses were keyed out and my head was throwing off the trackers. The flame kept locking on to the edge of my head where the matte was. Since my head was moving differently that the monitor the track was bad. You can delete these tracker points by crtl drag and hitting delete and update, but it never seemed to work. Masking my head out all together was the best way to do it. Also, there was no texture on the green screen for the flame to lock onto. The flame locked onto the crosses very nicely which is why I ended up making a garbage mask around the crosses only and fed that into the stabalizer. When I included the green screen, since there was no texture (but plenty of 5d compression artifacts) the track went nuts.
5. Do not be afraid to go into the animation channel. As I said in the post above, there were a few KF where the track went crazy for a single KF. What I started to do was reanalyze and hope for the best, but this was killing me. When I got the track pretty good, I simply cracked open the animation channels. I found the offending KF under Analysis and simply moved them by hand so the track was better. Again, we did not need perfect. Just good.
6. Here is how I understand the effect% and smoothness to work. When you turn lock off on X Y or Rotation, you enable effect% and smoothness. Effect % is just that. How much of the stabilize do you want in your shot. For example, if you stabilize an entire shot and remove all camera shake, maybe you want to dial some back in. By lowering the effect%, you will slowly reintroduce some of the original camera jitter. Smoothness will attempt to smooth our X Y or rotation on the shot. Turning Lock off on rotation for example would enable you to smooth only the rotation channel. So X and Y could be perfectly locked down and you could manually move smoothness on rotation to introduce some rotation back in. What I discovered was that if I cranked it up, it helped me smooth out my rotation when it was really screwed up.
7 Use you offsets. Making a huge plate like I did and then shoving it into an action, can really slow down your system. Use the offsets on 2d transformation to keep your shot in the center of the huge frame. This way you can adjust the size of you canvas to be not so huge.