VFX Compositing
VFX Composting is the art where we combine different elements from different sources into single images so that it seems like all those elements are part of the same scene. It is the last Step of the VFX pipeline. To understand how composting works let us consider an example where we have a building destruction scene due to the impact of a missile for that animator adds the missile animation, the smoke effects, and an explosion is added by the Effects artist. Once this is all figured out compositor then layers the missile, smoke explosion, and the building destruction into a single scene using techniques such as color correction, z-depth, etc. The compositor is the one who takes all these different elements and blends them together in an artistic manner so that it appears as a whole. The primary goal of the compositor is to make them appear as if all the scenes were shot at the same time and together by making various color corrections and making use of different techniques.
The main aim of compositing is to combine the different visual effects element into a single scene to make it looks like one whole scene, and the viewer can't figure out the difference.
Composting is one of the important parts of VFX, which is used widely in the entertainment industries. Composting is growing and the two primary reasons behind them are the steady increases in the use of CGI for Visual effects and every CGI element needs to be composited and the other is the advancing of different composting software and hardware technologies.
Photographic truth claim
In one of the papers of tom gunning named "What’s the Point of an Index? or, Faking Photographs", the term photographic truth claim was coined. He beliefs that traditional photography portraits and accurately shows reality. Gunning claims that traditional methods of photography are more truthful. That they are more honest and a closer reflection of reality than modern methods.
But, I disagree. Because we have to understand that most of the craft in today's world is improvised in some way or the other to look good. Sturken and Cartwright also find this question in the book "Visual Culture and Public Policy: Towards a Visual Policy" by Victor Bekkers and Rebecca Moody. Simply because cameras are seen as tools and something that is neutral doesn’t mean that the image taken is an exact replica of reality. they claim that it is an analogy of reality.
they also claim that the images are a version of reality which are totally based on the decisions and assumptions made by the one who takes the picture which includes light, camera lens, angle, and many other factors that can be taken into consideration.
Photorealism
The word Photorealism was coined by Louis K. Meisel in 1969 and appeared in print for the first time in 1970 in a Whitney Museum catalog for the show "Twenty-two Realists." It is also sometimes labeled as Super-Realism, New Realism, Sharp Focus Realism, or Hyper-Realism.
Photorealism is a form of art that consists of painting, drawing, and other graphic media, this is the form in which the artist studies the photograph and attempts to make that photograph look as real as possible.
In VFX, there are two ways to achieve one way is when it directly depends upon the content of the sequence. If the object in question is something that abides by the laws of nature, say, a dog or a table, the visual effects supervisor will choose to photograph as much of the real thing as possible to get the right texture, compositing and light. This may also involve video shooting a live prop.
The other way involves featuring objects that do not obey laws of nature. When an object in question is an alien, there isn’t really much to compare. Different people in the production team may have various ideas on how the creature should sound or move. In this case, the director may choose to select an animal that is closest to his vision of the alien. For example, if the director envisions the creature as Medusa then he will probably ask his effects team to show live octopuses or snakes in place of the creature’s head.
So it solely depends upon the skills of VFX artists to make it seem as realistic as possible.
Motion vs Keyframe
Here we compare two kinds of motion design methods, traditional keyframe and motion capture so we can get a better understanding of which is the best among both. The results showed that in the originality factor, for designing unusual motions the keyframe method has an upper hand and higher performance than other. On the other hand, using motion capture we can create valid actions in quantity due to which fluency factor of creativity was achieved. Now if we compare the flexibility factor we see that although motion capture creates more emotion but keyframe method offered a higher proportion of sentiment design.
it was observed that Keyframe was helpful to design extreme poses. while using motion we get an intuitive design tool to explore more possibilities.
I feel that the motion capture technology is appropriate in certain situations and applications, but also feel like it has become overused and is an easy way for calling a movie animated. Take the character Gollum from “The Lord of The Rings” trilogy. This is a perfect example of how motion capture should be used. Actor Andy Serkis performed each scene in “The Two Towers” in a motion capture suite. However, Andy’s performance was only the basis for the animation. In many cases, motion capture does not record hands and feet very well (at the time of the making of the movie) or facial movements. They then relied on a team of 18 animators to create the facial emotions and movement of the feet and hands giving an end result of a digital character that meshes well with real actors.
I think this is why companies like Pixar and Dreamworks remain at the top of the game. They both rely strongly on the traditional principles of animation and still keep their character concepts and designs very simple and seem to retain that cartoon style.
Assignment
Before making the comparison on whether the new trends of Capture and Real-time filmmaking have made us believe that traditional approaches such as 3D modeling, Rendering, and Animation are no needed, we should get an insight into every term which will help us to understand the advantages of them. What is Real-Time filmmaking?
Real-time filmmaking is an art in which you can make your project ‘live’ as soon as you start production. Using Real-time filmmaking you can start making changes to your camera angles, you can perform experiments on lighting not only that using Real-time filmmaking you can start manipulating your characters and this can be before any render farm to see what your content looks like.
Real-time filmmaking is quite similar to that of traditional cinematography in which we have actors, actresses, and cameras. You film the scene and then end up with different camera angles and shots that you can then edit into a sequence.
The real-time technique strives to enhance the pre-visualization process, especially in the world of complicated Visual Effects. This technique is the game-changer for VFX heavy films because using this director is able to see the action in context to the rest of the film. Directors do not need to depend on the guesswork, rather they’re able to edit and reassess on the day of shooting which helps them to make the finest visuals possible for their film.
By making use of Real-time filmmaking directors can see what is being created while filming, and make decisions about the action and the way things are shot in real-time, rather than reviewing the footage later down the line.
The success of Real-time Filmmaking in the production of the movies like ‘The Lion King’ and ‘The Mandalorian’, and the speed at which they continue to progress are very interesting. What are New Trends in Capture?
❖ Motion Capture
Motion capture is by far the most widely used system in today's realistic world including sports, entertainment, video gaming, and many more.
Motion capture (mocap) is the process of recording the movement of objects or people and then making use of that recording to translate it into actionable data. Motion capture is also called performance capture, it is the technology that combines real-life and animation, allowing you to film someone live and transfer them into computerized form. To use motion capture a specialist places sensors all over the actor. Then they track their movement using those sensors and those movements are then mapped on a computer screen in real-time as a virtual 'skeleton'. Motion capture today:- Now, mo-cap actors will put on a full-body suit covered in tracking dots, which corresponds to key positions in a computer-generated image (CGI) model. The camera records the position of the dots and the changing distances between them, building a framework to base the animation on. These suits allow the filmmakers to track the actor’s entire body. For example, facial mo-cap is used to create realistic facial animations, like Benedict Cumberbatch as Smaug in The Hobbit trilogy.
❖ Reality Capture
Reality Capture is the process in which we scan an object, building, or site, and then we produce a digital model representation for the same. This results in a 3D model based on million data points of the particular object. Reality Capture offers a variety of tools that helps you to build your model. Filter unwanted parts, smooth and simplify your mesh to achieve the best possible result. Reality Capture is the fastest photogrammetry solution among academic and commercial applications, already 5 years in a row. What is 3D modelling? 3D modelling is a technique in computer graphics in which a computer-generated representation of an object or character is created in a 3D space. In 3D modelling, the 3D modeler has to build the 3D model before the Animation or VFX is Applied to them. 3D modeler plays a very crucial role in the Visual effects pipeline. It is used in many different types of TV shows, movies, games, and VR apps. It is used in movies like ‘The Incredible Hulk’ where the modelers have to create a legendary character. In cases like this, precision is crucial in keeping the integrity of the original Hulk character, while creating it with today’s amazing technology standards. Also, we can consider the example of Blizzard's hit game ‘Overwatch’, many of the characters are wearing armor. The character modelers working on games like this have to be well versed in both hard-surface and organic modelling. The 3D modelling process produces a digital object capable of being fully animated, making it an essential process for character animation and special effects. The Modelling technique is responsible for shaping and adding volume to the nascent steps of the CG and VFX process. Hence, Modelling artists need to be well versed in their concepts to avoid snowballing predicaments in the subsequent steps. What is Rendering? Rendering is the process in which by the means of a computer data is collected from the 3D Scene and the calculation are done to display the final result. The output obtained is generally a single image or a series of images rendered & compiled together. Rendering is generally the last phase of the 3D creation process. If we render an animation then it is exported as a video file or it can be a number of images combined. Rendering helps increase efficiency and reduce cost in design. There are two categories of rendering: pre-rendering and real-time rendering. ○ Real-Time Rendering:- It is a technique that is used in interactive graphics and gaming where images must be created at a quick pace. This is because user interaction is quite high and real-time image creation is required. ○ Pre-Rendering:- This rendering technique is used in environments where speed is not a concern and the image calculations are performed using multi-core central processing units rather than dedicated graphics hardware. There are many software’s available in the market to perform these process. It is one of the important stages of the graphics pipeline giving model and animation their final appearance. The rendering of animation also depends on the system in use it is generally better to use a system with a high-powered GPU so the time taken for rendering is decreased, if we make use of CPU it takes a lot of time for an animation to get rendered. After getting a piece of brief information on every term now let us compare how they are different and which one of them is beneficial in respect to other Keyframe vs Motion Capture Now let us understand in perspective of filming so in the case of Keyframe we have to create pose to pose animation and then twinning needs to be carried out in case of 2D each frame will be a separate drawing while in case of 3D each frame would be a separate pose for the character which makes it a very tedious task. Now while using Motion capture the same tasks are made easy because in motion capture an actor is dressed in a suit that has a visuals marker on it which is then used to track the actor’s body movement. Then a specials camera is used to track and record the movement of those markers and then animators apply those movements to 3D models using various software.
In my personal opinion, I feel that motion capture is the perfect technology in some situations or applications but sometimes people might overuse it. Now we can take the example of the character Gollum from “The Lord of The Rings” trilogy. We can say that it is the perfect example of how motion capture should be used. It is much harder to get emotion across in Keyframed animations than with motion capture because of all the subtle movements that the MC records have to be seen and replicated. Also, we know that it takes a lot of time to get those emotions using keyframe then as compared to the Motion Capture. In the originality factor, we see that the Keyframe method has slightly higher performance in designing unusual emotions as compared to the Motion Capture, while in the case of fluency factor motion capture had shown more support in making valid actions. Also taking into consideration the flexibility factor we see that motion capture created more emotions in amount but keyframe offered a higher proportion of sentiments. So to conclude it is proposed to combine both approaches so that it increases digital ability, stimulates creativity, and establishes solid motion knowledge. Real-Time filmmaking A large number of industry leaders and specialists started predicting that a change was coming in the field of animation, and this change was nothing but making animated films in real-time. Many of the top studios and industry leaders are experimenting with real-time projects. They believe that this is the future of filmmaking. But despite all these their some application where we need to make use of different approaches apart from real-time. One of the key advantages of using real-time as the center platform is that each and every department can start their work immediately as no one has to wait for anything because a project is ‘live’ as soon as the production starts. It brings numerous invaluable benefits: One of the most important benefit is what you thought of as ‘pre-production’ is actually now just ‘production’. You don’t need to start from scratch if you want to remove or replace temporary animation and shots with updated or refined versions of it you can do it on a go as a project is live. You can achieve your goals in half the time as the speed of the production is doubled because of improvement in review cycles, feedback loops, and the speed of iteration. Experimentation becomes risk-free because speed is increased, so for the same scene various experiments can be performed and the director has the liberty to choose the best among those. Even if the creative direction changes, the project will always move forward. It has also made an overall positive impact on teams, this is because artists can now truly collaborate, participate, and feel more ownership of the project as a whole. There many more benefits such as: The Skillsets of ‘Upstream’ and ‘Down Stream’ which were separated due to process and time are able to work together and simultaneously. Artists are allowed to bring their thoughts, their sense of mentality, their ideas which lead to empowerment. The impact of work by a particular person can be seen by the larger context of overall story.
Case study Jon Favreau was one of the directors who started experimenting with real-time filmmaking. After exploring its possibilities with The Jungle Book he took one step further by using virtual production to create the live-action remake of “The Lion King”. The film was to be completely CGI, except for the opening shot. For this film, the creators at MPC mirrored the entire classic landscape of the original film by using a 3D virtual environment to encapsulate the entire savanna. Favreau and his team used to wear the VR headsets to find the best location for the film visuals. Apart from the directors and his team’s virtual world, the rest of the team worked in a technology blank space called volume. this space consists of a grid that had a series of virtual reality sensors, which were giving the information back to the computers. The crew still used largely traditional production methods and equipment, like handheld cameras, dollies, and tracks. Using real-time production the directors did not need to worry about the shot instead they can directly focus on the story they can create and light their scenes and see the changes as they are made in real-time. Although this technique helps and makes the work easy, it helped the crew to remove some scenes or to make some tweaks, but this method doesn’t always achieve the highest quality results. After the production was finished there was still a great deal required of post-production team and VFX artist to craft the visuals into quality which can be seen on the cinema screens. Conclusion While Real-Time Filmmaking and new trends in Capture opens many doors of possibilities and also make the work easy for the directors and the filmmakers, but there are some areas where the use of traditional methods is required for further enhancing the quality and for providing the best results to the audience. So it is proposed combining both methods will help to achieve the best standards, increase digital ability, stimulate creativity, and establish solid motion knowledge.