Course Title: Advanced Techniques in Human Modeling, Animation and Rendering Course Organizer Name: Daniel Thalmann Company/University: Swiss Federal Institute of Technology Mailing Address: Computer Graphics Lab Swiss Federal Institute of Technology CH 1015 Lausanne Switzerland Telephone Number: Office: 41-21-693 52 14 Home: 41-22-48 69 06 FAX Number:41-21-693 5328 Electronic Mail Address:thalmann@eldi.epfl.ch Course Level: Intermediate Who Should Attend: This course is for artists and/or scientists, who would like to know how to recreate any human being. Recommended Background: A basic knowledge in computer graphics and computer animation is recommended. Highlight of Course Content: This course is not a traditional human animation course. It does not discuss the techniques of motion control of articulated bodies (keyframe, inverse kinematics, dynamics...), but emphasize problems in human appearance. We will explore recent methods to improve the realism of human appearance in computer-generated films. We will emphasize human shape modeling and deformations, hair rendering, textures and cloth animation. We will also discuss the impact of natural language on human animation. Course Objectives: 1) to bring attendees up-to-date on the frontiers of research in human modeling, animation, and rendering 2) to show attendees what is needed to generate realistic human and well-known personalities 3) to introduce several advanced techniques: hair rendering, cloth animation, use of natural language in human animation, physics-based facial animation, vision-based behavioral animation Course Description: This course will discuss several important problems to be solved to incorporate realistic human characters in computer-generated films. Research in this area implies the development of techniques: for improving the physical aspects of the actors: shapes, colors, textures, for improving the deformation of limbs during motion, for improving facial expressions and deformations, for specifying the tasks to be performed using natural language, for simulating behaviors. The first problems we will discuss are the problems of shape creation and animation. For the shape creation, we will show the impact of new 3D devices (Spaceball, Polhemus, dataglove) on the design of human body and face. We will then discuss the problem of improving the realism of motion not from the joint point-of-view as for robots, but in relation to the deformations of human bodies during animation. Two methods for improving these deformations are described. The Joint-dependent Local Deformation (JLD) approach is convenient when there is no contact between the human being and the environment. A finite element method is used to model the deformations of human flesh due to flexion of members and/or contact with objects. We will describe recent applications of deformable models to the real-time physically-based modeling and animation of the human face. We will also introduce a technique for estimating muscle contractions from video sequences of human faces performing For synthesized images containing humans beings, realistic hair has long been an unresolved problem and therefore has often been absent from these images. The great number of geometrical primitives involved and the potential diversity of the curvature of each strand of hair makes it a formidable task to manage. In this course, we will review techniques for rendering fur and hair and modeling hairstyle. We will emphasize a method based on alpha-blending for generating images completely free of aliasing artifacts. A simple but effective anisotropic illumination model will be formulated to simulate diffuse backlighting and strong reflected highlights present around each hair. A method will be also shown to incorporate the hair among objects generated by other conventional rendering algorithms. Another problem in the generation of realistic human beings is the problem of texture. The specification of texture maps has been done manually for many years. We describe ways to automatically construct texture maps with certain properties to represent, in particular, "dirty" natural textures and skin texture. The specification is stated in language and a rule-based system figures out the appropriate placement and parameters for texture generation based on fractal subdivision and distribution models. Clothes in computer-generated films are often simulated as a part of the body with no autonomous motion. In this course, we will present methods for designing and animating clothes. Two separate problems have to be solved for cloth animation: the motion of the cloth without collision detection and the collision detection of the cloth with the body and with itself. Deformable models provide a powerful approach to the first problem. In addition to free-form geometry, the formulation of deformable models involves physical principles that govern rigid and nonrigid dynamics, including elastic, inelastic, and thermoplastic deformations. The physical underpinnings of deformable models offer significant advantages over conventional, geometric modeling techniques for computer animation. Deformable models also suggest novel user interaction and shape reconstruction methodologies through the use of simulated forces. For the collision and self- collision problem, we present a method of collision detection especially efficient for dynamic models. The process of interpreting Natural Language instructions shows deep and fascinating connections between language and behavior. When the behavior is to be portrayed by a synthetic human agent, various questions arise regarding the types and roles of planning, geometric reasoning, constraint satisfaction, human capabilities, and human motion strategies. We discuss the realization of a language-to-animation connection through computational models of verb definitions executed by a simulator accessing a Knowledge Base, and animated through a graphical human figure. Finally, we will present an innovative way of animating actors at a high level based on the concept of synthetic vision. The objective is simple: to create an animation involving a synthetic actor automatically moving in a corridor avoiding objects and other synthetic actors. To simulate this behavior, each synthetic actor uses a synthetic vision as its perception of the world and so as the unique input to its behavioral model. Techniques for Presentation - computer-generated films illustrating physics-based facial animation, hair animation, cloth animation - video demo illustrating the use of software for modeling of human shape, hairstyle and clothes - slides Speakers 1.Name: Norman Badler Company/University: University of Pennsylvania Title: Professor Mailing Address:Computer and Information Science University of Pennsylvania 200 South 33rd Street Philadelphia PA 19104-6389 Telephone Number: Office: (215) 898 5862 FAX Number:(215) 898 0587 Electronic Mail Address:badler@central.cs.upenn.edu ___________________________________________________________ 2.Name: Nadia Magnenat-Thalmann Company/University: University of Geneva Title: Professor Mailing Address:MIRALab Centre Universitaire d'Informatique 12 rue du Lac CH 1207 Geneva Switzerland Telephone Number: Office:41-22-787 6581 Home: 41-22-48 69 06 FAX Number:41-22-735 3905 Electronic Mail Address: thalmann@uni2a.unige.ch ___________________________________________________________ 3.Name: Demetri Terzopoulos Company/University: University of Toronto and Schlumberger Lab. Title: Professor Mailing Address:Computer Science University of Toronto 10 King's College Road Toronto, Ontario M5S 1A4 Canada Telephone Number: Office: (416) 978-7777 FAX Number:(416) 978-1455 Electronic Mail Address: dt@orasis.vis.toronto.edu ___________________________________________________________ 4.Name: Daniel Thalmann Company/University: Swiss Federal Institute of Technology Mailing Address:Computer Graphics Lab Swiss Federal Institute of Technology CH 1015 Lausanne Switzerland Telephone Number: Office: 41-21-693 5214 Home: 41-22-48 69 06 FAX Number:41-21-693 5328 Electronic Mail Address:thalmann@eldi.epfl.ch About the lecturers Nadia Magnenat Thalmann is currently full Professor of Computer Science at the University of Geneva, Switzerland and Adjunct Professor at HEC Montreal, Canada. She has served on a variety of government advisory boards and research program committees in Canada. She has received several awards, including the 1985 Communications Award from the Government of Quebec. She is the President of the Computer Graphics Society. Dr. Magnenat Thalmann received a BS in psychology, an MS in biochemistry, and a Ph.D in quantum chemistry and computer graphics from the University of Geneva. Daniel Thalmann is currently full Professor, Head of Computer Science, and Director of the Computer Graphics Laboratory at the Swiss Federal Institute of Technology in Lausanne, Switzerland. Since 1977, he was Professor at the University of Montreal and codirector of the MIRALab research laboratory. He received his diploma in nuclear physics and Ph.D in Computer Science from the University of Geneva. He was visiting Professor at the University of Nebraska and invited researcher in the Computer Graphics Group at CERN. He cochairs the EUROGRAPHICS Working Group on Computer Simulation and Animation Nadia Magnenat-Thalmann's and Daniel Thalmann's research interests include 3D computer animation, image synthesis, and scientific visualization. They have published more than 100 papers in these areas and are coauthors of several books including: Computer Animation: Theory and Practice and Image Synthesis: Theory and Practice. They are also codirectors of several computer-generated films Dream Flight, Eglantine, Rendez-vous Montreal, Galaxy Sweetheart, IAD, Flashback, and Still Walking. They cochaired several conferences included Graphics Interface '85, CGI '88, Computer Animation '89, '90, and '91. They are also co-editors-in-chief of the Journal of Visualization and Computer Animation and editors of the Visual Computer. Dr. Norman I. Badler is the Cecilia Fitler Moore Professor and Chair of Computer and Information Science at the University of Pennsylvania and has been on that faculty since 1974. Active in computer graphics since 1968 with more than 80 technical papers, his research focuses on human figure modeling, manipulation, and animation. Badler received the BA degree in Creative Studies Mathematics from the University of California at Santa Barbara in 1970, the MSc in Mathematics in 1971, and the Ph.D. in Computer Science in 1975, both from the University of Toronto. He is Co-Editor of the Journal of Graphical Models and Image Processing. He also directs the Computer Graphics Research Facility with two full time staff members and about 40 students. Demetri Terzopoulos is an associate professor of computer science at the University of Toronto and a fellow of the Canadian Institute for Advanced Research. For the past five years he has been affiliated with Schlumberger, Inc., serving as a program leader at the Laboratory for Computer Science, Austin, TX, and at the former Palo Alto Research Laboratory. Previously he was a research scientist at the MIT Artificial Intelligence Laboratory, Cambridge, MA. His areas of interest include computer vision, computer animation, visualization, and massively parallel computation. Terzopoulos received a PhD in artificial intelligence from MIT in 1984. He received an MEng in electrical engineering in 1980 and a BEng in honours electrical engineering in 1978, both from McGill University. He is a member of the editorial boards of CVGIP: Graphical Models and Image Processing and the Journal of Visualization and Computer Animation and is a member of the IEEE, AAAI, NY Academy of Sciences, and Sigma Xi. Detailed outline of course *** THIS IS A PRELIMINARY OUTLINE: IT MAY BE MODIFIED WITHOUT ANY NOTICE *** 1. The Complexity of Models in Human Animation: an Overview (Magnenat-Thalmann, 30 min) 2. Human Shape Design and Deformations Human Body Shape Design (Magnenat-Thalmann, 30 min) Actor shape modelling Human prototyping The Use of 3D Input Devices in Human Modeling and Animation (Space Ball, Polhemus, DataGlove) Local deformations A Sculptor Approach to Human Modeling Human Body Deformations (Thalmann, 30 min) Joint Local Deformation (JLD) operators for body mapping JLD operators for hand covering Mapping algorithm Deformations based on Finite-Element theory Case study: ball grasping and pressing Animation control The physically-based approach (Terzopoulos 10 min) Overview of the physically-based approach to human modeling and comparison with geometric techniques. Aspects of human character animation that can benefit from physically-based modeling. Modeling deformable materials in and on the human body (Terzopoulos 20 min) Review of deformable models of curves, surfaces, and solids. Implementation recipes for dynamic deformable meshes. Associated physically-based constraint methods. Physics-based Facial Modelling and Animation (Terzopoulos, 30 min) Biophysics of facial tissue. Capturing facial geometry using adaptive deformable meshes. Deformable models of facial tissue. Anatomical facial muscle models. Controlling facial muscles. Physically-based approaches to capturing facial expressions from video for realistic facial animation. 3. Human Rendering Texture Synthesis (Badler, 15 min) Texture generation models Interaction between texture and geometry Determining textures from rules Interfacing to the texture generator through natural language Skin Texture, Hair Modelling and Rendering (Thalmann, 30 minutes) Skin Texture Fur models Earlier hair models Pixel-blending techniques An anisotropic light model Shadow Buffers Composition stage Use with conventional renderers Hairstyle generation HAIRDRESSER an animator-interface 4. Cloth Modelling and Animation Dynamic cloth models (Terzopoulos 15 min) Basic physics of cloth Deformable models of clothing External forces Draping effects Constraints from impenetrable obstacles Wrinkles, Collisions and User Interface (Magnenat Thalmann, 30 min) Wrinkles Self-collision avoidance Force field model User interface for designing clothes Case-study: Marilyn's skirt in the film Flashback 5. The Use of Natural Language in Human Animation (Badler, 1h30) Animation from Instructions The graphical basis: Human figure capability models Biomechanical primitives: Torso Motion primitives: Reach, grasp, lift, move, look-at,... Motion strategies: strength, posture Simulation of Processes Knowledge base Control algorithm Monitors and interruptions Motion Verb Semantics Component analysis of motion verbs Kinematic, dynamic, and constraint types Making, breaking, and maintaining constraints Spatial prepositions Adverbs and manner Planning Issues Reactive and incremental planning Coarse versus fine motion planning Natural Language Understanding Facial animation from language intonation 6. Emotions, individualized models and Behavioral Human Animation (Thalmann, 30 min) Emotion, Generation and Synchronization with Speech Behavioral animation Vision-based obstacle avoidance Displacement local automata Case study: vision-based walking Mechanisms of locomotion An allure-based walking model