Computer graphics for creativity

Imagine yourself drawing a tree. A line for the trunk, a silhouette for the foliage, a pinch of detail for the shape of the leaves. Without going any further, a three-dimensional tree appears on the screen, close to the one in your imagination. Continue, for example, with a zigzag representing a stream, then a waterfall symbolised by a line, a lake by a circle. Here is your tree in a waterfall landscape with a remarkable visual effect. These are not just graphics or art, they are examples of the scientific work led by Marie-Paule Cani, who joined the Computer Science Laboratory of the École Polytechnique (LIX*) in 2017. This renowned researcher in the field of computer graphics, who has already received several distinctions such as the CNRS silver medal in 2012 and was elected to the Academy of Sciences at the end of 2019, has just been named a fellow of the Hi!Paris interdisciplinary center for her "Creative AI" research project and promoted to the rank of Commander of the National Order of Merit.

Creating computer-generated images requires three stages: geometric modelling (e.g. for the tree), the animation of this geometry (the movements of the water, for example) and rendering, i.e. the calculation of the final image. The first two stages are at the heart of Marie-Paule Cani's research, who compares 3D creators to "little gods who create their virtual worlds and choose their laws". Indeed, it is not always a question of applying precise simulations of the laws of physics, even if the cursor is sometimes very close, for example for the animation of liquids. On the other hand, the calculations must be carried out in a reasonable time. "Recently, our visual simulation of the movement of clouds in the sky was not based on the models of climatologists or fluid physicists. We started from our own approximation," explains the researcher. Computer graphics does not seek to follow physics point by point or particle by particle, but to respect a visual plausibility that allows a spectator of this virtual world to immerse himself in it without a hitch.

To reconcile this creative freedom with the constraints of plausibility and computing time, Marie-Paule Cani contributed to the development of methodologies specific to computer graphics, in particular when she was a professor at Grenoble-INP. Among these tools, implicit surfaces, a way of geometrically modelling highly deformable surfaces, are currently the model used to represent the surface of water in film special effects for example. Recently, his team at the Ecole Polytechnique has extended implicit representations to allow virtual characters to wear several layers of clothing without entanglement.

This panoply of computer graphics tools is used to build these virtual worlds using 'multi-layer' models. Firstly, identify the sub-phenomena that appear, i.e. the different layers, then seek for each one which model is the most suitable, implicit surfaces for instance. The trick is finally to couple these different models over time to ensure the plausibility of the result. "For example, in our work on cloud movement, a layer made up of several horizontal calculation grids simulates fluid mechanics on a large scale, with convection movements and interaction with the terrain, while a layer on a finer scale generates the details in a procedural way, based on a priori knowledge of the different types of cloud," continues the computer scientist.

Artificial intelligence (AI) has also begun to integrate the toolbox of computer graphics.  "But certainly not to create for us! warns the researcher. AI should come in as an assistant that provides us in real time with the most suitable tool, which allows the models to react better to our creative gestures. For example, we may want to make intelligent cut and paste, such as taking the clothes of one character and putting them on another, making sure that they automatically adapt to the new morphology. The AI can also learn to place scenery in a virtual world, providing us, for example, with a "bush" brush that allows us to create landscapes in a plausible way, considering the climate and nature of the soil, the local slope, and the interactions between plant species.

With such achievements, it is easy to understand why the creative industries, from video games to cinema, are the primary users of computer graphics. Marie-Paule Cani participates in the Ecole Polytechnique chair financed by the company Ubisoft, which recently recruited two students from the Master of Science and Technology (MScT) 'Artificial Intelligence and advanced Visual Computing' that she set up. In the film industry, the studio Weta digital has used the results of some of its research papers, for example, to create some of the hair for characters in the film Avatar. "Talking to these creative industries helps us find interesting scientific problems. Conversely, they draw on our methods, adapting them to their needs. Often, we only find out about it afterwards. But we remain in our role as researchers by producing prototype software to test our ideas, and leaving the industrialisation to them. "

Marie-Paule Cani also collaborates a lot with scientists from other disciplines: biologists for the modelling of trees, geomorphologists for the formation of mountain ranges, vulcanologists for the visual simulation of volcanic plumes, or prehistorians for the reconstruction of the ecosystem of the Tautavel valley 450,000 years ago... and the list goes on. My 'Creative AI' project with Hi!Paris is aimed at such interactions with other researchers. The idea of the project is to provide them with a testbed environment so that they can experiment with their ideas: draw a 2D sketch of their mental model, a sketch of a cell for a biologist, and immediately see a 3D world that matches, and then inject their knowledge on the fly, for example, that such and such an object deforms at constant volume. AI will be used to make the models react in an expected way to their creative gestures. "This could help with thinking in science, and also with communication," she hopes.

*LIX: a joint research unit CNRS, École Polytechnique - Institut Polytechnique de Paris