Master 1 & 2

Parallel and Distributed Systems


Overview


Parallel and distributed systems are ubiquitous to many applications of our daily life (IA applications, online games, social networks, web services, healthcare simulations…). These systems distribute computation over many computing units because they have to sustain massive workloads that cannot fit into a single computer.

Designing efficient, correct and easy-to-maintain parallel and distributed systems is challenging. This is inherent to the complexity of managing multiple machines, many users and very large data sets. In the PDS track, students learn how to address such challenges and their related research questions. Students registered in the track learn both the theory and practice that underpin such systems with courses ranging from operating systems, compilation and large-scale distributed middleware to the algorithmic and mathematical foundations of distributed and parallel systems.

Language of instruction: english
ECTS: 120 (60 during first year, 60 during second year)
Oriented: research
Duration: 2 years
Courses Location: Palaiseau


Educational objectives


The main objective of the program is to train the next generation of researchers and R&D engineers that will lead the innovation in the field of parallel and distributed systems.

The basic courses include architecture, compilation, system programming and distributed algorithms. The advanced courses give the theoretical foundations and the advanced techniques used to design and implement efficient, correct and easy-to-maintain parallel and distributed systems. These courses also show how to apply these techniques to different fields (cloud computing, artificial intelligence, blockchain or internet-of-things).

The program also aims at developing the skills required to pursue a research career through research seminars and research-oriented projects.


Program structure


Mandatory courses:

  • A Programmer’s Introduction to Computer Architectures and Operating Systems (Francesco Zappa Nardelli, Timothy Bourke and Théophile Bastian, 48h, 5 ECTS)

    We will explain the enduring concepts underlying all computer systems, and show the concrete ways that these ideas affect the correctness, performance, and utility of any application program.

    This course serves as an introduction to the students who go on to implement systems hardware and software. But this course also pushes students towards becoming the rare programmers who know how things work and how to fix them when they break.

    This course will cover most of the key interfaces between user programs and the bare hardware, including:
    – The representation and manipulation of information
    – Machine-level representation of programs
    – Processor architecture
    – The memory hierarchy
    – Exceptional Control Flow
    – Virtual memory

  • Basics of distributed systems development (in Java) and software project management (Ada Diaconescu, 24h, 2.5 ECTS)

    This course aims to introduce the basic notions necessary for the understanding and development of distributed applications (focused on Java). It also aims to introduce the concept of middleware and to provide two concrete examples of different middleware types: based on remote method invocations and message-oriented communication models.

  • Fundamentals in distributed computing 1 (Bernadette Charron-Bost, 48h, 5 ECTS)

    Distributed systems are composed of several computational units, classically called processes, that run concurrently and independently, without any central control. Additional difficulties are introduced by asynchrony (processes and channels operate at different speeds) and by limited local knowledge (each process has only a local view of the system and has a limited amount of information).

    Distributed algorithms are algorithms designed to run in this quite challenging setting. They arise in a wide range of applications, including telecommunications, internet, peer-to-peer computing, blockchain technology…

    This course aims at giving a comprehensive introduction to the field of distributed algorithms. A collection of significant algorithms will be presented for asynchronous networked systems, with a particular emphasis on their correctness proofs. Algorithms will be analyzed according to various measures of interest (eg., time and space complexities, communication costs). We will also present some « negative » results, i.e., impossibility theorems and lower bounds as they play a useful role for a system designer to determine what problems are solvable and at what cost.

    Content:
    – Modelling of distributed networked systems
    – Wave and traversal algorithms
    – Leader election
    – Logical time and global snapshots
    – Detection of stable properties
    – Synchronizers
    – Link reversal algorithms

  • Fundamentals in distributed computing 2 (Petr Kuznetsov, 24h, 2.5 ECTS)

    The primary focus of the module is on understanding the foundations of distributed computing. The module will discuss basics of synchronization, fault-tolerant shared-memory computations, consensus and transactional memory.

  • Introduction to compilation and architecture: from the transistor to the programming language (Pascal Hennequin and Gaël Thomas, 48h, 5 ECTS)

    Through the implementation of a compiler from a subset of Java to the MIPS assembler, the objectives of the course are threefold:
    – To study the architecture of the processors and the writing of assembler code.
    – Introduce the theory of languages and master the practical applications in computer science. In particular, compiler’s compiler generation tools (compiler’s compiler)
    – Enforcing programming skills in an modern language: (i) by understanding how programming paradigms are implemented for their executions, and (ii) through practice in writing a compiler

  • Operating systems (François Trahay and Gaël Thomas, 48h, 5 ECTS)

    This course presents the design principles of modern operating systems. In this course you will learn:
    – how applications interact with the operating system and how the operating system interacts with the hardware
    – the main internal mechanisms of an operating system (memory manager, I/O subsystem, scheduler)
    – how these mechanisms are implemented in a real operating system through the study of the XV6 operating system
    – how to develop parallel applications and parallel operating systems with threads

  • PDS research projects (20 ECTS in M1 and 7.5 ECTS in M2)

    During the master, a student will learn research by doing research. During the two years of the master, a student will thus spend between one or two days each week in a research group in order to do research projects with professors and PhD students of IP Paris.

  • PDS seminars (5 ECTS)

    The PDS seminar consists in presentations of ongoing research works, both by students on papers of recent conferences or journals and by professors from IP Paris and other universities.

  • 6-month research internship (30 ECTS)

Optional courses:

  • Advanced programming of multi-core architectures (Gaël Thomas, 48h, 5 ECTS)

    This course presents advanced programming techniques for multi-core architectures: lock-free algorithms, transactional memory, virtualization techniques or techniques to mitigate non-uniform memory architectures. This module presents the theoretical concepts underlying these systems and their practical implementation.

  • Algorithmic basics of distributed storage systems and blockchains (Petr Kuznetsov and Matthieu Rambaud, 24h, 2.5 ECTS)

    Understanding of the algorithmic foundations behind the blockchain technology: we study the algorithmic basics behind decentralized and consistent storage systems that tolerate churn and malicious behavior of participating nodes. We begin with the study of classical protocols for reconfigurable storage and Byzantine fault-tolerance. Then we explore the protocols designed for « open » systems, exemplified by the prominent blockchain technology.

  • Algorithms for dynamic and reconfigurable distributed systems (Petr Kuznetsov, 24h, 2.5 ECTS)

    This course presents algorithms commonly used to implement dynamic and reconfigurable distributed systems.

  • Cloud infrastructure (Pierre Sutra, 48h, 5 ECTS)

    This course presents cloud infrastructures in order to:
    – acquire an overview of Cloud computing (e.g., data centers, everything-as-a-service, on-demand computing, cloud economy model)
    – apprehend the fundamental notions in Cloud computing (e.g., fault-tolerance, elasticity, scalability, load balancing)
    – understand how virtualization works (VM, container)
    – deconstruct and classify a distributed data store
    – recognize data consistency problems and know common solutions

    In details, a student will learn how to:
    – deploy and maintain IaaS
    – construct base data storage services (e.g., key-value store, coordination kernels)
    – construct and deploy a micro-service architecture
    – think for dependability & scalability

  • Consensus in multiagent systems (Bernadette Charron-Bost, 24h, 2.5 ECTS)

    This course presents a study of different consensus problems: exact consensus, possible consensus, or asymptotic consensus. These problems will be studied within the framework of a dynamic topology and under the hypothesis that some system components (communication channels or agents) may make errors.

  • Data analytic infrastructures (Amel Bouzeghoub, 48h, 5 ECTS)

    This course presents data analytics infrastructures with a strong focus on Spark and on Web semantic.

  • Dynamic, autonomic and self-adapting systems (Ada Diaconescu, 24h, 2.5 ECTS)

    This course presents both algorithms and techniques used to implement self-adapting systems.

  • Distributed Computing and Combinatorial Topology (Petr Kuznetsov, 24h, 2.5 ECTS)

    The principal goal of this course is to learn the techniques for analyzing distributed algorithms for real systems combining parallelism with unpredictable delays, such as multi-cores, wireless networks, distributed systems, and Internet protocols. The use of mathematical tools of combinatorial topology has recently been used to close several long-standing open questions in fault-tolerant distributed computability. In this class, we learn how to master these tools. The course is based on the textbook by Maurice Herlihy, Dmitry Kozlov, and Sergio Rajsbaum, « Distributed Computing through Combinatorial Topology ».

  • GPU programming for learning (Goran Frehse, 24h, 2.5 ECTS)

    The aim of this course is to give a vision of algorithms and their implementations in modern machine learning libraries on neural networks. In particular, the use of specific hardware, such as graphics cards, to improve performance is at the heart of these libraries. It is important to understand how the calculations are shared between the hardware and the CPU.

  • High performance runtimes (Élisabeth Brunet and Patrick Carribault, 48h, 5 ECTS)

    With the advent of multicore processors (and now many-core processors with several dozens of execution units), expressing parallelism is mandatory to enable high performance on different kinds of applications (scientific computing, big-data…). In this context, this course details multiple parallel programming paradigms to help exploiting such a large number of cores on different target architectures (regular CPUs and GPUs). The course introduces distributed-memory model (MPI), shared-memory model (OpenMP) and heterogeneous model (CUDA). All these approaches would allow leveraging the performance of differents computers (from small servers to large supercomputers listed in Top500).

  • Introduction to technologies of distributed software development (Ada Diaconescu and Remi Sharrock, 24h, 2.5 ECTS)

    This course introduces the basic concepts of tools for distributed systems and middleware and presents concrete examples of middleware of different styles: RMI , MOM/JMS and Akka. These examples will be compared in order to better understand their applicability to different types of applications. The concepts explored here, via fairly simple examples, are reused in most middleware and application server platforms available in the industry.

  • Middleware and software architecture for distributed applications (Sophie Chabridon, Denis Conan and Chantal Taconet, 48h, 5 ECTS)

    This course presents how modern distributed applications are implemented. After this course, a student should be able to:
    – Identify and use the main interaction patterns between distributed components (synchronous call, callbacks, orchestration, asynchronous calls, publish/subscribe) through the study and practise of several middleware (RestFul Services, Web Services, Rabbit-MQ, JavaEE).
    – Design and implement a distributed application made of several functional modules with computing components, persistent components, client components. The application will be realised in the context of a microproject lasting during all the teaching unit.
    – Identify design patterns and architectural patterns used by middleware for distributed applications.
    Define the quality of a distributed architecture and discuss architecture choices to respond to quality of software architecture requirements (e.g. interoperability, security, scalability).

  • Object-oriented modelling: structural model and behaviour simulation (Ada Diaconescu, 24h, 2.5 ECTS)

    The objective of this course is to learn the basics of the most popular standard in industry to model software architectures: UML. In the scope of a simple and generic design process, this course covers (i) the modelling of the functional requirements of a software application, (ii) the design of software architectures with object oriented paradigms, and (iii) the behavioural modelling of software applications.

    This course will address the following topics:
    – Requirements specifications, expressed with use case diagrams.
    – Object oriented architecture modelling, based on class diagrams refined with well-known design patterns from the Gang Of Four (GOF).
    – Interactions modelling with sequence diagrams.
    – Behavioural models using state-chart diagrams.

    This course also proposes a modelling approach, linking together these different UML2 diagrams, and following the recommendations that were standardised as a consensus in industry. Students will have to apply this modelling approach on a project using tools.

  • Safe Intelligent Systems (Sylvie Putot, 48h, 5 ECTS)

    The interaction of computer components, which compute and communicate, with their environment governed by physical laws, such as an aircraft or an implanted medical system, is at the centre of the emerging field of cyber-physical systems. Among the challenges posed by these systems that we will focus on is the increasing use of artificial intelligence algorithms (typically neural networks) for both perception and control.

    Mastering the modelling, control and verification of the behaviour of such systems is crucial to guarantee the efficiency, functionality and reliability of these systems, which are increasingly complex and most often critical in terms of safety or cost.

    The course will focus on finding a balance between modeling and verification, and between theoretical foundations and practical aspects. In particular, it will introduce the principles and the use of some tools representative of the state of the art, and will focus on realistic case studies (notably a temporal model of a pacemaker modelled and verified with the Uppaal tool).

    Content:
    – Introduction, models and synchronous languages
    – Synchronous Languages: Specification and Verification
    – Time-delayed automatons
    – Time logic and model-checking verification
    – Modeling and simulation of hybrid systems
    – Reachability analysis of hybrid systems
    – Stability and control of hybrid systems
    – Safety of intelligent stand-alone systems

    The course is illustrated by applicative exercises (in progress), and by practical work sessions using different modeling and verification tools. The last four sessions are dedicated to a project, in which the students can choose to focus on the modeling and verification of a cyber-physical system, or on the further study of an aspect of a verification method.

    The course will be validated for a small half on the practical work, and the remaining half on the project and associated oral.

  • Free ECTS (7.5 ECTS)

    A student chooses 7.5 ECTS from any track of the master degree in computer science.


IP Paris labs involved



Career prospects


A graduate student from the master program will acquire the skills to pursue a Ph.D. in Computer Science. A graduate student will be able to apply for a Ph.D. program in a highly ranked University, including the IP Paris partner schools.

A graduate student will also acquire the background to join the R&D departments of the most innovative IT companies.


Institutional partners


Polytechnique
ENSTA
Telecom Paris
Telecom SudParis


Admissions


Application guidelines for a master’s program at IP Paris

Academic Prerequisites:

Year 1: Candidates for the Master programme should hold at least a Bachelor degree in computer science or in applied mathematics and have an outstanding academic record.

Year 2: Admission can be granted directly in the 2nd year (M2), provided that necessary prerequisites have been met. Any student (from France or abroad) who has completed at least the first year of a Master programme in computer science may apply.

Application timeline

Deadlines for the Master application sessions are as follows:
– First session: February 28, 2020
– Second session: April 30, 2020
– Third Session (optional): June 30, 2020 (only if there are availabilities remaining after the 2 first sessions)
Applications not finalized for a session will automatically be carried over to the next session.

You shall receive an answer 2 months after the application deadline of the session.
You can check your application status by logging in your candidate space.


Tuition fees


National Master: Official tuition fees of the Ministry of Higher Education, Research and Innovation (2019-2020, EU students: 243 euros / Non-EU students: 3770 euros)


Contact


Gaël THOMAS, parallel and distributed systems,
Email
Profil