Probability, Statistics and Information Theory (ex Probability and Statistics)

Professor: Stefan Wolf
Assistant: Alexander Malafeev
Assistant: Marcel Pfaffhauser

Tipo di corso: Lecture
Valore in crediti ECTS: 6
Riferimenti bibliografici sul sito della biblioteca (CoRe)


Academic year 2012/2013 - Fall semester

Prerequisites: Calculus I and Discrete Mathematics I
Bachelor course available to masters students enrolled in the Management and Informatics Master

Objectives
Randomness and information are ubiquitous in computer science as well as in our everyday lives. Many events and observations are steered by random phenomena. The dynamic behavior of systems can often be modeled as a random process. Experimental analysis requires basic knowledge of probabilities and statistical methods. Further, in computer science, randomness often is an important tool for efficient solutions. The course will cover the basic mathematical techniques to argue about these topics. Most of the course will be devoted to a general introduction to probability theory. In a second part, we will give an introduction to statistical methods and discuss some of the most important statistical tests. The third goal deals with the mathematical treatment of the notions of uncertainty and Information. The main objective of the course is to learn, understand, and apply the necessary basic tools for coping with randomness, probability, and information.

Contents

Probability Theory

  • Basic Notions
    • Probability spaces
    • Elementary events and distributions
    • Events, independence
  • Conditional probabilities
    • Bayes´ rule
  • Random variables
    • Expectation value
    • Standard deviation and variance
  • Laws of large numbers
  • Important special distributions
  • Binomial and normal distributions
  • Poisson distribution
  • Hypergeometric distribution
  • Stochastic processes and Markov chains

Basics of Statistics

  • Estimation
  • Confidence intervals
  • Hypothesis testing

Information Theory

  • Basic Quantities
    • Entropy and conditional entropy
      • Chain rule
    • Information and conditional information
    • Fano´s lemma
  • Asymptotic Equipartition Property
  • Information Sources
  • Data Compression
    • Bounds
    • Huffman codes
    • Universal compression
    • Kolmogorov complexity
  • Channel Coding
    • Channel capacity
    • Shannon´s theorem
    • Linear codes

Teaching mode
Lectures and exercises