uncertainty/deep-ppl.md

@@ -10,7 +10,7 @@ As illustrated below the user specified probabilistic program specifies how to g
 
 ![PPL-overview](uploads/8746dead4ee50539540a6da708a9ceb5/PPL-overview.png)
 
-**How to Set-up a PPL Program**
+**How to Set-up a PPL Program?**
 
 The goal is to infer a probabilistic model θ for observed data x<sub>1</sub>, ..., x<sub>n</sub>: p(θ|x<sub>1</sub>, ..., x<sub>n</sub>). 
 This is done by using Bayesian statistics and requires usually the following steps:
@@ -18,6 +18,23 @@ This is done by using Bayesian statistics and requires usually the following ste
 2.  Specify the measured / observed data (i.e. import real world data)
 3.  Run Inference using the model and the data (e.g. by Monte Carlo Sampling) to acquire posterior distribution. In other words, the inference adjusts the prior distribution using the observed data data to give a more precise model.
 
+*PPL Problem:*
+The central problem of PPLs is that comptuting the posterior distribution of latent parameters θ and observations x<sub>1</sub>, ..., x<sub>n</sub> (i.e. p(θ|x<sub>1</sub>, ..., x<sub>n</sub>)) can be costly of even intractable. 
+*Solution:*
+Use variational Inference
+
+
+**Why Deep Learning and PPL?**
+
+The core advantages of deep Learning (DL) are 
+*  Automatic hierarchical feature learning (i.e. automatic representation learning)
+*  DL approaches have proven to be very powerful in many applications (such as speech and image recognition)
+*  Nowadays easy to handle since there are good DL libraries available wich offer (1) automatic differentiation; (2) GPU support; and (3) a Python programming interface
+
+The downside of DL is however, that the trained models are a black box and does not allow a meaningful interpretation of its parameters (weights and biases). In contrast, Bayesian models offer direct interpretation but are difficult to design and handle. 
+
+By combining DL with PPL the hope is to have the best of both worlds: easy to handle, powerful yet interpretable models which utilise reale world data to be trained automatically.  
+
 **Conceptual Categorisation of Deep Probabilistic Programming Languages (PPL)**
 
 As illustrated in the Figure below, deep PPLs combine three existing concepts into one new concept: "programming languages", "Bayesian statistics" and "deep learning".