An EM Exercise

11 Nov 2016 The Expectation-Maximization (EM) algorithm is a popular method to obtain the Maximum Likelihood Estimate (MLE) when some of the data may be missing. See [Roche, 2012] for a nice tutorial on EM. The following problem is a nice exercise in working out the algorithm details to reinforce the concepts. Read more...

Essential PCA

29 Oct 2016 Assume that we are given a matrix $X \in \mathbb{R}^{n \times p}$. Each row of the matrix $X$ is considered to be an observation represented by a data vector that measures $p$ features of some phenomenon. We can think of Principal Component Analysis (PCA) as trying to trying to solve two related problems. Read more...

Bayes Classifier with Asymmetric Costs

12 Oct 2016 Thanks to Prof. Larry for this problem! Consider the following binary classification problem. Every individual of a population is associated with an independent replicate of the pair $(\mathbf{X}, Y)$, having known joint distribution and where the (observed) covariate $\mathbf{X}$ has a (marginal) distribution $\pi$, and the (unobserved) response $Y \in \{-1, 1\}$. Suppose the costs of misclassifying an individual with $Y = 1$ and $Y = -1$ are $a > 0$ and $b > 0$, respectively. What’s the Bayes decision rule? Read more...