The Expectation Maximization Algorithm
Frank Dellaert
College of Computing, Georgia Institute of Technology
Technical Report number GIT-GVU-02-20
February 2002
Abstract
This note represents my attempt at explaining the EM algorithm (Hartley, 1958; Dempster et al., 1977; McLachlan and Krishnan, 1997). This is just a slight variation on Tom Minka’s tutorial (Minka, 1998), perhaps a little easier (or perhaps not). It includes a graphical example to provide some intuition. 1 Intuitive Explanation of EM

EM is an iterative optimization method to estimate some unknown parameters, given measurement data U. However, we are not given some “hidden” nuisance variables J, which need to be integrated out. In particular, we want to maximize the posterior probability of the parameters given the data U, marginalizing over J: = argmax

X
J2Jn
P(; JjU) (1)
The intuition behind EM is an old one: alternate between estimating the unknowns and the hidden variables J. This idea has been around for a long time. However, instead of finding the best J 2 J given an estimateat each iteration, EM computes a distribution over the space J . One of the earliest papers on EM is (Hartley, 1958), but the seminal reference that formalized EM and provided a proof of convergence is the “DLR” paper by Dempster, Laird, and Rubin (Dempster et al., 1977). A recent book devoted entirely to EM and applications is (McLachlan and Krishnan, 1997), whereas (Tanner, 1996) is another popular and very useful reference. One of the most insightful explanations of EM, that provides a deeper understanding of its operation than the intuition of alternating between variables, is in terms of lowerbound maximization (Neal and Hinton, 1998; Minka, 1998). In this derivation, the E-step can be interpreted as constructing a local lower-bound to the posterior distribution, whereas the M-step optimizes the bound, thereby improving the estimate for the unknowns. This is demonstrated below for a simple...

...INFORMATION TECHNOLOGY ASSIGNMENT
ON ALGORITHM
Done by
Densil Hamilton
INTRODUCTION
This Assignment was done to show the methods of algorithm. It outlines the meaning of algorithm and steps to be carried out to complete a give problem. Examples were also shown for the methods of representing algorithm.
What is an Algorithm?
An algorithm consists of a set of explicit and unambiguous...

...Three Expectations for the Next Generation of Gaming Consoles
The 2012 E3 Expo introduced gamers around the world to the newest projects and games under development. However, there was nothing announced about the next generation of consoles. As the current Xbox 360, Playstation 3, and Nintendo Wii get older, many fans have begun to question when we'll see new systems developed and released, and with those questions come expectations. Here's a look at three...

...In computer science, the analysis of algorithms is the determination of the amount of resources (such as time and storage) necessary to execute them. Most algorithms are designed to work with inputs of arbitrary length. Usually, the efficiency or running time of an algorithm is stated as a function relating the input length to the number of steps (time complexity) or storage locations (space complexity).
Algorithm analysis is an...

...Algorithm
is a list of instructions for carrying out some process step by step
Flowchart
is a diagram representing the logical sequence in which a combination of steps or operations is to be performed
Types of Flowcharts
Program Flowchart – describes graphically in detail the logical operations and steps within a program and sequence in which these steps are to be executed for the transformation of data to produce the needed output
System Flowchart – is a graphic...

...Algorithms (Pseudo Code and Flowcharts) Exercises
Exercise 1: Design an algorithm and the corresponding flowchart for adding the test scores as given below:
26, 49, 98, 87, 62, 75
Tip: The algorithm and the flowchart should illustrate the steps for solving the problem of adding six test scores where one test score is added to sum at a time. Both the algorithm and flowchart should always have a Start step at the beginning of the...

...
BUS 475 Undergraduate Internship in Business
Professor Baptiste
BUS 475 Expectations
As a Finance major at UVI, I feel I have a very fitting job. I am a service representative at First Bank. I have been in this field for five years and I honestly do love it. Now that my internship supervisor has a hold of the BUS 475 package, I think she will be paying close attention to me. She was very excited when she heard the news that she would be giving me a grade for the work...

...Wealth Maximization:
Wealth maximization has been accepted by the finance managers, because it overcomes the limitations of profit maximization. Wealth maximization means maximizing the net wealth of the company’s share holders. Wealth maximization is possible only when the company pursues policies that would increase the market value of shares of the company.
Wealth maximization is based on the concept of...