The Expectation Maximization Algorithm
Frank Dellaert
College of Computing, Georgia Institute of Technology
Technical Report number GIT-GVU-02-20
February 2002
Abstract
This note represents my attempt at explaining the EM algorithm (Hartley, 1958; Dempster et al., 1977; McLachlan and Krishnan, 1997). This is just a slight variation on Tom Minka’s tutorial (Minka, 1998), perhaps a little easier (or perhaps not). It includes a graphical example to provide some intuition. 1 Intuitive Explanation of EM

EM is an iterative optimization method to estimate some unknown parameters, given measurement data U. However, we are not given some “hidden” nuisance variables J, which need to be integrated out. In particular, we want to maximize the posterior probability of the parameters given the data U, marginalizing over J: = argmax

X
J2Jn
P(; JjU) (1)
The intuition behind EM is an old one: alternate between estimating the unknowns and the hidden variables J. This idea has been around for a long time. However, instead of finding the best J 2 J given an estimateat each iteration, EM computes a distribution over the space J . One of the earliest papers on EM is (Hartley, 1958), but the seminal reference that formalized EM and provided a proof of convergence is the “DLR” paper by Dempster, Laird, and Rubin (Dempster et al., 1977). A recent book devoted entirely to EM and applications is (McLachlan and Krishnan, 1997), whereas (Tanner, 1996) is another popular and very useful reference. One of the most insightful explanations of EM, that provides a deeper understanding of its operation than the intuition of alternating between variables, is in terms of lowerbound maximization (Neal and Hinton, 1998; Minka, 1998). In this derivation, the E-step can be interpreted as constructing a local lower-bound to the posterior distribution, whereas the M-step optimizes the bound, thereby improving the estimate for the unknowns. This is demonstrated below for a simple...

...Algorithms Homework – Fall 2000
8.1-1 Using Figure 8.1 as a model, illustrate the operation of PARTITION on the array A =
13 19 9 5 12 8 7 4 11 2 6 21
i j j
6 19 9 5 12 8 7 4 11 2 13 21
i i j j
6 2 9 5 12 8 7 4 11 19 13 21
i ………………………… j
return 11, SPLIT = and
8.1-2 What value of q does PARTITION return when all elements in the array A[p…r] have the same value?
q = (p+r)/2, where p = index 0, and r = highest index
8.1-3 Give a brief argument that the running time of PARTITION on a subarray of size n is (n).
In the worst case, PARTITION must move the j pointer by one element (to the 2nd to last element), and the i pointer all the way to j, making a comparison at each element along the way. Since there are n comparisons made, the running time is (n)
In the average (and best) case, PARTITION must move the j pointer to an element at or near the half-way point in the array and the i pointer all the way to j, making a comparison at each element along the way. Once again there are n comparisons made and the running time is (n)
8.2-1 Show that the running time of QUICKSORT is (n lg n) when all elements of array A have the same value.
T(n) =...

...INFORMATION TECHNOLOGY ASSIGNMENT
ON ALGORITHM
Done by
Densil Hamilton
INTRODUCTION
This Assignment was done to show the methods of algorithm. It outlines the meaning of algorithm and steps to be carried out to complete a give problem. Examples were also shown for the methods of representing algorithm.
What is an Algorithm?
An algorithm consists of a set of explicit and unambiguous finite steps which, when carried out for a given set of initial conditions, produce the corresponding output and terminate in finite time. (How to Solve it by Computer, RG Dromey, Prentice Hall UK, 1982)
This is done by a series of steps:
1. Input: there are zero or more quantities which are externally supplied;
2. Output: at least one quantity is produced;
3. Definiteness: each instruction must be clear and unambiguous;
4. Finiteness: if we trace out the instructions of an algorithm, then for all cases the algorithm will terminate after a finite number of steps;
5. Effectiveness: every instruction must be sufficiently basic that a person using only pencil and paper can in principle carry it out. It is not enough that each operation is definite, but it must also be feasible.
WAYS OF REPRESENTING ALGORITHMS
Two ways of represent an algorithm are:
Flowcharts
Pseudo Code
FLOWCHARTS
This is a...

...Three Expectations for the Next Generation of Gaming Consoles
The 2012 E3 Expo introduced gamers around the world to the newest projects and games under development. However, there was nothing announced about the next generation of consoles. As the current Xbox 360, Playstation 3, and Nintendo Wii get older, many fans have begun to question when we'll see new systems developed and released, and with those questions come expectations. Here's a look at threeexpectations that I have for the next generation of video game consoles:
Have the accessories ready
The Xbox Kinect was a brilliant idea, and it's certainly helped expand the lifespan of the current platform. I just wish that Microsoft had developed that technology from the very beginning. After all, Nintendo had a handle on it with the Wii, so we know that it was possible for the accessory to be available upon the Xbox 360's release.
There are certain things that we know will be improved on all of the next-gen platforms when they are released. We know they will be faster, and we know they will have updated disc-reading technology, for example. But it would be nice to have all of the accessories like the next-generation version of Kinect ready to go when the consoles launch. Now that we've grown spoiled by these fun new additions to the normal platform experience, it would be disappointing to make us wait for them again.
Online games for the console
We've talked about it...

...In computer science, the analysis of algorithms is the determination of the amount of resources (such as time and storage) necessary to execute them. Most algorithms are designed to work with inputs of arbitrary length. Usually, the efficiency or running time of an algorithm is stated as a function relating the input length to the number of steps (time complexity) or storage locations (space complexity).
Algorithm analysis is an important part of a broader computational complexity theory, which provides theoretical estimates for the resources needed by any algorithm which solves a given computational problem. These estimates provide an insight into reasonable directions of search for efficient algorithms.
In theoretical analysis of algorithms it is common to estimate their complexity in the asymptotic sense, i.e., to estimate the complexity function for arbitrarily large input. Big O notation, Big-omega notation and Big-theta notation are used to this end. For instance, binary search is said to run in a number of steps proportional to the logarithm of the length of the list being searched, or in O(log(n)), colloquially "in logarithmic time". Usually asymptotic estimates are used because different implementations of the same algorithm may differ in efficiency. However the efficiencies of any two "reasonable" implementations of a given algorithm...

...Algorithm
is a list of instructions for carrying out some process step by step
Flowchart
is a diagram representing the logical sequence in which a combination of steps or operations is to be performed
Types of Flowcharts
Program Flowchart – describes graphically in detail the logical operations and steps within a program and sequence in which these steps are to be executed for the transformation of data to produce the needed output
System Flowchart – is a graphic representation of the procedures involved in converting data on input media to data in output form
Flowcharting Symbols
[pic]
Basic Control Structure
SEQUENCE – process is executed from one to another in a straight forward manner
SELECTION – a choice is provided between two alternatives based on conditions outcome as either TRUE or FALSE
ITERATION – or loop, enables certain operations/tasks/instructions to repeat. These tasks are repeated based on conditions outcome as either TRUE or FALSE
Sequence
[pic]
Examples
The length of a rectangle is equal to 5cm and its width is equals to 3cm. Create a flowchart that will display the area and perimeter of the rectangle.
Given three numbers A=3, B=6, C=9, and D=5. Draw a flowchart to compute and print out the sum, average, and product of the given values.
Selection
[pic]
Examples
Create a flowchart for a program that will accept two inputted values. And based on the inputted values the program will display the...

...Algorithms (Pseudo Code and Flowcharts) Exercises
Exercise 1: Design an algorithm and the corresponding flowchart for adding the test scores as given below:
26, 49, 98, 87, 62, 75
Tip: The algorithm and the flowchart should illustrate the steps for solving the problem of adding six test scores where one test score is added to sum at a time. Both the algorithm and flowchart should always have a Start step at the beginning of the algorithm or flowchart and at least one stop step at the end, or anywhere in the algorithm or flowchart.
Since we want the sum of six test score, then we should have a container for the resulting sum. In this example, the container should be called sum and we must make sure that sum should start with a zero value by step 2.
Exercise 2: The problem with the above algorithm is that some of the steps appear more than once example step 5 - get second number, step 7 - get third number, etc. Your job is to try and shorten the algorithm and the flowchart.
Tip: This algorithm and its corresponding flowchart should be a bit shorter than the first one. In this algorithm, step 3 to 5 will be repeated, where a number is obtained and added to sum. Similarly the flowchart should indicate a flow line being drawn back to the previous step indicating that the portion of the flowchart is being repeated....

...
BUS 475 Undergraduate Internship in Business
Professor Baptiste
BUS 475 Expectations
As a Finance major at UVI, I feel I have a very fitting job. I am a service representative at First Bank. I have been in this field for five years and I honestly do love it. Now that my internship supervisor has a hold of the BUS 475 package, I think she will be paying close attention to me. She was very excited when she heard the news that she would be giving me a grade for the work I put forth. I expect her to put a little more pressure on me, which is fine. I actually work a lot better under pressure. Currently I am on probation for the head teller position. My tardiness and attendance record needs to improve and this internship package will push me in the right direction to get the job done. At work right now, I am solely responsible for the end of day branch balancing and I also maintain our two ATM machines. Working at First Bank has helped my customer service skills and it has also helped my timid manner. I have been through various trainings such as FDIC and FinCEn compliance that will help me. These qualities along with my clerical and banking knowledge will help me on my future career path.
I have set two goals for myself during this internship period. The scheduled time to report to work is 7:45 AM with a seven minute grace period. My goal is to be there before 7:45 without use of the grace period to approve my attendance record. Also, as in any...

...Wealth Maximization:
Wealth maximization has been accepted by the finance managers, because it overcomes the limitations of profit maximization. Wealth maximization means maximizing the net wealth of the company’s share holders. Wealth maximization is possible only when the company pursues policies that would increase the market value of shares of the company.
Wealth maximization is based on the concept of cash flows. Cash flows are a reality and not based on any subjective interpretation. On the other hand there are many subjective elements in the concept of profit maximization.
It considers time value of money translates cash flows occurring of different periods into a comparable value of cash flows is considered critically in all decisions as it incorporates the risk associated with the cash flow stream.
About 3i Infotech
3i Infotech is one of the top 4 Indian Software Products Companies.* The company provides software products and IT services (Managed IT Services, Application Software Development & Maintenance, Payment Services, Business Intelligence, IT Consulting, BPO, Document Imaging & Digitization and Data Warehousing) for the Insurance, Banking, Capital Markets, Mutual Funds and Government verticals. The company services customers in over 50 countries across 5 continents.
FINANCIAL HIGHLIGHTS
Consolidated results for the year ended March 31, 2008
1....