EECS 491 Assignment 1

140 points total.

Submitting assignments to Canvas

For jupyter notebooks, submit the .ipynb file and a pdf export of the notebook.

Make sure you check that the pdf export represents the latest state of your notebook and that the equations

and figures are properly rendered.

If your are not using notebooks, writeup your assignment using latex and submit a pdf with your code. The

writeup should include relevant code with description if it can fit on a page.

Use the following format for filenames:

EECS491-A1-yourcaseid.ipynb

EECS491-A1-yourcaseid.pdf

If you have more than these two files, put any additional files in a directory named EECS491-A1-

yourcaseid . Do not include binaries or large data files. Then zip this directory and submit it with the name

EECS491-A1-yourcaseid.zip . Do not use other compression formats. The .ipynb file can be included in

the zipped directory, but make sure you submit the .pdf file along with the .zip file. This is so it appears at

the top level on canvas, which allows for easier grading.

Some of questions below aren’t specified in great detail and you may need to spend sometime making sense of the

questions themselves, which you can do from the reads and other sources. You also might need to fill in some

blanks or make some assumptions. The spirit behind this approach is explained in The Problem with Problems by

Eric Mazur, which I encourage everyone to read.

Put your name and Case ID here

Q1. Basic probability (10 pts)

In the proofs below you should use general probability distributions (as opposed to specific examples) and the

basic laws of probability. Be concise and clear. The proof should be in terms of mathematical facts of probability

theory.

1.1. Prove (5 pts)

1.2. Prove (5 pts)

Q2. Independence (10 pts)

Again these proofs should use general probability distributions and the basic laws of probability. Note that the proof

should be in terms of mathematical facts. It should not be an argument that depends on real-world knowledge. The

example should use common real-world knowledge and interpretation should convey the ideas of the proof.

2.1 Prove that independence is not transitive, i.e. . Define a joint probability distribution

for which the previous expression holds and provide an example with an interpretation. (5 pts)

2.2 Prove that conditional independence does not imply marginal independence, i.e. . Again

provide an example that illustrates the statement. (5 pts)

Q3. Inspector Clouseau re-revisited (20 pts)

3.1 Write a program to evaluate in Example 1.3 in Barber. Write your code and choose your data

representations so that it is easy to use it to solve the remaining questions. Show that it correctly computes the

value in the example. (5 pts)

3.2 Define a different distribution for . Your new distribution should result in the outcome that

is either or , i.e. reasonably strong evidence. Use the original values of and from the

example. Provide (invent) a reasonble justification for the value of each entry in . (5 pts)

3.3 Derive the equation for . (5 pts)

3.4 Calculate it’s value for both the original and the one you defined yourself. Is it possible to provide a

summary of the main factors that contributed to the value? Why/Why not? Explain. (5 pts)

Q4. Biased views (20 pts)

4.1 Write a program that calculates the posterior distribution of the (probability of heads) from the Binomial

distribution given heads out of trials. Feel to use a package where the necessary distributions are defined as

primitives. (5 pts)

4.2 Imagine three different views on the coin bias:

“I believe strongly that the coin is biased to either mostly heads or mostly tails.”

“I believe strongly that the coin is unbiased”.

“I don’t know anything about the bias of the coin.”

Define and plot prior distributions that expresses each of these beliefs. Provide a brief explanation. (5 pts)

4.3 Perform Bernoulli trials where one of these views is correct. Show how the posterior distribution of changes

for each view for =0, 1, 2, 5, 10, and 100. Each view should have its own plot, but with the curves of the posterior

after different numbers of trials overlayed. (5 pts)

4.4 Is it possible that each view will always arrive at an accurate estimate of ? How might you determine which

view is most consistent with the data after trials? (5 pts)

Q5. Inference using the Poisson distribution (20 pts)

Suppose you observe for 3 seconds and detect a series of events that occur at the following times (in seconds):

0.53, 0.65, 0.91, 1.19, 1.30, 1.33, 1.90, 2.01, 2.48.

5.1 Model the rate at which the events are produced using a Poisson distribution where is the number of events

observed per unit time (1 second). Show the likelihood equation and plot it for three different values of : less,

about equal, and greater than what you estimate (intuitively) from the data. (5 pts)

5.2 Derive the posterior distribution of assuming a Gamma prior (usually defined with parameters and ). The

posterior should have the form where is the total duration of the observation period and is the

number of events observed within that period. (5 pts)

5.3 Show that the Gamma distribution is a conjugate prior for the Poisson distribution, i.e. it is also a Gamma

distribution, but defined by parameters and that are functions of the prior and likelihood parameters. (5 pts)

5.4 Plot the posterior distribution for the data above at times = 0, 0.5, and 1.5. Overlay the curves on a single plot.

Comment how it is possible for your beliefs to change even though no new events have been observed. (5 pts)

Q6. Probability Distribution Example (20 pts)

In this problem you will illustrate a probability distribution in a settings of your choosing. It can be discrete or

continuous. This is meant to be a simpler version of the letter seqeunce example shown in class (so don’t use that).

Your example should use two random variables that each have at least three distinct values (if it is discrete), i.e.

don’t use binary variables. The variables should not be independent, in other words, the setting you are modeling

should have structure, and ideally structure that is interesting and interpretable in some way. Your example should

include the following:

a decription of the setting

an illustration of the joint probability and how it captures the structure

an illustration of a conditional probability

an illustration of marginal probability

Note that “illustration” here means to explain with tables or figures that convey the ideas of the mathematical

operations. The motivation behind this exercise is to help you develop a better understanding of how joint

probability distributions model probabilistic structure in a simplified setting, so try to choose something you are

very familiar with. If find this is getting too long, you can continue it as part of the exploration, but there you will

also need to add and inference problem.

Exploration (40 pts)

In these problems, you are meant to do creative exploration. Define and explore:

E.1 A discrete inference problem (20 pts)

E.2 A continuous inference problem (20 pts)

This is meant to be open-ended; you should not feel the need to write a book chapter; but neither should you just

change the numbers in one of the problems above. After doing the readings and problems above, you should pick a

concept you want to understand better or an simple modeling idea you want to try out. You can also start to explore

ideas for your project. The general idea is for you to teach yourself (and potentially a classate) about a concept

from the assignments and readings or solidify your understanding of required technical background. For additional

guidance, see the grading rubric below.

You can use the readings and other sources for inspiration, but here are a few ideas:

An inference problem using categorical data

A disease for which there are two different tests

A two-dimensional continuous inference problem

The idea of a conjugate prior

Exploration Grading Rubric

Exploration problems will be graded according the elements in the table below. The scores in the column headers

indicate the number of points possible for each rubric element (given in the rows). A score of zero for an element is

possible if it is missing entirely.

Substandard (+1) Basic (+2) Good (+3) Excellent (+5)

Pedagogical

Value

No clear statement of

idea or concept being

explored or explained;

lack of motivating

questions.

Simple problem with

adequate motivation;

still could be a useful

addition to an

assignment.

Good choice of problem with

effective illustrations of

concept(s). Demonstrates a

deeper level of understanding.

Problem also illustrates or

clarifies common conceptual

difficulties or

misconceptions.

Novelty of

Ideas

Copies existing

problem or makes only

a trivial modification;

lack of citation(s) for

source of inspiration.

Concepts are similar to

those covered in the

assignment but with

some modifications of

an existing exericse.

Ideas have clear pedagogical

motivation; creates different

type of problem or exercise to

explore related or foundational

concepts more deeply.

Applies a technique or

explores concept not

covered in the assignment or

not discussed at length in

lecture.

Clarity of

Explanation

Little or confusing

explanation; figures

lack labels or useful

captions; no

explanation of

motivations.

Explanations are

present, but unclear,

unfocused, wordy or

contain too much

technical detail.

Clear and concise explanations

of key ideas and motivations.

Also clear and concise, but

includes illustrative figures;

could be read and

understood by students from

a variety of backgrounds.

Depth of

Exploration

Content is obvious or

closely imitates

assignment problems.

Uses existing problem

for different data.

Applies a variation of a

technique to solve a problem

with an interesting motivation;

explores a concept in a series of

related problems.

Applies several concepts or

techniques; has clear focus

of inquiry that is approached

from multiple directions.

p(x, y|z) = p(x|z)p(y|x, z)

p(x|y, z) = p(y|x, z)p(x|z)

p(y|z)

a ⊥ b ∧ b ⊥ c ⇏ a ⊥ c

p(a, b, c)

a ⊥ b|c ⇏ a ⊥ b

p(B|K)

p(K|M, B) p(B|K)

< 0.1 > 0.9 p(B) p(M)

p(K|M, B)

p(M|K)

p(K|M, B)

θ

y n

θ

n

θ

n

λ n

λ

λ α β

p(λ|n, T, α, β) T n

α′ β′

T

In [ ]:

Sale!