Some clarification:
(1) Yes, MLEs are invariant to re-parameterization (i.e. going from an
bounded to an unbounded and back to an bounded scale when estimating your
parameters). But, measures of uncertainty are not invariant to
re-parameterization. There are two main ways to get your standard errors
(and confidence intervals) back to the original scale after you have
estimated your model by re-parameterizing your parameters. (1) The delta
method -- this is an analytical method to get the variance for our
parameters back to the original scale. Basically, it involves some math to
convert what we know about the distribution of our estimates on the
unbounded scale -- that is that they are distributed N(theta, Var(theta)) --
to get to what we want to know about the distribution of our estimates on
the bounded scale -- that is that they are distributed N(g(theta),
g(var(theta))). (2) The simpler way is to just simulate our parameters. We
will take repeated draws of our parameters from their distribution on the
unbounded scale (i.e. N(theta, sigma.sq) ) and convert each of these draws
back to the bounded scale. We can then simply calculate our standard errors
and confidence intervals by using this simulated sampling distribution.
(2) In PS 5 question 1.2.b, we recommend you use the "L-BFGS-B" method to
get your estimates on the original bounded scale. However, if you are
interested in the delta method, we encourage you to learn it and use it to
answer this question.
Best,
Miya
On Mon, Mar 9, 2009 at 9:44 PM, Lee, Clarence <clee at hbs.edu> wrote:
I agree with Olena in that you need to use L-BFGS-B
to get over that.
My question is more of a ?good-standard? practice:
I notice that the problem to the reparametrization trick is that the SE?s
gets messed up when we reparamterize ? which seems to be a bit kludgy since
usually we care about the mean and the SE?s. If reparamterization is a
standard technique that most statistician use, then is there a standard
technique with BFGS that helps us find the SE? (e.g. without specifying the
bounds using L-BFGS-B)
Any insights?
Thanks,
Clarence
*From:* gov2001-l-bounces at
lists.fas.harvard.edu [mailto:
gov2001-l-bounces at
lists.fas.harvard.edu] *On Behalf Of *Olena Ageyeva
*Sent:* Monday, March 09, 2009 8:46 PM
*To:* gov2001-l at
lists.fas.harvard.edu
*Subject:* Re: [gov2001-l] EXO 1
I've got the same problem with re-parametrization and then I found out that
instead re-parametrization in this particular case we are advised to use
L-BFGS-B method in optim(), so we can set a lower and an upper limits for
out point of interest. So, the point of interest will never get out of this
interval, which is sort of reparameterization, right?
The section notes says that log-likelihood (I guess it's in general for any
LL-function?) is invariant to re-parametrization (any?), but I was not much
successful applying this technique for exponential log-likelihood function.
With re-parametrization done exactly the way it's in section note I get
totally different result from what I derived analytically.
As for pnorm(opt.1000 - 1.96*se) I think that it could be done separately
for se and opt.1000 - they both are found for reparameterized function. So,
we basically looking for opt and se in different coordinate system and then
re-parameterized to get them back to our original coordinate system. Am I
right?
Sincerely,
Olena Ageyeva
Date: Mon, 9 Mar 2009 20:00:14 -0400
From: charlotte.cavaille at
gmail.com
To: gov2001-l at
lists.fas.harvard.edu
Subject: [gov2001-l] EXO 1
hi!
Quick (but fundamental question)
in exercise 1 gamma needs to be positive, so I re-parameterized...In
order to get the right MLE from optim() i didn't
forget to "re-parameterize" again...and i get the same result as the
one i found analytically. However, things change when i look for the
SE. Indeed, I get different answers analytically and with R (using the
hessian). I know this comes from the re-parameterization because I
don't have this issue when I change the whole function and do not
re-parameterize gamma. So my question is:
- if I re-parametterize, how do I apply the transformation to the
hessian to get the right result
how does that fit with the section notes that follows, why do we take
"pnorm" (this is the first transformation that is applied in the
ll.binom function) of "opt.1000 - 1.96*se" and not of "se" for
instance???
#binomial log-likelihood (N = # of trials for each observation)
ll.binom <- function(par, y, N){
# reparameterize pi; only search over [0,1]
p <- pnorm(par)
# log-likelihood
out <- sum(y*log(p) + (N-y)*log(1-p))
return(out)
}
# compare to wald ci
se <- sqrt(solve(-optim(par=2, fn=ll.binom, y=samp.1000, N=10,
method="BFGS",
control=list(fnscale=-1), hessian=T)$hessian))
#$
wald.ci <- c( pnorm(opt.1000 - 1.96*se), pnorm(opt.1000 + 1.96*se))
wald.ci # 0.7364839 0.7535663
- if i do not re-parameterize in order to be done with it and have
both my analytical
and my R result fit, how can i justify i am not re-parameterizing gamma?!
thanks!
charlotte
_______________________________________________
gov2001-l mailing list
gov2001-l at
lists.fas.harvard.edu
http://lists.fas.harvard.edu/mailman/listinfo/gov2001-l
------------------------------
Windows Live? Groups: Create an online spot for your favorite groups to
meet. Check it
out.<http://windowslive.com/online/groups?ocid=TXT_TAGLM_WL_groups_03200…
_______________________________________________
gov2001-l mailing list
gov2001-l at
lists.fas.harvard.edu
http://lists.fas.harvard.edu/mailman/listinfo/gov2001-l
--
Miya Woolfalk
Ph.D. Student
Harvard University
Government and Social Policy