1 Mar
2005
1 Mar
'05
6:44 p.m.
Hello:
Have standards evolved regarding how to report results derived from imputed
data generated by Amelia or other EM / MI methods?
At a minimum, it seems that the analyst should report (or at least have
readily available in appendix form):
* the proportion of data missing on each variable in the original
dataset, and the extent to which each variable is jointly observed with
other variables
* the variables used in the imputation models, and documentation that
findings are robust to different imputation models
* the number of datasets imputed and the estimated corresponding gain
in efficiency (per Rubin 1987)
* any ridge priors employed, and any changes to the convergence
criteria used (e.g., changing the tolerance for convergence, or limiting the
number of iterations of EM)
What do Amelia's authors think of these criteria? Are there others we
should be aware of? Or, alternately, is this overkill?
Patrick Egan
----------------------------------------------------------------------
Patrick J. Egan
Ph.D. Candidate
Department of Political Science
University of California, Berkeley
http://socrates.berkeley.edu/~pjegan
2 Mar
2 Mar
9:09 a.m.
Good questions. I don't think practices have evolved much in using any
type of MI methods, but these would be good additions. The 2 issues your
suggestions imply address (1) replicability of the imputation model
results (in which case we need to know the options chosen), and (2) model
dependence (for which we need to know how much imputating was done overall
or by variable). Finding ways of meeting these 2 rules seem worthwhile in
any modeling exercise, including multiple imputation.
Gary
: Gary King, King(a)Harvard.Edu http://GKing.Harvard.Edu :
: Institute for Quantitative Social Science Direct (617) 495-2027 :
: 34 Kirkland Street, Rm. 2 Assistant (617) 495-9271 :
: Harvard University, Cambridge, MA 02138 eFax (617) 812-8581 :
On Tue, 1 Mar 2005, Patrick Egan wrote:
Hello:
Have standards evolved regarding how to report results derived from imputed
data generated by Amelia or other EM / MI methods?
At a minimum, it seems that the analyst should report (or at least have
readily available in appendix form):
* the proportion of data missing on each variable in the original
dataset, and the extent to which each variable is jointly observed with
other variables
* the variables used in the imputation models, and documentation that
findings are robust to different imputation models
* the number of datasets imputed and the estimated corresponding gain
in efficiency (per Rubin 1987)
* any ridge priors employed, and any changes to the convergence
criteria used (e.g., changing the tolerance for convergence, or limiting the
number of iterations of EM)
What do Amelia's authors think of these criteria? Are there others we
should be aware of? Or, alternately, is this overkill?
Patrick Egan
----------------------------------------------------------------------
Patrick J. Egan
Ph.D. Candidate
Department of Political Science
University of California, Berkeley
http://socrates.berkeley.edu/~pjegan
-
Amelia mailing list served by Harvard-MIT Data Center
[Un]Subscribe/View Archive: http://lists.hmdc.harvard.edu/?info=amelia
4 Mar
4 Mar
8:20 a.m.
These seem well considered. I would believe that these are more than are
"standard" but not in any sense excessive, especially if you are
interested in having future readers replicate your work.
Slightly more precisely, for the third thing you list, the "estimated
efficiency" from the number of imputed datasets, you would need to
calculate the "missing information" measure. If you knew this, that is
probably more useful (though maybe less understandable to a general
audience) than the "proportion of data missing on each variable" which you
mention first.
If you were looking for a very complete appendix, you might also want to
include tests for the type of missingness, for example tests between MAR
and MCAR (most easily done by seeing if the missingness matrix can be
predicted by any of the covariates) or tests for Non-Ignorability (these
often can only be computed in datasets with small numbers of variables,
see for example http://www.hss.caltech.edu/~sherman/odds.pdf). Also, you
might compare the coefficients you get with or without imputation, which
could signal to your reader whether you have corrected bias, or simply
increased efficiency (or both). I have some graphical methods for this,
if you are interested.
regards,
James Honaker.
On Tue, 1 Mar 2005, Patrick Egan wrote:
Hello:
Have standards evolved regarding how to report results derived from imputed
data generated by Amelia or other EM / MI methods?
At a minimum, it seems that the analyst should report (or at least have
readily available in appendix form):
* the proportion of data missing on each variable in the original
dataset, and the extent to which each variable is jointly observed with
other variables
* the variables used in the imputation models, and documentation that
findings are robust to different imputation models
* the number of datasets imputed and the estimated corresponding gain
in efficiency (per Rubin 1987)
* any ridge priors employed, and any changes to the convergence
criteria used (e.g., changing the tolerance for convergence, or limiting the
number of iterations of EM)
What do Amelia's authors think of these criteria? Are there others we
should be aware of? Or, alternately, is this overkill?
Patrick Egan
----------------------------------------------------------------------
Patrick J. Egan
Ph.D. Candidate
Department of Political Science
University of California, Berkeley
http://socrates.berkeley.edu/~pjegan
-
Amelia mailing list served by Harvard-MIT Data Center
[Un]Subscribe/View Archive: http://lists.hmdc.harvard.edu/?info=amelia
6998
days inactive
7001
days old
amelia@lists.gking.harvard.edu
2 comments
3 participants
participants (3)
-
Gary King -
James Honaker -
Patrick Egan