Hi Sergio, you can adjust the coarsening rather than using the defaults in
CEM. more coarse bins will generate more observations. you want to make
the choices based on the substance of the variables, and which ones are
more important to match finely on
Gary
--
*Gary King* - Albert J. Weatherhead III University Professor - Director,
IQSS <http://iq.harvard.edu/> - Harvard University
GaryKing.org - King(a)Harvard.edu - @KingGary <https://twitter.com/kinggary> -
617-500-7570 - Assistant <king-assist(a)iq.harvard.edu>du>: 617-495-9271
On Wed, Apr 5, 2017 at 10:04 AM, Sergio Salis <Sergio.Salis(a)natcen.ac.uk>
wrote:
Hi all,
I’m considering using the cem Stata programme to evaluate the impact of a
welfare-to-work programme in the UK. However, I have never used cem before
so I am trying to understand some basic issues before proceeding with the
estimation.
The first thing I’d be interested in understanding is: How does one handle
situations where after running cem the number of matched strata (and units
within them) are very small?
Applying the cem algorithm to data from a previous impact evaluation I get:
Number of strata: 8883
Number of matched strata: 132
0 1
All 8208 1584
Matched 179 141
Unmatched 8029 1443
If I calculate the ATT using cem matched data I get an impact estimate
which is positive (around 5ppts; based on 320 obs only) while using
psmatch2 on all data (i.e. not only those in cem matched strata; around
8,237 obs are used) with kernel weights I get an estimate of around
-5.7ppts. This means I reach opposite conclusions about the impact of the
programme of interest using cem and psmatch2.
I understand the cem-based estimates are based on better matched data
(i.e. produce less biased estimates) compared to my psmatch2 estimate with
kernel weights) but this comes at the expense of external validity:
inference on the initial population is made based on a very small subset of
data (estimates based on cem are not statistically significant while my
original estimate was highly significant). Any advice about how one can
handle situations of this type?
Many thanks,
Sergio
NatCen Social Research
35 Northampton Square
London EC1V 0AX
020 7250 1866
Visit our website.
www.natcen.ac.uk
Read our latest blog.
http://www.natcen.ac.uk/blog
Follow us. @NatCen <https://twitter.com/natcen>
Email us. <info@natcen.ac.uk>info@natcen.ac.uk
NatCen Social Research is certificated to ISO/IEC 27001:2013 for
Information Security Management Systems and to ISO 20252:2012, the
international standard for market, opinion and social research.
Company limited by guarantee. Registered in England No. 4392418. Charity
registered in England and Wales (1091768) and in Scotland (SC038454).
Confidentiality: The information in this email and any attachments are
confidential and may include some that is legally privileged. It must not
be disclosed to or used by persons other than the intended recipient. If
received in error, please notify us immediately and then delete this
document.
Content: Any views or opinions expressed do not necessarily represent
those of NatCen Social Research. Please note the content of this e-mail may
be intercepted, monitored or recorded for compliance purposes. Sensitive
personal data should not normally be transmitted by e-mail.
Copyright: Copyright in this e-mail and any attachments created by NatCen
Social Research belong to NatCen Social Research unless otherwise stated.
Care: NatCen Social Research shall not be liable to the recipient or any
third party for any loss or damage howsoever arising from this e-mail
and/or its content, including loss or damage caused by virus. It is the
responsibility of the recipient to ensure the opening or use of this
message and any attachments shall not adversely affect systems or data.