tag:blogger.com,1999:blog-367845512024-03-17T05:15:05.811+00:00Christopher Gandrud (간드루드 크리스토파)Unknownnoreply@blogger.comBlogger84125tag:blogger.com,1999:blog-36784551.post-18052720440027008842017-02-27T14:17:00.001+00:002017-02-27T14:17:56.405+00:00Posting ElsewhereFYI: most of my blog-type writing on software development and political economy is now on <a href="https://medium.com/@chrisgandrud">Medium</a> or also at <a href="http://bruegel.org/author/christopher-gandrud/">Bruegel</a> (only political economy at Bruegel). Unknownnoreply@blogger.com5tag:blogger.com,1999:blog-36784551.post-6279944838268184302015-12-30T15:46:00.000+00:002015-12-30T15:46:27.669+00:00End of 2015 Blog Roundup<p>Over the past few months I've mostly been blogging at a number of other venues. These include:</p>
<ul>
<li><p>A piece with Mark Hallerberg in <a href="http://www.democraticaudit.com/?p=16085">Democracy Audit UK</a> summarising <a href="http://rap.sagepub.com/content/2/3/2053168015589335">our research</a> on how, despite previous findings, democratic governments run similarly sizable bank bailout tabs as autocracies. This wasn't noticed in previous work, because democratic governments have incentives (possiblilty of losing elections) to shift the realisation of these costs into the future.</p>
</li>
<li><p>A post over at <a href="http://bruegel.org/2015/12/financial-regulatory-transparency-new-data-and-implications-for-eu-policy">Bruegel</a> introducing the Financial Supervisory Transparency Index that Mark Copelovitch, Mark Hallerberg, and I created. We also discuss supervisory transparency's implications for a European capital markets union.</p>
</li>
<li><p>At <a href="http://voxukraine.org/2015/12/24/causes-and-possible-solutions-to-brawling-in-the-ukrainian-parliament-en-2/">VoxUkraine, I discuss</a> the causes and possible solutions to brawling in the Ukranian parliament based on my <a href="http://jpr.sagepub.com/content/early/2015/11/02/0022343315604707.abstract">recent research in the <em>Journal of Peace Research</em></a>.</p>
</li>
<li><p>I didn't write this one, but my co-author Tom Pepinsky, <a href="http://tompepinsky.com/2015/12/21/another-reason-why-predicting-financial-crises-is-hard/">wrote a nice piece</a> about a new working paper we have on the (difficulty) of predicting financial crises.</p>
</li>
</ul>Unknownnoreply@blogger.com4tag:blogger.com,1999:blog-36784551.post-73921015356363081422015-06-20T09:17:00.000+01:002015-06-20T09:17:07.443+01:00More Corrections the the DPI’s yrcurnt Election Timing Variable: OECD Edition<p>Previously on The Political Methodologist, I <a href="http://thepoliticalmethodologist.com/2015/03/03/corrections-and-refinements-to-the-database-of-political-institutions-yrcurnt-election-timing-variable/">posted</a> updates to the <a href="http://go.worldbank.org/2EAGGLRZ40">Database of Political Institutions'</a> election timing variable: <code>yrcurnt</code>. That set of corrections was only for the current 28 EU member states.</p>
<p>I’ve now expanded the corrections to include most other OECD countries.<a href="#fn1" class="footnoteRef" id="fnref1"><sup>1</sup></a> Again, there were many missing elections:</p>
<div id="change-list" class="section level2">
<h2>Change list</h2>
<table>
<thead>
<tr class="header">
<th align="left">Country</th>
<th align="left">Changes</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td align="left">Australia</td>
<td align="left">Corrects missing 1998 election year.</td>
</tr>
<tr class="even">
<td align="left">Canada</td>
<td align="left">Corrects missing 2000, 2006, 2008, 2011 election years.</td>
</tr>
<tr class="odd">
<td align="left">Iceland</td>
<td align="left">Corrects missing 2009 election year.</td>
</tr>
<tr class="even">
<td align="left">Ireland</td>
<td align="left">Corrects missing 2011 election.</td>
</tr>
<tr class="odd">
<td align="left">Japan</td>
<td align="left">Corrects missing 2005 and 2012 elections. Corrects misclassification of the 2003 and 2009 elections, which were originally erroneously labeled as being in 2004 and 2008, respectively.</td>
</tr>
</tbody>
</table>
</div>
<div id="import-into-r" class="section level2">
<h2> Import into R</h2>
<p>To import the most recent corrected version of the data into R simply use:</p>
<pre class="r"><code>election_time <- rio::import('https://raw.githubusercontent.com/christophergandrud/yrcurnt_corrected/master/data/yrcurnt_original_corrected.csv')</code></pre>
</div>
<div class="footnotes">
<hr />
<ol>
<li id="fn1"><p>Australia, Canada, Iceland, Israel, Japan, South Korea, New Zealand, Norway, Switzerland, USA<a href="#fnref1">↩</a></p></li>
</ol>
</div>Unknownnoreply@blogger.com4tag:blogger.com,1999:blog-36784551.post-56092916673815219262015-05-08T07:41:00.000+01:002015-05-08T07:41:41.918+01:00A Link Between topicmodels LDA and LDAvis<p>Carson Sievert and Kenny Shirley have put together the really nice <a href="https://github.com/cpsievert/LDAvis">LDAvis</a> R package. It provides a Shiny-based interactive interface for exploring the output from Latent Dirichlet Allocation topic models. If you've never used it, I highly recommend checking out their <a href="http://cpsievert.github.io/xkcd/">XKCD example</a> (this <a href="http://nlp.stanford.edu/events/illvi2014/papers/sievert-illvi2014.pdf">paper</a> also has some nice background).</p>
<p>LDAvis doesn't fit topic models, it just visualises the output. As such it is agnostic about what package you use to fit your LDA topic model. They have a useful <a href="http://cpsievert.github.io/LDAvis/reviews/reviews.html">example</a> of how to use output from the <a href="http://cran.r-project.org/web/packages/lda/lda.pdf">lda</a> package.</p>
<p>I wanted to use LDAvis with output from the <a href="http://cran.r-project.org/web/packages/topicmodels/index.html">topicmodels</a> package. It works really nicely with texts preprocessed using the <a href="http://cran.r-project.org/web/packages/tm/index.html">tm</a> package. The trick is extracting the information LDAvis requires from the model and placing it into a specifically structured JSON formatted object.</p>
<p>To make the conversion from topicmodels output to LDAvis JSON input easier, I created a linking function called <code>topicmodels_json_ldavis</code>. The full function is below. To use it follow these steps:</p>
<ol>
<li><p>Create a <code>VCorpus</code> object using the tm package's <code>Corpus</code> function.</p>
</li>
<li><p>Convert this to a document term matrix using <code>DocumentTermMatrix</code>, also from tm.</p>
</li>
<li><p>Run your model using topicmodel's <code>LDA</code> function.</p>
</li>
<li><p>Convert the output into JSON format using <code>topicmodels_json_ldavis</code>. The function requires the output from steps 1-3.</p>
</li>
<li><p>Visualise with LDAvis' <code>serVis</code>.</p>
</li>
</ol>
<script src="https://gist.github.com/christophergandrud/00e7451c16439421b24a.js"></script>Unknownnoreply@blogger.com21tag:blogger.com,1999:blog-36784551.post-37546467075759020422014-12-16T14:36:00.000+00:002014-12-16T14:36:36.090+00:00Simulated or Real: What type of data should we use when teaching social science statistics?<p>I just finished teaching a new course on collaborative data science to social science students. The materials are <a href="https://github.com/HertieDataScience2014/SyllabusAndLectures">on GitHub</a> if you're interested.</p>
<h2 id="what-did-we-do-and-why-">What did we do and why?</h2>
<p>Maybe the most unusual thing about this class from a statistics pedagogy perspective was that it was entirely focused on real world data; data that the students gathered themselves. I gave them virtually no instruction on what data to gather. They gathered data they felt would help them answer their research questions.</p>
<p>Students directly confronted the data warts that usually consume a large proportion of researchers' actual time. My intention was that the students systematically learn <a href="http://vita.had.co.nz/papers/tidy-data.pdf">tools</a> and <a href="http://biostatistics.oxfordjournals.org/content/10/3/405.full">best practices</a> for how to address these warts.</p>
<p>This is in contrast to many social scientists' statistics education. Typically, students are presented with pre-arranged data. They are then asked to perform some statistical function with it. The end.</p>
<p>This leaves students underprepared for actually using statistics in an undirected project (their thesis, in a job). Typically when confronted with data gathering and transformation issues in the real world most muddle through, piecing together ad hoc techniques as they go along in an decidedly non-efficient manner and often with <a href="http://christophergandrud.blogspot.de/2014/10/do-political-scientists-care-about.html">poor results</a>. A fair number of students will become frustrated and may never actually succeed in using any of the statistical tools they did learn.</p>
<h2 id="what-kind-of-data-">What kind of data?</h2>
<p>How does this course fit into a broader social science statistical education?</p>
<p>Zachary Jones had a <a href="http://zmjones.com/monte-carlo/">really nice post</a> the other day advocating that statistics courses use Monte Carlo simulation rather than real world data. The broad argument being that the messiness of real world data distracts students from carefully learning the statistical properties that instructors intend them to learn.</p>
<p>Superficially, it would seem that the course I just finished and Zachary's prescription are opposed. We could think of stats courses as using one of two different types of data:</p>
<p align="center"><strong>simulated</strong> --- <strong>real world</strong></p>
<h2 id="simulated-vs-real-">Simulated vs. Real?</h2>
<p>As you'll see I almost entirely agree with Zachary's post, but I think there is a more important difference between the social science statistic course <em>status quo</em> and less commonly taught courses such as mine and (what I think) Zachary is proposing. The difference is where the data comes from: is it gathered/generated by students or is it prepackaged by an instructor?</p>
<p>Many status quo courses use data that is prepackaged by instructors. Both simulated and real world data can be prepackaged. I suppose there are many motivations for this, but an important one surely is that it is easier to teach. As an instructor, you know what the results will be and you know the series of clicks or code that will generate this answer. There are no surprises. Students may also find prepackaged data comforting as they know that there is a correct answer out there. They just need to decode the series of clicks to get it.</p>
<p>Though prepackaged data is easier for instructors and students, it surely is counterproductive in terms of learning how to actually answer research questions with data analysis.</p>
<p>Students will not learn necessary skills needed to gather and transform real world data so that it can be analysed. Students who simply load a prepackaged data set of simulated values will often not understand where it came. They can succumb to the temptation to just click through until they get the right answer.</p>
<p>On the other hand I've definitely had the experience teaching with student simulated data that Zachary describes:</p>
<blockquote>
<p>I think many students find [hypothesis testing] unintuitive and end up leaving with a foggy understanding of what tests do. With simulation I don't think it is so hard to explain since you can easily show confidence interval coverage, error rates, power, etc.</p>
</blockquote>
<p>The actually important distinction in social science statistics education for thinking about what is more or less effective is:</p>
<p align="center"><strong>student gathered/generated</strong> --- <strong>instructor gathered/generated</strong></p>
<h2 id="prepackaged-vs-student-generated-data">Prepackaged vs. student generated data</h2>
<p>There is of course a pedagogical difference between data that students gathered from the real world and data they simulated with a computer. Simulated data is useful for teaching the behaviour of statistical methods. Real world data is useful for teaching students how to plan and execute a project using these methods to answer research questions in a way that is reproducible and introduces fewer data munging biases into estimates. Though almost certainly too much to take on together in one course, both should be central to a well-rounded social science statistics education.</p>Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-36784551.post-9286910582379928612014-12-10T15:12:00.000+00:002014-12-10T15:12:22.353+00:00Set up R/Stan on Amazon EC2<p>A few months ago I <a href="http://christophergandrud.blogspot.de/2014/06/simple-script-from-setting-up-r-git-and.html">posted the script</a>
that I use to set up my R/JAGS working environment on an Amazon EC2 instance.</p>
<p>Since then I've largely transitioned to using R/<a href="http://mc-stan.org/">Stan</a> to
estimate my models. So, I've updated my setup script (see below). </p>
<p>There are a few other changes:</p>
<ul>
<li><p>I don't install/use RStudio on Amazon EC2. Instead, I just use R from the terminal.
Don't get me wrong, I love RStudio. But since what I'm doing on EC2 is
just running simulations (I handle the results on my local machine), RStudio is
overkill. </p>
</li>
<li><p>I don't install git anymore. Instead I use <code>source_url</code> (from devtools) and <code>source_data</code> (from repmis) to
source scripts from GitHub. Again all of the manipulation I'm doing to these
scripts is on my local machine. </p>
</li>
</ul>
<script src="https://gist.github.com/christophergandrud/c01760a65c03f7047df0.js"></script>Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-36784551.post-48501123940631389232014-12-04T13:51:00.000+00:002014-12-04T13:51:17.483+00:00Our developing Financial Regulatory Transparency Index<p>Here is a presentation I just gave on a work in-progress. We are developing a new Financial Regulatory Transparency Index using a Bayesian IRT approach.</p>
<iframe src="//www.slideshare.net/slideshow/embed_code/42351226" width="476" height="400" frameborder="0" marginwidth="0" marginheight="0" scrolling="no"></iframe>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-36784551.post-25614301970661726942014-10-13T07:53:00.000+01:002015-01-08T13:00:08.047+00:00Do Political Scientists Care About Effect Sizes: Replication and Type M Errors<p>Reproducibility has come a long way in political science. Many major journals now require replication materials be made available either on their websites or some service such as the <a href="http://thedata.org/">Dataverse Network</a>. Most of the top journals in political science have formally committed to reproducible research best practices by signing up to the <a href="http://www.dartstatement.org/">The (DA-RT) Data Access and Research Transparency Joint Statement</a>.</p>
<p>This is certainly progress. But what are political scientists actually supposed to do with this new information? Data and code availability does help avoid effort duplication--researchers don't need to gather data or program statistical procedures that have already been gathered or programmed. It promotes <a href="http://books.google.com/books/about/Reproducible_Research_With_R_and_Rstudio.html?id=u-nuzKGvoZwC&redir_esc=y">better research habits</a>. It definitely provides ''<a href="http://onlinelibrary.wiley.com/doi/10.1111/1468-5965.00220/abstract">procedural oversight</a>''. We would be highly suspect of results from authors that were unable or unwilling to produce their code/data.</p>
<p>However, there are lots of problems that data/code availability requirements do not address. Apart from a few journals like <a href="http://journals.cambridge.org/action/displayJournal?jid=RAM">Political Science Research and Methods</a>, most journals have no standing policy to check the replication materials' veracity. Reviewers rarely have access to manuscripts' code/data. Even if they did have access to it, few reviewers would be willing or able to undertake the time consuming task of reviewing this material.</p>
<h2 id="do-political-science-journals-care-about-coding-and-data-errors-">Do political science journals care about coding and data errors?</h2>
<p>What do we do if someone replicating published research finds clear data or coding errors that have biased the published estimates?</p>
<p>Note that I'm limiting the discussion here to honest mistakes, not active attempts to deceive. <a href="http://christophergandrud.blogspot.de/2013/04/reinhart-rogoff-everyone-makes-coding.html">We all make these mistakes</a>. To keep it simple, I'm also only talking about clear, knowable, and <a href="http://polmeth.wustl.edu/mediaDetail.php?docId=1534">non</a>-<a href="http://zmjones.com/replication-reproduction-prediction/">causal</a> coding and data errors.</p>
<p>Probably the most responsible action a journal could take when clear cut coding/data biased results have been found would be to <em>directly adjoin to the original article a note detailing the bias</em>. This way readers will always be aware of the correction and will have the best information possible. This is a more efficient way of getting out corrected information than relying on some probabilistic process where readers may or may not stumble upon the information posted elsewhere.</p>
<p>As far as I know, however, no political science journal has a written procedure (please correct me if I'm wrong) for dealing with this new information. My sense is that there are a series of ad hoc responses that closely correspond to how the bias affects the results:</p>
<h3 id="statistical-significance">Statistical significance</h3>
<p>The situation where a journal is most likely to do anything is when correcting the bias makes the results no longer statistically significant. This might get a journal to append a note to the original article. But maybe not, they could just ignore it.</p>
<h3 id="sign">Sign</h3>
<p>It might be that once the coding/data bias is corrected, the sign of an estimated effect flips--the result of what Andrew Gelman calls <a href="http://andrewgelman.com/2004/12/29/type_1_type_2_t/">Type S errors</a>. I really have no idea what a journal would do in this situation. They might append a note or maybe not.</p>
<h3 id="magnitude">Magnitude</h3>
<p>Perhaps the most likely outcome of correcting honest coding/data bias is that the effect size changes. These errors would be the result of Gelman's <strong>Type M</strong> errors. My sense (and experience) is that in a context where <a href="http://edr.sagepub.com/content/early/2014/07/23/0013189X14545513.abstract">novelty is greatly privileged over facts</a> journal editors will almost certainly ignore this new information. It will be buried.</p>
<h2 id="do-political-scientists-care-about-effect-size-">Do political scientists care about effect size?</h2>
<p>Due to the complexity of what political scientists study, we rarely (perhaps with the exception of specific topics like election forecasting) think that we are very close to estimating a given effect's real magnitude. Most researchers are aiming for statistical significance and a sign that matches their theory.</p>
<p>Does this mean that we don't care about trying to estimate magnitudes as closely as possible?</p>
<p>Looking at political science practice pre-publication, there is a lot of evidence that we do care about Type M errors. Considerable effort is given to finding new estimation methods that produce less biased results. Questions of omitted variable bias are very common at research seminars and in journal reviews. Most researchers do carefully build their data sets and code to minimise coding/data bias. Many of these efforts are focused on the headline stuff--whether or not a given effect is significant and what the direction of the effect is. But, these efforts are also part of a desire to make the most accurate estimate of an effect as possible.</p>
<p>However, the review process and journals' responses to finding Type M errors caused by honest coding/data errors in published findings suggest that perhaps we don't care about effect size. Reviewers almost never look at code and data. Journals (as far as I know, please correct me if I'm wrong) never append information on replications that find Type M errors to original papers.</p>
<h2 id="prescription">Prescription</h2>
<p>I have a simple prescription for demonstrating that we actually care about estimating accurate effect sizes:</p>
<blockquote>
<p>Develop a standard practice of including a short authored write up of the data/code bias with corrected results in the original article's supplementary materials. Append a notice to the article pointing to this.</p>
</blockquote>
<p>Doing this would not only give readers more accurate effect size estimates, but also make replication materials more useful.</p>
<p>Standardising the practice of publishing authored notes will incentivise people to use replication materials, find errors, and publicly correct them.</p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-36784551.post-49903308326228134632014-07-22T10:51:00.000+01:002014-07-22T10:51:01.838+01:00Note to self: brew cleanup r<p>Note to self: after updating R with <a href="http://brew.sh/">Homebrew</a> remember to <code>cleanup</code> old versions:</p>
<pre><code class='sh'>brew cleanup r
</code></pre>
Otherwise I'm liable to get a <a href="http://xkcd.com/371/">segfault</a>. (<a href="https://sagebionetworks.jira.com/browse/SYNR-590">see also</a>)Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-36784551.post-36728672620483344712014-06-28T15:38:00.002+01:002014-06-28T15:38:51.699+01:00Simple script from setting up R, Git, and Jags on Amazon EC2 Ubuntu Instance<p>Just wanted to put up the script I've been using to create an Amazon EC2 Ubuntu instance for running RStudio, Git, and Jags. There isn't anything really new in here, but it it has been serving me well.</p>
<p>The script begins after the basic instance has been set up in the Amazon EC2 console (<a href="http://blog.yhathq.com/posts/r-in-the-cloud-part-1.html">yhat</a> has a nice post on how to do this, though some of their screenshots are a little old). Just SSH into the instance and get started. </p>
<script src="https://gist.github.com/christophergandrud/67cf1261d2fc8aefbc6f.js"></script>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-36784551.post-10899526951373109302014-06-06T10:39:00.000+01:002014-06-06T10:39:06.635+01:00Slides for a simple introduction to Bayesian data analysis<p>Here is the slide deck from a <a href="http://christophergandrud.github.io/BasicBayesianPresent/">simple introductory talk</a> I gave yesterday on Bayesian data analysis:</p>
<iframe src="http://christophergandrud.github.io/BasicBayesianPresent/" width="100%" height="500px"></iframe>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-36784551.post-26779269675484975452014-05-11T16:51:00.000+01:002014-05-11T16:51:15.027+01:00Updates to repmis: caching downloaded data and Excel data downloading<p>Over the past few months I’ve added a few improvements to the <a href="http://christophergandrud.github.io/repmis/">repmis</a>–miscellaneous functions for reproducible research–R package. I just want to briefly highlight two of them:</p>
<ul>
<li><p><strong>Caching</strong> downloaded data sets.</p></li>
<li><p><code>source_XlsxData</code> for downloading data in Excel formatted files.</p></li>
</ul>
<p>Both of these capabilities are in <strong>repmis</strong> version 0.2.9 and greater.</p>
<div id="caching" class="section level2">
<h2>Caching</h2>
<p>When working with data sourced directly from the internet, it can be time consuming (and make the data hoster angry) to repeatedly download the data. So, <strong>repmis</strong>’s <code>source</code> functions (<code>source_data</code>, <code>source_DropboxData</code>, and <code>source_XlsxData</code>) can now <a href="http://en.wikipedia.org/wiki/Cache_(computing)">cache</a> a downloaded data set by setting the argument <code>cache = TRUE</code>. For example:</p>
<pre class="r"><code>DisData <- source_data("http://bit.ly/156oQ7a", cache = TRUE)</code></pre>
<p>When the function is run again, the data set at <a href="http://bit.ly/156oQ7a">http://bit.ly/156oQ7a</a> will be loaded locally, rather than downloaded.</p>
<p>To delete the cached data set, simply run the function again with the argument <code>clearCache = TRUE</code>.</p>
</div>
<div id="source_xlsxdata" class="section level2">
<h2><code>source_XlsxData</code></h2>
<p>I recently added the <code>source_XlsxData</code> function to download Excel data sets directly into R. This function works very similarly to the other <code>source</code> functions. There are two differences:</p>
<ul>
<li><p>You need to specify the <code>sheet</code> argument. This is either the name of one specific sheet in the downloaded Excel workbook or its number (e.g. the first sheet in the workbook would be <code>sheet = 1</code>).</p></li>
<li><p>You can pass other arguments to the <a href="http://www.inside-r.org/packages/cran/xlsx/docs/read.xlsx">read.xlsx</a> function from the <a href="http://cran.r-project.org/web/packages/xlsx/index.html">xlsx</a> package.</p></li>
</ul>
<p>Here’s a simple example:</p>
<pre class="r"><code>RRurl <- 'http://www.carmenreinhart.com/user_uploads/data/22_data.xls'
RRData <- source_XlsxData(url = RRurl, sheet = 2, startRow = 5)</code></pre>
<p><code>startRow = 5</code> basically drops the first 4 rows of the sheet.</p>
</div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-36784551.post-88912835280252424402014-05-09T15:43:00.000+01:002014-05-09T15:44:10.419+01:00d3Network Plays Nice with Shiny Web Apps<p>After some delay (and because of helpful prompting by Giles Heywood and code contributions by <a href="https://github.com/johndharrison">John Harrison</a>) <a href="http://christophergandrud.github.io/d3Network/">d3Network</a> now plays nicely with <a href="http://shiny.rstudio.com/">Shiny web apps</a>. This means you can fully integrate R/D3.js network graphs into your web apps.</p>
<p>Here is what one simple <a href="https://github.com/christophergandrud/d3ShinyExample">example</a> looks like:</p>
<div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-a5mZkkM9a6k/U2zn3KW77UI/AAAAAAAAG2E/I0PXqDX6db4/s1600/d3Network_Shiny_Example.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-a5mZkkM9a6k/U2zn3KW77UI/AAAAAAAAG2E/I0PXqDX6db4/s1600/d3Network_Shiny_Example.png" /></a></div>
<p>An explanation of the code is <a href="http://christophergandrud.github.io/d3Network/#shiny">here</a> and you can download the app and play with it using:</p>
<pre><code class="r">shiny::runGitHub('d3ShinyExample', 'christophergandrud')
</code></pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-36784551.post-37117262120576651842014-05-09T11:21:00.000+01:002014-05-09T11:21:05.934+01:00European Parliament Candidates Have a Unique Opportunity to Advocate for Banking Union Transparency and Accountability<blockquote>This is reposted from the original on the <a href="http://www.hertie-school.org/blog/european-parliament-candidates-unique-opportunity-advocate-banking-union-transparency-accountability/">Hertie School of Governance European Elections blog</a>.</blockquote>
<p>The discussion of issues around the European Parliament Elections has been beating around the bush for quite some time now. Karlheinz Reif and Hermann Schmitt famously described European Elections as ”<a href="http://is.muni.cz/el/1423/jaro2009/POL288/um/Reif_and_Schmitt.pdf">second-order elections</a>”, in that they are secondary to national elections. A few weeks ago on this blog <a href="http://www.hertie-school.org/blog/issues-people/#more-561" >Andrea Römmele and Yann Lorenz</a> argued that the current election cycle has been characterised by personality politics between candidates vying for the Commission presidency, rather than substantive issues.</p>
<p>However, the election campaigns could be an important opportunity for the public to <a href="http://www.theguardian.com/commentisfree/2013/jun/03/2014-european-elections-shape-eu">express their views on</a> and even learn more about one of the defining changes to the European Union since the introduction of the Euro: the European Banking Union.</p>
<p>Much of the framework for the Banking Union has been established in the past year after <a href="http://europeansting.com/2014/02/19/banking-union-ecofin-and-parliament-ready-to-compromise/">intense debate between the EU institutions</a>. A key component of the Union is that in November 2014, the European Central Bank (ECB) will become the <a href="http://www.ecb.europa.eu/ssm/html/index.en.html">primary regulator</a> for about 130 of the Euro area’s largest banks and will have the power to become the main supervisor of any other bank, should it deem this necessary to ensure ”high standards”.</p>
<p>A perennial complaint made against the EU is that it lacks <a href="http://www.ombudsman.europa.eu/activities/speech.faces/en/10959/html.bookmark">transparency and accountability</a>. While there are many causes of this (not least of which is <a href="http://ukcatalogue.oup.com/product/9780199665686.do">poor media coverage of EU policy-making</a>), the ECB’s activities in the Banking Union certainly are less than transparent according to the rules currently set out. As <a href="http://www.hertie-school.org/hallerberg/" >Prof. Mark Hallerberg</a> and I document in a <a href="http://www.bruegel.org/publications/publication-detail/publication/807-supervisory-transparency-in-the-european-banking-union/" >recent Bruegel Policy Note</a>, financial regulatory transparency in Europe and especially the Banking Union is very lacking. Unlike in another large banking union –the United States, where detailed supervisory data is released every quarter – the ECB does not plan to regularly release any data on the individual banks it supervises.</p>
<p>This makes it difficult for citizens, especially informed watchdog groups, to independently evaluate the ECB’s supervisory effectiveness before it is too late, i.e. before there is another crisis.</p>
<p>The European Parliament has been somewhat successful in <a href="http://onlinelibrary.wiley.com/enhanced/doi/10.1111/jcms.12054/">improving the transparency and accountably</a> (paywall) of the ECB’s future supervisory activities. Unlike originally proposed, the Parliament now has the power to scrutinise the ECB’s supervisory activities. It will nonetheless be constrained by strict confidentiality rules in its ability to freely access information and publish the information it does find.</p>
<p>In our paper, we also show how a lack of supervisory transparency is not exclusive to EU supervisors – the member state regulators, who will still directly oversee most banks, are in general similarly opaque. We found that only 11 (five in the Eurozone) out of 28 member states regularly release any supervisory data. Member state <a href="http://www.eba.europa.eu/supervisory-convergence/supervisory-disclosure/aggregate-statistical-data">reporting of basic aggregate supervisory data</a> to the European Banking Authority is also very inconsistent.</p>
<p>European Parliamentarians could use the increased attention that they receive during the election period to improve public awareness of the important role they have played in improving the transparency and accountability of new EU institutions. Perhaps, after the election, they could even use popular support that they may build for these activities during the election period to get stronger oversight capabilities and improve financial supervisory transparency in the European Banking Union.</p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-36784551.post-88976117338643903522014-03-30T14:45:00.000+01:002014-03-30T14:47:23.685+01:00Numbering Subway Exits<p>In a bit of an aside from what I usually work on, I've put together a small website with a simple purpose: advocating for subway station exits to be numbered. These are really handy for finding your way around and are common in East Asia. But I've never seen them in Western countries.</p>
<p>If you're interested check out <a href="http://christophergandrud.github.io/WayOut/">the site:</a>
<a href="http://christophergandrud.github.io/WayOut/"><div class="separator" style="clear: both; text-align: center;"><img border="0" src="http://3.bp.blogspot.com/-toYHNkMLzag/UzgfZk_duQI/AAAAAAAAGz8/ec9LWw3it_U/s400/Way_Out.png" /></div></a>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-36784551.post-58649394649227982762014-02-23T16:41:00.000+00:002014-02-24T07:29:53.855+00:00Programmatically download political science data with the psData package<p>A lot of progress has been made on improving political scientists’ ability to access data ‘programmatically’, e.g. data can be downloaded with source code R. Packages such as <a href="http://cran.r-project.org/web/packages/WDI/index.html">WDI</a> for World Bank Development Indicator and <a href="http://cran.r-project.org/web/packages/dvn/index.html">dvn</a> for many data sets stored on the <a href="http://thedata.org/">Dataverse Network</a> make it much easier for political scientists to use this data as part of a highly <a href="http://christophergandrud.blogspot.de/2013/07/getting-started-with-reproducible.html">integrated and reproducible workflow</a>. </p>
<p>There are nonetheless still many commonly used political science data sets that aren’t easily accessible to researchers. Recently, I’ve been using the <a href="http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/EXTRESEARCH/0,,contentMDK:20649465~pagePK:64214825~piPK:64214943~theSitePK:469382,00.html">Database of Political Institutions (DPI)</a>, <a href="http://www.systemicpeace.org/polity/polity4.htm">Polity IV</a> democracy indicators, and <a href="http://www.carmenreinhart.com/data/browse-by-topic/topics/7/">Reinhart and Rogoff’s (2010)</a> financial crisis occurrence data. All three of these data sets are freely available for download online. However, getting them, cleaning them up, and merging them together is kind of a pain. This is especially true for the Reinhart and Rogoff data, which is in 4 Excel files with over 70 individual sheets, one for each country’s data. </p>
<p>Also, I’ve been using variables that are combinations and/or transformations of indicators in regularly updated data sets, but which themselves aren’t regularly updated. In particular, <a href="http://www.nyu.edu/gsas/dept/politics/data/bdm2s2/Logic.htm">Bueno de Mesquita et al. (2003)</a> devised two variables that they called the ‘winset’ and the ‘selectorate’. These are basically specific combinations of data in DPI and Polity IV. However, the winset and selectorate variables haven’t been updated alongside the yearly updates of DPI and Polity IV. </p>
<p>There are two big problems here:</p>
<ol>
<li><p>A lot of time is wasted by political scientists (and their RAs) downloading, cleaning, and transforming these data sets for their own research.</p></li>
<li><p>There are many opportunities while doing this work to introduce errors. Imagine the errors that might be introduced and go unnoticed if a copy-and-paste approach is used to merge the 70 Reinhart and Rogoff Excel sheets. </p></li>
</ol>
<p>As a solution, I’ve been working on a new R package called <em><a href="https://github.com/christophergandrud/psData">psData</a></em>. This package includes functions that automate the gathering, cleaning, and creation of common political science data and variables. So far (February 2014) it gathers DPI, Polity IV, and Reinhart and Rogoff data, as well as creates winset and selectorate variables. Hopefully the package will save political scientists a lot of time and reduce the number of data management errors. </p>
<p>There certainly could be errors in the way <em>psData</em> gathers data. However, once spotted the errors could be easily reported on the package’s <a href="https://github.com/christophergandrud/psData/issues">Issues Page</a>. Once fixed, the correction will be spread to all users via a package update.</p>
<h2 id="typesoffunctions">Types of functions</h2>
<p>There are two basic types of functions in <em>psData</em>: <strong>Getters</strong> and <strong>Variable Builders</strong>. Getter functions automate the gathering and cleaning of particular data sets so that they can easily be merged with other data. They do not transform the underlying data. Variable Builders use Getters to gather data and then transform it into new variables suggested by the political science literature.</p>
<h2 id="examples">Examples</h2>
<p>To download only the <strong>polity2</strong> variable from <a href="http://www.systemicpeace.org/polity/polity4.htm">Polity IV</a>:</p>
<pre><code class="r"># Load package
library(psData)
# Download polity2 variable
PolityData <- PolityGet(vars = "polity2")
# Show data
head(PolityData)
## iso2c country year polity2
## 1 AF Afghanistan 1800 -6
## 2 AF Afghanistan 1801 -6
## 3 AF Afghanistan 1802 -6
## 4 AF Afghanistan 1803 -6
## 5 AF Afghanistan 1804 -6
## 6 AF Afghanistan 1805 -6
</code></pre>
<p>Note that the <strong>iso2c</strong> variable refers to the <a href="http://en.wikipedia.org/wiki/ISO_3166-1_alpha-2">ISO two letter country code country ID</a>. This standardised country identifier could be used to easily merge the Polity IV data with another data set. Another country ID type can be selected with the <code>OutCountryID</code> argument. See the package documentation for details.</p>
<p>To create <strong>winset</strong> (<strong>W</strong>) and <strong>selectorate</strong> (<strong>ModS</strong>) data use the following code:</p>
<pre><code class="r">WinData <- WinsetCreator()
head(WinData)
## iso2c country year W ModS
## 1 AF Afghanistan 1975 0.25 0
## 2 AF Afghanistan 1976 0.25 0
## 3 AF Afghanistan 1977 0.25 0
## 15 AF Afghanistan 1989 0.50 0
## 16 AF Afghanistan 1990 0.50 0
## 17 AF Afghanistan 1991 0.50 0
</code></pre>
<hr />
<h2 id="install">Install</h2>
<p><em>psData</em> should be on <a href="http://cran.r-project.org/">CRAN</a> soon, but while it is in the development stage you can install it with the <a href="https://github.com/hadley/devtools">devtools</a> package:</p>
<pre><code class="r">devtools::install_github('psData', 'christophergandrud')
</code></pre>
<h2 id="suggestions">Suggestions</h2>
<p>Please feel free to suggest other data set downloading and variable creating functions. To do this just leave a note on the package’s <a href="https://github.com/christophergandrud/psData/issues">Issues page</a> or make a pull request with a new function added.</p>Unknownnoreply@blogger.com5tag:blogger.com,1999:blog-36784551.post-15825998981202000122014-01-04T15:03:00.000+00:002014-01-04T15:03:50.088+00:00How I Accidentally Wrote a Paper on Supervisory Transparency in the European Union and Why You Should Too<p>Research is an unpredictable thing. You head in one direction, but end up going another. Here is a recent example:</p>
<p>A co-author and I had an idea for a paper. It's a long story, but basically we wanted to compare banks in the US to those in the EU. This was a situation where our desire to explore a theory was egged on by, what we believed, was available data. In the US it's easy to gather data on banks because the regulators have a <a href="http://www.ffiec.gov/">nice website</a> where they release the filings banks send them. The data is in a really good format for statistical analysis. US done. We thought our next move would be to quickly gather similar data for EU banks and we would be on our way.</p>
<p>First we contacted the UK's <a href="http://www.fca.org.uk/">Financial Conduct Authority</a>. Surprisingly, they told us that not only did they not release this data, but it was actually illegal for them to do so. Pretty frustrating. Answers to one question stymied by a lack of data. Argh. I guess we'll just keep looking to see what kind of sample of European countries we can come up with and hope it doesn't lead to ridiculous sampling bias.</p>
<p>We eventually found that only 11 EU countries (out of 28) release any such data and it is in a hodgepodge of different formats making it very difficult to compare across countries. This is remarkable when compared to the US and kind of astounded me given <a href="http://christophergandrud.github.io/RepResR-RStudio/">my open data priors</a>. We then looked at EU level initiatives to increase supervisory transparency. The <a href="http://www.eba.europa.eu/">European Banking Authority</a> has made a number of attempts to increase transparency. For example, it asks Member States to submit some basic aggregate level data about their banking systems. It makes this data available on <a href="http://www.eba.europa.eu/supervisory-convergence/supervisory-disclosure/aggregate-statistical-data">its website</a>.</p>
<p>Countries have a lot of reporting flexibility. They can even choose to label specific items as non-material, non-applicable, or even confidential. Remarkably, a fair number of countries don't even do this. They just report <em>nothing</em>, as we can see here:</p>
<h4>Number of Member States without missing aggregate banking data reported to the EBA</h4>
<br/>
<div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-CZehD4_OSZQ/UsgiQool49I/AAAAAAAAGww/LaU3KMxmzhM/s1600/EBAReportingFigureOriginal.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-CZehD4_OSZQ/UsgiQool49I/AAAAAAAAGww/LaU3KMxmzhM/s640/EBAReportingFigureOriginal.png" /></a></div>
<p>Though almost all Member States reported data during the height of the crisis (2009) this is an aberration. In fact a lot of countries just don't report anything.</p>
<p>What are the implications of this secrecy we asked? We decided to do more research to find out. We published the first outcome of this project <a href="http://www.bruegel.org/publications/publication-detail/publication/807-supervisory-transparency-in-the-european-banking-union/">here</a>. Have a look if you're interested, but that isn't the point of this blog post. The point is that we set off to research one thing, but ended up stumbling upon another problem that was worthy of investigation.</p>
<p>This post's title is clearly a bit facetious. But the general point is serious. We need to be flexible enough and curious enough to be able to answer the questions we didn't even know to ask before we go looking.</p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-36784551.post-80688848895014024552013-12-06T17:43:00.000+00:002013-12-06T17:43:17.532+00:00Three Quick and Simple Data Cleaning Helper Functions (December 2013)<p>As I go about cleaning and merging data sets with R I often end up creating and using simple functions over and over. When this happens, I stick them in the <a href="http://christophergandrud.github.io/DataCombine/">DataCombine</a> package. This makes it easier for me to remember how to do an operation and others can possibly benefit from simplified and (hopefully) more intuitive code. </p>
<p>I've talked about some of the commands in <strong>DataCombine</strong> in <a href="http://christophergandrud.blogspot.de/2013/05/slide-one-function-for-laglead.html">previous</a> <a href="http://christophergandrud.blogspot.de/2013/02/fillin-function-for-filling-in-missing.html">posts</a>. In this post I'll give examples for a few more that I've added over the past couple of months. Note: these examples are based on <strong>DataCombine</strong> version 0.1.11.</p>
<p>Here is a brief run down of the functions covered in this post:</p>
<ul>
<li><p><a href="#FindReplace"><code>FindReplace</code></a>: a function to replace multiple patterns found in a character string column of a data frame.</p></li>
<li><p><a href="#MoveFront"><code>MoveFront</code></a>: moves variables to the front of a data frame. This can be useful if you have a data frame with many variables and want to move a variable or variables to the front.</p></li>
<li><p><a href="#rmExcept"><code>rmExcept</code></a>: removes all objects from a work space except those specified by the user.</p></li>
</ul>
<h1 id="FindReplace"><code>FindReplace</code></h1>
<p>Recently I needed to replace many patterns in a column of strings. Here is a short example. Imagine we have a data frame like this:</p>
<pre><code class="r">ABData <- data.frame(a = c("London, UK", "Oxford, UK", "Berlin, DE", "Hamburg, DE", "Oslo, NO"), b = c(8, 0.1, 3, 2, 1))
</code></pre>
<p>Ok, now I want to replace the <code>UK</code> and <code>DE</code> parts of the strings with <code>England</code> and <code>Germany</code>. So I create a data frame with two columns. The first records the pattern and the second records what I want to replace the pattern with:</p>
<pre><code class="r">Replaces <- data.frame(from = c("UK", "DE"), to = c("England", "Germany"))
</code></pre>
<p>Now I can just use <code>FindReplace</code> to make the replacements all at once:</p>
<pre><code class="r">library(DataCombine)
ABNewDF <- FindReplace(data = ABData, Var = "a", replaceData = Replaces, from = "from", to = "to", exact = FALSE)
# Show changes
ABNewDF
</code></pre>
<pre><code>## a b
## 1 London, England 8.0
## 2 Oxford, England 0.1
## 3 Berlin, Germany 3.0
## 4 Hamburg, Germany 2.0
## 5 Oslo, NO 1.0
</code></pre>
<p>If you set <code>exact = TRUE</code> then <code>FindReplace</code> will only replace exact pattern matches. Also, you can set <code>vector = TRUE</code> to return only a vector of the column you replaced (the <code>Var</code> column), rather than the whole data frame.</p>
<h1 id="MoveFront"><code>MoveFront</code></h1>
<p>On occasion I've wanted to move a few variables to the front of a data frame. The <code>MoveFront</code> function makes this pretty simple. It only has two arguments: <code>data</code> and <code>Var</code>. Data is the data frame and <code>Var</code> is a character vector with the columns I want to move to the front of the data frame in the order that I want them. Here is an example:</p>
<pre><code class="r"># Create dummy data
A <- B <- C <- 1:50
OldOrder <- data.frame(A, B, C)
names(OldOrder)
</code></pre>
<pre><code>## [1] "A" "B" "C"
</code></pre>
<pre><code class="r"># Move B and A to the front
NewOrder2 <- MoveFront(OldOrder, c("B", "A"))
names(NewOrder2)
</code></pre>
<pre><code>## [1] "B" "A" "C"
</code></pre>
<h1 id="rmExcept"><code>rmExcept</code></h1>
<p>Finally, sometimes I want to clean up my work space and only keep specific objects. I want to remove everything else. This is straightforward with <code>rmExcept</code>. For example:</p>
<pre><code class="r"># Create objects
A <- 1
B <- 2
C <- 3
# Remove all objects except for A
rmExcept("A")
</code></pre>
<pre><code>## Removed the following objects:
## ABData, ABNewDF, B, C, NewOrder2, OldOrder, Replaces
</code></pre>
<pre><code class="r"># Show workspace
ls()
</code></pre>
<pre><code>## [1] "A"
</code></pre>
<p>You can set the environment you want to clean up with the <code>envir</code> argument. By default is is your global environment.</p>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-36784551.post-29992893745404923042013-09-02T14:27:00.000+01:002014-02-02T12:16:39.350+00:00Showing results from Cox Proportional Hazard Models in R with simPH<p><strong>Update 2 February 2014:</strong> A new version of simPH (Version 1.0) will soon be available for download from CRAN. It allows you to plot using points, ribbons, and (new) lines. See the <a href="http://ssrn.com/abstract=2318977">updated package description paper</a> for examples. Note that the <code>ribbons</code> argument will no longer work as in the examples below. Please use <code>type = 'ribbons'</code> (or <code>'points'</code> or <code>'lines'</code>).
</p>
<p>Effectively showing estimates and uncertainty from <a href="http://en.wikipedia.org/wiki/Proportional_hazards_models">Cox Proportional Hazard (PH) models</a>, especially for interactive and non-linear effects, can be challenging with currently available software. So, researchers often just simply display a results table. These are pretty useless for Cox PH models. It is difficult to decipher a simple linear variable’s estimated effect and basically impossible to understand time interactions, interactions between variables, and nonlinear effects without the reader further calculating quantities of interest for a variety of fitted values.</p>
<p>So, I’ve been putting together the <a href="https://github.com/christophergandrud/SimPH">simPH</a> R package to hopefully make it easier to show results from Cox PH models. The package uses plots of post-estimation simulations (the same idea behind the plotting facilities in the <a href="http://projects.iq.harvard.edu/zelig">Zelig</a> package) to show Cox PH estimates and the uncertainty surrounding them.</p>
<p>Here I want to give an overview of how to use <code>simPH</code>. First the general process, then a few examples. Have a look at <a href="http://ssrn.com/abstract=2318977">this paper</a> for details about the math behind the graphs.</p>
<h3 id="generalsteps">General Steps</h3>
<p>There are three steps to use <code>simPH</code>:</p>
<ol>
<li><p>Estimate a Cox PH model in the usual way with the <code>coxph</code> command in the <a href="http://cran.r-project.org/web/packages/survival/index.html">survival</a> package.</p></li>
<li><p>Simulate quantities of interest--<a href="http://en.wikipedia.org/wiki/Hazard_ratio">hazard ratios</a>, <a href="http://pan.oxfordjournals.org/content/19/2/227.abstract">first differences</a>, <a href="http://pan.oxfordjournals.org/content/14/1/63">marginal effect</a>, <a href="http://pan.oxfordjournals.org/content/19/2/227.abstract">relative hazards</a>, or <a href="http://en.wikipedia.org/wiki/Survival_analysis#Hazard_function_and_cumulative_hazard_function">hazard rates</a>--with the appropriate <code>simPH</code> simulation command.</p></li>
<li><p>Plot the simulations with the <code>simGG</code> method.</p></li>
</ol>
<h3 id="afewexamples">A Few Examples</h3>
<p>Here are some basic examples that illustrate the process and key syntax. Before getting started you can install <code>simPH</code> in the normal R way from CRAN.</p>
<h4 id="lineareffects">Linear effects</h4>
<p>Let’s look at a simple example with a linear non-interactive effect. The data set I’m using is from <a href="http://www.jstor.org/stable/3088394">Carpenter (2002)</a>. It is included with <code>simPH</code>. See the package documentation for details.</p>
<p>First, let’s estimate a Cox PH model where the event of interest is a drug receiving FDA approval.</p>
<pre><code># Load packages
library(survival)
library(simPH)
# Load Carpenter (2002) data
data("CarpenterFdaData")
# Estimate survival model
M1 <- coxph(Surv(acttime, censor) ~ prevgenx + lethal +
deathrt1 + acutediz + hosp01 + hhosleng +
mandiz01 + femdiz01 + peddiz01 + orphdum +
vandavg3 + wpnoavg3 + condavg3 + orderent +
stafcder, data = CarpenterFdaData)
</code></pre>
<p>Now say we want to examine the effect that the number of FDA staff reviewing a drug application has on it being accepted. This variable is called <code>stafcder</code> in the model we just estimated. To do this let’s use <code>simPH</code> to simulate hazard ratios. We will simulate hazard ratios for units \(j\) and \(l\), i.e. \(\frac{h_{j}(t)}{h_{l}(t)}\) using <code>simPH</code>’s <code>coxsimLinear</code> command, because we estimate the effect of the number of staff as linear. In the following code we use the <code>Xj</code> argument to set the \(j\) values. We could use <code>Xl</code> also, but as we don’t <code>coxsimLinear</code> assumes all \(x_{l}\) are 0. </p>
<p>Notice that the argument <code>spin = TRUE</code>. This finds the shortest 95% probability interval of the simulations using the <a href="http://arxiv.org/pdf/1302.2142v1.pdf">SPIn method</a>. SPIn can be especially useful for showing simulated quantities of interest generated from Cox PH models, because then can often be crowded close to a lower boundary (0 in the case of hazard rates). We should be more interested in the area with the highest probability–most simulated values–rather than an arbitrary central interval. That being said, if <code>spin = FALSE</code>, then we will simply find the central 95% interval of the simulations.</p>
<pre><code># Simulate and plot Hazard Ratios for stafcder variable
Sim1 <- coxsimLinear(M1, b = "stafcder",
qi = "Hazard Ratio", ci = 0.95,
Xj = seq(1237, 1600, by = 2), spin = TRUE)
# Plot
simGG(Sim1)
</code></pre>
<p><a href="http://figshare.com/articles/Example_Linear_Effect_Created_by_simPH/785759"><img src="http://figshare.com/download/file/1187133/1" style="display: block; margin: auto;" /></a></p>
<p>Notice in the plot that each simulation is plotted as an individual point. These are all of the simulations in the shortest 95% probability interval. Each point has a bit of transparency (they are 90% transparent by default). So the plot is <a href="https://dl. dropboxusercontent.com/u/3011470/WorkingPapers/HSIANG_VISUALLY_WEIGHTED_ REGRESSION_v1.pdf">visually weighted</a>; the darker areas of the graph have a higher concentration of the simulations. This gives us a very clear picture of the simulation distribution, i.e. the estimated effect and our uncertainty about it. </p>
<p>If you don’t want to plot every point, you can simply use ribbons showing the constricted 95% and the middle 50% of this interval. To do this simply use the <code>ribbons = TRUE</code> argument.</p>
<pre><code>simGG(Sim1, ribbons = TRUE, alpha = 0.5)
</code></pre>
<p><a href="http://figshare.com/articles/Example_Linear_Effect_Graph_Created_By_simPH/785758"><img src="http://figshare.com/download/file/1187132/1" title="plot of chunk unnamed-chunk-4" alt="plot of chunk unnamed-chunk-4" style="display: block; margin: auto;" /></a></p>
<p>Notice the <code>alpha = 0.5</code> argument. This increased the transparency of the widest ribbon to 50%.</p>
<p>The syntax we’ve used here is very similar to what we use when we are working with nonlinear effects estimated with polynomials and splines. Post-estimation simulations can be run with the <code>coxsimPoly</code> and <code>coxsimSpline</code> commands. See the <code>simPH</code> documentation for more examples.</p>
<h4 id="interactions">Interactions</h4>
<p>The syntax for two-way interactions simulated with the <code>coxsimInteract</code> command is a little different. Using the same data, let’s look at how to show results for interactions. In the following model we are interacting two variables <code>lethal</code> and <code>prevgenx</code>. We can think of these variables as <code>X1</code> and <code>X2</code>, respectively. For interactions it can be useful to examine the <a href="http://pan.oxfordjournals.org/content/14/1/63">marginal effect</a>. To find the marginal effect of a one unit increase in the <code>lethal</code> variable given various values of <code>prevgenx</code> let’s use the following code:</p>
<pre><code># Estimate the model
M2 <- coxph(Surv(acttime, censor) ~ lethal*prevgenx, data = CarpenterFdaData)
# Simulate Marginal Effect of lethal for multiple values of prevgenx
Sim2 <- coxsimInteract(M2, b1 = "lethal", b2 = "prevgenx",
qi = "Marginal Effect",
X2 = seq(2, 115, by = 2), nsim = 1000)
# Plot the results
simGG(Sim2, ribbons = TRUE, alpha = 0.5, xlab = "\nprevgenx",
ylab = "Marginal effect of lethal\n")
</code></pre>
<p><a href="http://figshare.com/articles/simPH_Marginal_Effect_Example/785760"><img src="http://figshare.com/download/file/1187134/1" title="plot of chunk unnamed-chunk-5" alt="plot of chunk unnamed-chunk-5" style="display: block; margin: auto;" /></a></p>
<p>The order of the <code>X1</code> and <code>X2</code> variables in the interactions matters. The marginal effect is always calculated for the <code>X1</code> variable over a range of <code>X2</code> values.</p>
<p>Notice also that we set the <code>x</code> and <code>y</code> axis labels with the <code>xlab</code> and <code>ylab</code> arguments.</p>
<h4 id="time-varyingeffects">Time-varying Effects</h4>
<p>Finally, let’s look at how to use with the <code>coxsimtvc</code> command to show results from effects that we estimate to vary over time. Here we are going to use another data set that is also included with <code>simPH</code>. The event of interest in the following model is when deliberation is stopped on a European Union directive (the model is from <a href="http://pan.oxfordjournals.org/content/19/2/227.short">Licht (2011)</a>). We will create hazard ratios for the effect that the number of backlogged items (<code>backlog</code>) has on deliberation time. We will estimate the effect as a log-time interaction.</p>
<pre><code># Load Golub & Steunenberg (2007) data. The data is included with simPH.
data("GolubEUPData")
# Create natural log-time interactions
Golubtvc <- function(x){
tvc(data = GolubEUPData, b = x, tvar = "end", tfun = "log")
}
GolubEUPData$Lcoop <- Golubtvc("coop")
GolubEUPData$Lqmv <- Golubtvc("qmv")
GolubEUPData$Lbacklog <- Golubtvc("backlog")
GolubEUPData$Lcodec <- Golubtvc("codec")
GolubEUPData$Lqmvpostsea <- Golubtvc("qmvpostsea")
GolubEUPData$Lthatcher <- Golubtvc("thatcher")
# Estimate model
M1 <- coxph(Surv(begin, end, event) ~ qmv + qmvpostsea + qmvpostteu +
coop + codec + eu9 + eu10 + eu12 + eu15 + thatcher +
agenda + backlog + Lqmv + Lqmvpostsea + Lcoop + Lcodec +
Lthatcher + Lbacklog,
data = GolubEUPData, ties = "efron")
</code></pre>
<p>Much of the first half of the code is dedicated to creating the log-time interactions with the <code>tvc</code> command.</p>
<p>Now we simply create the simulations for a range of values of <code>backlog</code> and plot them. Note in the following code that we tell <code>coxsimtvc</code> the name of both the base <code>backlog</code> variable and its log-time interaction term <code>Lbacklog</code> using the <code>btvc</code> argument. We also need to tell <code>coxsimtvc</code> that it is a log-time interaction with <code>tfun = "log"</code>.</p>
<pre><code># Create simtvc object for relative hazard
Sim2 <- coxsimtvc(obj = M1, b = "backlog", btvc = "Lbacklog",
qi = "Relative Hazard", Xj = seq(40, 200, 40),
tfun = "log", from = 1200, to = 2000, by = 10,
nsim = 500)
# Create relative hazard plot
simGG(Sim2, xlab = "\nTime in Days", ribbons = TRUE,
leg.name = "Backlogged \n Items", alpha = 0.2)
</code></pre>
<p><a href="http://figshare.com/articles/Relative_Hazards_of_Backlog_on_EU_Directive_Deliberation_Time/785761"><img src="http://figshare.com/download/file/1187135/1" title="plot of chunk unnamed-chunk-7" alt="plot of chunk unnamed-chunk-7" style="display: block; margin: auto;" /></a></p>
<h4 id="simggplotsareggplot2">simGG Plots are ggplot2</h4>
<p>Finally, because almost every plot created by <code>simGG</code> is a <a href="http://docs.ggplot2.org/current/">ggplot2</a> plot, you can use almost the full range of customisation options that that package allows. See the <a href="http://docs.ggplot2.org/current/">ggplot2</a> documentation for many examples.</p>Unknownnoreply@blogger.com16tag:blogger.com,1999:blog-36784551.post-58452723746992276862013-08-23T04:42:00.000+01:002013-08-23T04:42:27.794+01:00GitHub renders CSV in the browser, becomes even better for social data set creation<p>I've <a href="http://christophergandrud.github.io/RepResR-RStudio/">written</a> in a <a href="http://polmeth.wustl.edu/methodologist/tpm_v20_n2.pdf">number</a> of <a href="http://christophergandrud.blogspot.kr/2013/01/sourcegithubdata-simple-function-for.html">places</a> about how <a href="https://github.com/">GitHub</a> can be a great place to store data. Unlike basically all other web data storage sites (many of which I really like such as <a href="http://thedata.org/">Dataverse</a> and <a href="http://figshare.com/">FigShare</a>) GitHub enables deep social data set development and fits nicely into a reproducible research workflow with R.</p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://github.com/christophergandrud/Disproportionality_Data/blob/master/Disproportionality.csv#L2" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="http://3.bp.blogspot.com/-_b1nzjNlQwc/UhbZmrx8YYI/AAAAAAAAGtk/GYT8stzlMQs/s320/Disproportionality_Data_Disproportionality.csv_at_master_%C2%B7_christophergandrud_Disproportionality_Data.png" /></a></div>
<p>One negative though, especially compared to FigShare, was that there was no easy way to view CSV or TSV data files in the browser. Unless you downloaded the data and opened it in Excel or an R viewer or whatever, you had to look at the raw data file in the browser. It's basically impossible to make sense of a data set of any size like this.</p>
<p>However, from at least today, GitHub now renders the data set in the browser as you would expect. Take a look at <a href="https://github.com/blog/1601-see-your-csvs">their blog post</a> for the details.</p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-36784551.post-71234827372873440542013-07-16T00:35:00.000+01:002013-12-20T10:23:30.785+00:00Getting Started with Reproducible Research: A chapter from my new book<p><a href="http://www.amazon.com/Reproducible-Research-RStudio-Chapman-Series/dp/1466572841/ref=sr_1_1?ie=UTF8&qid=1365636095&sr=8-1&keywords=christopher+gandrud" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="http://3.bp.blogspot.com/-f8MFbNEoyGU/UYNGekqEkTI/AAAAAAAAGOM/Dq36pI06kTQ/s320/RepResCover.jpg" width="160" height="220"/></a></p>
<p><strong>This is an abridged excerpt from Chapter 2 of my new book <a href="http://christophergandrud.github.io/RepResR-RStudio/">Reproducible Research with R and RStudio</a>. It's published by <a href="http://www.crcpress.com/product/isbn/9781466572843">Chapman & Hall/CRC Press</a>.</strong></p>
<p><strong>You can purchase it on <a href="http://www.amazon.com/Reproducible-Research-RStudio-Chapman-Series/dp/1466572841/ref=sr_1_1?ie=UTF8&qid=1365636095&sr=8-1&keywords=christopher+gandrud">Amazon</a>. "Search inside this book" includes a complete table of contents.</strong></p>
<p>Researchers often start thinking about making their work reproducible near the end of the research process when they write up their results or maybe even later when a journal requires their data and code be made available for publication. Or maybe even later when another researcher asks if they can use the data from a published article to reproduce the findings. By then there may be numerous versions of the data set and records of the analyses stored across multiple folders on the researcher’s computers. It can be difficult and time consuming to sift through these files to create an accurate account of how the results were reached. Waiting until near the end of the research process to start thinking about reproducibility can lead to incomplete documentation that does not give an accurate account of how findings were made. Focusing on reproducibility from the beginning of the process and continuing to follow a few simple guidelines throughout your research can help you avoid these problems. Remember "reproducibility is not an afterthought–it is something that must be built into the project from the beginning" (<a href="http://biostatistics.oxfordjournals.org/content/11/3/385.short">Donoho, 2010</a>, 386).</p>
<p>This chapter first gives you a brief overview of the reproducible research process: a workflow for reproducible research. Then it covers some of the key guidelines that can help make your research more reproducible.</p>
<h1>The Big Picture: A Workflow for Reproducible Research</h1>
<p>The three basic stages of a typical computational empirical research project are:</p>
<ul>
<li>data gathering,</li>
<li>data analysis,</li>
<li>results presentation.</li>
</ul>
<p>Instead of starting to use the individual tools of reproducible research as soon as you learn them I recommend briefly stepping back and considering how the stages of reproducible research tie together overall. This will make your workflow more coherent from the beginning and save you a lot of backtracking later on. The figure below illustrates the workflow. Notice that most of the arrows connecting the workflow’s parts point in both directions, indicating that you should always be thinking how to make it easier to go backwards through your research, i.e. reproduce it, as well as forwards.</p>
<hr />
<h3>Example Workflow & A Selection of Commands to Tie it Together</h3>
<div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-u3BaetbDxtM/UeR4f8924nI/AAAAAAAAGlU/fnl_KfDSSb8/s1600/Workflow.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-u3BaetbDxtM/UeR4f8924nI/AAAAAAAAGlU/fnl_KfDSSb8/s640/Workflow.png" /></a></div>
<hr />
<p>Around the edges of the figure are some of the commands that make it easier to go forwards and backwards through the process. These commands tie your research together. For example, you can use <a href="http://en.wikipedia.org/wiki/Application_programming_interface">API</a>-based R packages to gather data from the internet. You can use R’s <code>merge</code> command to combine data gathered from different sources into one data set. The <code>getURL</code> from R’s <a href="http://cran.r-project.org/web/packages/RCurl/index.html">RCurl</a> package and the read.table commands can be used to bring this data set into your statistical analyses. The <a href="http://yihui.name/knitr/">knitr</a> package then ties your analyses into your presentation documents. This includes the code you used, the figures you created, and, with the help of tools such as the <a href="http://cran.r-project.org/web/packages/xtable/index.html">xtable</a> package, tables of results. You can even tie multiple presentation documents together. For example, you can access the same figure for use in a LaTeX article and a Markdown created website with the <code>includegraphics</code> and <code>![]()</code> commands, respectively. This helps you maintain a consistent presentation of results across multiple document types.</p>
<h1>Practical Tips for Reproducible Research</h1>
<p>Before learning the details of the reproducible research workflow with R and RStudio, it’s useful to cover a few broad tips that will help you organize your research process and put these skills in perspective. The tips are:</p>
<ol>
<li>Document everything!,</li>
<li>Everything is a (text) file,</li>
<li>All files should be human readable,</li>
<li>Explicitly tie your files together,</li>
<li>Have a plan to organize, store, and make your files available.</li>
</ol>
<p>Using these tips will help make your computational research really reproducible.</p>
<h2>Document everything!</h2>
<p>In order to reproduce your research others must be able to know what you did. You have to tell them what you did by documenting as much of your research process as possible. Ideally, you should tell your readers how you gathered your data, analyzed it, and presented the results. Documenting everything is the key to reproducible research and lies behind all of the other tips here.</p>
<h2>Everything is a (text) file</h2>
<p>Your documentation is stored in files that include data, analysis code, the write up of results, and explanations of these files (e.g. data set codebooks, session info files, and so on). Ideally, you should use the simplest file format possible to store this information. Usually the simplest file format is the humble, but versatile, text file.</p>
<p>Text files are extremely nimble. They can hold your data in, for example, comma-separated values (<strong>.csv</strong>) format. They can contain your analysis code in <strong>.R</strong> files. And they can be the basis for your presentations as markup documents like <strong>.tex</strong> or <strong>.md</strong>, for LaTeX and Markdown files, respectively. All of these files can be opened by any program that can read text files.</p>
<p>One reason reproducible research is best stored in text files is that this helps future proof your research. Other file formats, like those used by Microsoft Word (<strong>.docx</strong>) or Excel (<strong>.xlsx</strong>), change regularly and may not be compatible with future versions of these programs. Text files, on the other hand, can be opened by a very wide range of currently existing programs and, more likely than not, future ones as well. Even if future researchers do not have R or a LaTeX distribution, they will still be able to open your text files and, aided by frequent comments (see below), be able to understand how we conducted your research (<a href="http://polmeth.wustl.edu/methodologist/tpm_v18_n2.pdf">Bowers, 2011</a>, 3).</p>
<p>Text files are also very easy to search and manipulate with a wide range of programs–such as R and RStudio–that can find and replace text characters as well as merge and separate files. Finally, text files are easy to version and changes can be tracked using programs such as <a href="http://git-scm.com/">Git</a>.</p>
<h2>All files should be human readable</h2>
<p>Treat all of your research files as if someone who has not worked on the project will, in the future, try to understand them. Computer code is a way of communicating with the computer. It is 'machine readable' in that the computer is able to use it to understand what you want to do. However, there is a very good chance that other people (or you six months in the future) will not understand what you were telling the computer. So, you need to make all of your files 'human readable'. To make them human readable, you should comment on your code with the goal of communicating its design and purpose (<a href="http://arxiv.org/pdf/1210.0530v3">Wilson et al., 2012</a>). With this in mind it is a good idea to comment frequently (<a href="http://polmeth.wustl.edu/methodologist/tpm_v18_n2.pdf">Bowers, 2011</a>, 3) and format your code using a style guide (<a href="http://www.nyu.edu/classes/nagler/quant2/coding_style.html">Nagler, 1995</a>). For especially important pieces of code you should use literate programming–where the source code and the presentation text describing its design and purpose appear in the same document (<a href="http://www-cs-faculty.stanford.edu/~uno/lp.html">Knuth, 1992</a>). Doing this will make it very clear to others how you accomplished a piece of research.</p>
<h2>Explicitly tie your files together</h2>
<p>If everything is just a text file then research projects can be thought of as individual text files that have a relationship with one another. They are tied together. A data file is used as input for an analysis file. The results of an analysis are shown and discussed in a markup file that is used to create a PDF document. Researchers often do not explicitly document the relationships between files that they used in their research. For example, the results of an analysis–a table or figure–may be copied and pasted into a presentation document. It can be very difficult for future researchers to trace the table or figure back to a particular statistical model and a particular data set without clear documentation. Therefore, it is important to make the links between your files explicit.</p>
<p>Tie commands are the most dynamic way to explicitly link your files together (see the figure above for examples). These commands instruct the computer program you are using to use information from another file.</p>
<h2>Have a plan to organize, store, & make your files available</h2>
<p>Finally, in order for independent researchers to reproduce your work they need to be able access the files that instruct them how to do this. Files also need to be organized so that independent researchers can figure out how they fit together. So, from the beginning of your research process you should have a plan for organizing your files and a way to make them accessible.</p>
<p>One rule of thumb for organizing your research in files is to limit the amount of content any one file has. Files that contain many different operations can be very difficult to navigate, even if they have detailed comments. For example, it would be very difficult to find any particular operation in a file that contained the code used to gather the data, run all of the statistical models, and create the results figures and tables. If you have a hard time finding things in a file you created, think of the difficulties independent researchers will have!</p>
<p>Because we have so many ways to link files together there is really no need to lump many different operations into one file. So, we can make our files modular. One source code file should be used to complete one or just a few tasks. Breaking your operations into discrete parts will also make it easier for you and others to find errors (<a href="(http://www.nyu.edu/classes/nagler/quant2/coding_style.html">Nagler, 1995</a>, 490).</p>Unknownnoreply@blogger.com9tag:blogger.com,1999:blog-36784551.post-31720744410841118912013-06-09T03:00:00.000+01:002013-06-12T11:19:45.405+01:00Quick and Simple D3 Network Graphs from R<a href="http://dl.dropboxusercontent.com/u/12581470/Presentations/OddsAndEnds/NetworkD3.html" imageanchor="1" ><img border="0" style="float:right" src="http://1.bp.blogspot.com/-gFxgVvzBSr8/UbPgDW-7fnI/AAAAAAAAGiE/an2RbjOC-68/s320/NetworkD3.png"/></a>
<p>Sometimes I just want to quickly make a simple <a href="http://d3js.org/">D3</a> JavaScript directed <a href="http://bl.ocks.org/mbostock/4062045">network graph</a> with data in R. Because D3 network graphs can be manipulated in the browser–i.e. nodes can be moved around and highlighted–they're really nice for data exploration. They're also really nice in <a href="http://slidify.org/">HTML presentations</a>. So I put together a bare-bones simple function–called <a href="https://gist.github.com/christophergandrud/5734624">d3SimpleNetwork</a> for turning an R data frame into a D3 network graph. </p>
<h1>Arguments</h1>
<p>By bare-bones I mean other than the arguments indicating the <code>Data</code> data frame, as well as the <code>Source</code> and <code>Target</code> variables it only has three arguments: <code>height</code>, <code>width</code>, and <code>file</code>. </p>
<p>The data frame you use should have two columns that contain the source and target variables. Here's an example using fake data:</p>
<pre><code class="r">Source <- c("A", "A", "A", "A", "B", "B", "C", "C", "D")
Target <- c("B", "C", "D", "J", "E", "F", "G", "H", "I")
NetworkData <- data.frame(Source, Target)
</code></pre>
<p>The <code>height</code> and <code>width</code> arguments obviously set the graph's frame height and width. You can tell <code>file</code> the file name to output the graph to. This will create a standalone webpage. If you leave <code>file</code> as <code>NULL</code>, then the graph will be printed to the console. This can be useful if you are creating a document using <a href="http://yihui.name/knitr/">knitr</a> <a href="http://daringfireball.net/projects/markdown/">Markdown</a> <strike>or (similarly) <a href="http://slidify.org/">slidify</strike></a>. Just set the code chunk <code>results='asis</code> and the graph will be rendered in the document.</p>
<h1>Example</h1>
<p>Here's a simple example. First load <code>d3SimpleNetwork</code>:</p>
<pre><code class="r"># Load packages to download d3SimpleNetwork
library(digest)
library(devtools)
# Download d3SimpleNetwork
source_gist("5734624")
</code></pre>
<p>Now just run the function with the example <code>NetworkData</code> from before:</p>
<pre><code class="r">d3SimpleNetwork(NetworkData, height = 300, width = 700)
</code></pre>
<p><a href="http://dl.dropboxusercontent.com/u/12581470/Presentations/OddsAndEnds/NetworkD3.html">Click here</a> for the fully manipulable version. If you click on individual nodes they will change colour and become easier to see. In the future I might add more customisability, but I kind of like the function's current simplicity.</p>
<hr>
<strong>Update 12 June 2013:</strong> The original <code>d3SimpleNetwork</code> command discussed here doesn't work easily with slidify. I have created a new <a href="http://christophergandrud.github.io/d3Network/">d3Network</a> R package that does work well with slidify (and other knitr-created HTML slideshows). Use its <code>d3Network</code> command and set the argument <code>iframe = TRUE</code>.Unknownnoreply@blogger.com6tag:blogger.com,1999:blog-36784551.post-9857793767269692322013-05-21T11:18:00.000+01:002013-05-21T11:18:30.149+01:00Slide: one function for lag/lead variables in data frames, including time-series cross-sectional data<p>I often want to quickly create a lag or lead variable in an R data frame. Sometimes I also want to create the lag or lead variable for different groups in a data frame, for example, if I want to lag GDP for each country in a data frame.</p>
<p>I've found the various R methods for doing this hard to remember and usually need to look at old <a href="http://christophergandrud.tumblr.com/post/35695637523/r-lagged-variables-with-time-series-cross-sectional">blog</a> <a href="http://christophergandrud.tumblr.com/post/35758599399/r-1-lag">posts</a>. Any time we find ourselves using the same series of codes over and over, it's probably time to put them into a function. </p>
<p>So, I added a new command–<code>slide</code>–to the <a href="http://cran.r-project.org/web/packages/DataCombine/DataCombine.pdf">DataCombine</a> R package (v0.1.5).</p>
<p>Building on the <code>shift</code> function TszKin Julian posted on his <a href="http://ctszkin.com/2012/03/11/generating-a-laglead-variables/">blog</a>, <code>slide</code> allows you to slide a variable up by any time unit to create a lead or down to create a lag. It returns the lag/lead variable to a new column in your data frame. It works with both data that has one observed unit and with time-series cross-sectional data.</p>
<p>Note: your data needs to be in ascending time order with equally spaced time increments. For example 1995, 1996, 1997. </p>
<hr>
<h2>Examples</h2>
<h3>Not Cross-sectional data</h3>
<p>Let's create an example data set with three variables:</p>
<pre><code class="r"># Create time variable
Year <- 1980:1999
# Dummy covariates
A <- B <- 1:20
Data1 <- data.frame(Year, A, B)
head(Data1)
</code></pre>
<pre><code>## Year A B
## 1 1980 1 1
## 2 1981 2 2
## 3 1982 3 3
## 4 1983 4 4
## 5 1984 5 5
## 6 1985 6 6
</code></pre>
<p>Now let's lag the <code>A</code> variable by one time unit.</p>
<pre><code class="r">library(DataCombine)
DataSlid1 <- slide(Data1, Var = "A", slideBy = -1)
head(DataSlid1)
</code></pre>
<pre><code>## Year A B A-1
## 1 1980 1 1 NA
## 2 1981 2 2 1
## 3 1982 3 3 2
## 4 1983 4 4 3
## 5 1984 5 5 4
## 6 1985 6 6 5
</code></pre>
<p>The lag variable is automatically given the name <code>A-1</code>.</p>
<p>To lag a variable (i.e. the lag value at a given time is the value of the non-lagged variable at a time in the past) set the <code>slideBy</code> argument as a negative number. Lead variables, are created by using positive numbers in <code>slideBy</code>. Lead variables at a given time have the value of the non-lead variable from some time in the future.</p>
<h3>Time-series Cross-sectional data</h3>
<p>Now let's use <code>slide</code> to create a lead variable with time-series cross-sectional data. First create the example data:</p>
<pre><code class="r"># Create time and unit ID variables
Year <- rep(1980:1983, 5)
ID <- sort(rep(seq(1:5), 4))
# Dummy covariates
A <- B <- 1:20
Data2 <- data.frame(Year, ID, A, B)
head(Data2)
</code></pre>
<pre><code>## Year ID A B
## 1 1980 1 1 1
## 2 1981 1 2 2
## 3 1982 1 3 3
## 4 1983 1 4 4
## 5 1980 2 5 5
## 6 1981 2 6 6
</code></pre>
<p>Now let's create a two time unit lead variable based on <code>B</code> for each unit identified by <code>ID</code>:</p>
<pre><code class="r">DataSlid2 <- slide(Data2, Var = "B", GroupVar = "ID",
slideBy = 2)
head(DataSlid2)
</code></pre>
<pre><code>## Year ID A B B2
## 1 1980 1 1 1 3
## 2 1981 1 2 2 4
## 3 1982 1 3 3 NA
## 4 1983 1 4 4 NA
## 5 1980 2 5 5 7
## 6 1981 2 6 6 8
</code></pre>
<p>Hopefully you'll find <code>slide</code> useful in your own data analysis. Any suggestions for improvement are always welcome.</p>Unknownnoreply@blogger.com46tag:blogger.com,1999:blog-36784551.post-55036587423364719872013-04-17T07:36:00.000+01:002013-04-17T09:39:31.951+01:00Reinhart & Rogoff: Everyone makes coding mistakes, we need to make it easy to find them + Graphing uncertainty<p>You may have already seen a lot written on the replication of Reinhart & Rogoff’s (R & R) <a href="http://qz.com/75117/how-influential-was-the-study-warning-high-debt-kills-growth/">much cited</a> 2010 <a href="http://www.nber.org/papers/w15639.pdf">paper</a> done by <a href="http://www.peri.umass.edu/fileadmin/pdf/working_papers/working_papers_301-350/WP322.pdf">Herndon, Ash, and Pollin</a>. If you haven’t, here is a round up of some of some of what has been written: <a href="http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems#.UW14rDQo2L4.twitter">Konczal</a>, <a href="http://www.slate.com/blogs/moneybox/2013/04/16/reinhart_rogoff_coding_error_austerity_policies_founded_on_bad_coding.html">Yglesias</a>, <a href="http://krugman.blogs.nytimes.com/2013/04/16/holy-coding-error-batman/?utm_source=feedly">Krugman</a>, <a href="http://marginalrevolution.com/marginalrevolution/2013/04/an-update-on-the-reinhart-and-rogoff-critique-and-some-observations.html">Cowen</a>, <a href="http://simplystatistics.org/2013/04/16/i-wish-economists-made-better-plots/">Peng</a>, <a href="http://blogs.ft.com/ftdata/2013/04/17/the-reinhart-rogoff-response-i/?utm_source=dlvr.it&utm_medium=twitter">FT Alphaville</a>.</p>
<p>This is an interesting issue for me because it involves three topics I really like: political economy, reproducibility, and communicating uncertainty. Others have already commented on these topics in detail. I just wanted to add to this discussion by (a) talking about how this event highlights a real need for researchers to use systems that make <strong>finding and correcting mistakes</strong> easy, (b) <strong>incentivising mistake finding/correction</strong> rather than penalising it, and (c) showing <strong>uncertainty</strong>.</p>
<h2 id="systemsforfindingandcorrectingmistakes">Systems for Finding and Correcting Mistakes</h2>
<p>One of the problems Herndon, Ash, and Pollin found in R&R’s analysis was and <a href="http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems#.UW14rDQo2L4.twitter">Excel coding error</a>. I love to hate on Excel as much as the next R devotee, but I think that is missing the point. The real lesson is not “don’t use Excel” the real lesson is: <strong>we all make mistakes</strong>. </p>
<p>(Important point: I refer throughout this post to errors caused by coding mistakes rather than purposeful fabrications and falsifications.)</p>
<p>Coding mistakes are an ever present part of our work. The problem is not that we make coding mistakes. Despite our best efforts we always will. The problem is that we often use tools and practices that make it difficult to <strong>find and correct our mistakes</strong>. </p>
<p>This is where I can get in some Excel hating: tools and practices that make it difficult to find mistakes include binary files (like Excel’s) that can’t be version controlled in a way that fully reveals the research process, not commenting code, not making your data readily available in formats that make replication easy, not having a system for quickly fixing mistakes when they are found. Sorry R users, but the last three are definitely not exclusive to Excel. </p>
<p>It took Herndon, Ash, and Pollin a considerable amount of time to replicate R & R’s findings and therefore find the Excel error. This seems partially because R & R did not make their analysis files readily available (Herndon, Ash, and Pollin had to ask for them). I’m not sure how this error is going to be corrected and documented. But I imagine it will be like most research corrections: kind of on the fly, mostly emailing and reposting. </p>
<p><strong>How big of a detail is this?</strong> There is some debate over how big of a problem this mistake is. Roger Peng ends <a href="http://simplystatistics.org/2013/04/16/i-wish-economists-made-better-plots/">his really nice post</a>: </p>
<blockquote>
<p>The vibe on the Internets seems to be that if only this problem had been identified sooner, the world would be a better place. But my cynical mind says, uh, no. You can toss this incident in the very large bucket of papers with some technical errors that are easily fixed. Thankfully, someone found these errors and fixed them, and that’s a good thing. Science moves on.</p>
</blockquote>
<p>I agree with most of this paragraph. But, given how <a href="http://qz.com/75117/how-influential-was-the-study-warning-high-debt-kills-growth/">important R & R’s finding was</a> to major policy debates it would have been much better if the mistake was caught sooner rather than later. The tools and practices R & R used made it harder to find and correct the mistake, so policymakers were operating with less accurate information for longer. </p>
<p><strong>Solutions:</strong> I’ve written in some detail in the <a href="http://polmeth.wustl.edu/methodologist/tpm_v20_n2.pdf">most recent issue of The Political Methodologist</a> about how cloud-based version control systems like <a href="https://github.com/">GitHub</a> can be used to make finding and correcting mistakes easier. Pull requests, for example, are a really nice way to directly suggest corrections. </p>
<h2 id="incentivisingerrorfindingandcorrection">Incentivising Error Finding and Correction</h2>
<p>Going forward I think it will be interesting to see how this incident shapes researchers’ perceived <strong>incentives</strong> to make their work easily replicable. Replication is an important part of finding the mistakes that everyone makes. If being found to make a coding mistake (not a fabrication) has a negative impact on your academic career then there are incentives to make finding mistakes difficult, by for example making replication difficult. Most papers do not receive nearly as much attention as R & R’s. So, for most researchers making replication difficult will make it pretty unlikely that anyone will replicate your research and you’ll be home free. </p>
<blockquote class="twitter-tweet"><p>I can't send you my data b/c I think you might find out I made an error. <a href="https://twitter.com/search/%23overlyhonestmethods">#overlyhonestmethods</a></p>— Carlisle Rainey (@carlislerainey) <a href="https://twitter.com/carlislerainey/status/289097513926017024">January 9, 2013</a></blockquote>
<script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>This is a perverse incentive indeed. </p>
<p>What can we do? Many journals now require replicable code to accompany published articles. This is a good incentive. Maybe we should go further, and somehow directly incentivise the finding and correction of errors in data sets and analysis code. Ideas could include giving more weight to replication studies at hiring and promotion committees. Maybe even allowing these committees to include information on researchers’ GitHub pull requests that meaningfully improve other’s work by correcting mistakes. </p>
<p>This of course might create future perversion incentives to add errors so that they can then be found. I think this is a bit fanciful. There are surely enough negative social incentives (i.e. embarrassment) surrounding making mistakes to prevent this.</p>
<h2 id="showinguncertainty">Showing Uncertainty</h2>
<p>Roger Peng’s post highlighted the issue of graphing uncertainty, but I just wanted to build it out a little further. The interpretation of the correlation R & R’s found between GDP Growth and Government Debt could have been tempered significantly before any mistakes were found by more directly communicating their original uncertainty. In their original paper, they presented the relationship using bar graphs of average and median GDP growth per grouped debt/GDP level: </p>
<a href="http://4.bp.blogspot.com/-X0L2Xmd2w9Q/UW4-vIxxruI/AAAAAAAAGHQ/3_ygu72_LOQ/s1600/RAndR.png" imageanchor="1" ><img border="0" src="http://4.bp.blogspot.com/-X0L2Xmd2w9Q/UW4-vIxxruI/AAAAAAAAGHQ/3_ygu72_LOQ/s320/RAndR.png" /></a>
<p>Beyond showing the mean and median there is basically no indication of the distribution of the data they are from. </p>
<p>Herndon, Ash, and Pollin put together some nice graphs of these distributions (and avoid that thing economists do of using two vertical axis with two different meanings). </p>
<a href="http://4.bp.blogspot.com/-ROjeb9jEDdE/UW4--9aLR4I/AAAAAAAAGHY/UCtmtDbtqYo/s1600/Rep2.png" imageanchor="1" ><img border="0" src="http://4.bp.blogspot.com/-ROjeb9jEDdE/UW4--9aLR4I/AAAAAAAAGHY/UCtmtDbtqYo/s320/Rep2.png" /></a>
<p>Here is one that gets rid of the groups altogether:</p>
<a href="http://4.bp.blogspot.com/-sIJUoaU9P_s/UW4_C3FIaiI/AAAAAAAAGHg/nRrRsn-_UIY/s1600/Rep3.png" imageanchor="1" ><img border="0" src="http://4.bp.blogspot.com/-sIJUoaU9P_s/UW4_C3FIaiI/AAAAAAAAGHg/nRrRsn-_UIY/s320/Rep3.png" /></a>
<p>If R & R had shown a simple scatter plot like this (though they did exclude some of the higher GDP Growth country-years at the high debt end, so their's would have looked different), it would have been much more difficult to overly interpret the substantive–policy–value of a correlation between GDP/growth and debt/GDP. </p>
<p>Maybe this wouldn’t have actually changed the policy debate that much, As Mark Blyth argues in his <a href="http://www.oup.com/us/catalog/general/subject/Economics/Political/?view=usa&ci=9780199828302">recent book</a> on austerity “facts never disconfirm a good ideology” (p. 18). But at least Paul Krugman might not have had to debate debt/GDP cutoff points on CNBC (for example time point 12:40):</p>
<p><iframe width="420" height="315" src="http://www.youtube.com/embed/9dcPhfraCnY" frameborder="0" allowfullscreen></iframe></p>
<hr />
<p><strong>P.S.</strong> To R & R’s credit, they do often make their data <a href="http://www.carmenreinhart.com/data/">available</a>. Their data has been useful for at least one of my <a href="http://ssrn.com/abstract=2155986">papers</a>. However, it is often available in a format that is hard to use for cross-country statistical analysis, including, I would imagine, their own. Though I have never found any errors in the data, reporting and implementing corrections to this data would be piecemeal at best. </p>Unknownnoreply@blogger.com7tag:blogger.com,1999:blog-36784551.post-25319851146633276592013-04-11T23:05:00.000+01:002013-04-12T08:56:38.979+01:00Dropbox & R Data<p>I'm always looking for ways to download data from the internet into R. Though I prefer to host and access plain-text data sets (CSV is my personal favourite) from <a href="https://github.com/">GitHub</a> (see my <a href="http://polmeth.wustl.edu/methodologist/tpm_v20_n2.pdf">short paper</a> on the topic) sometimes it's convenient to get data stored on <a href="https://www.dropbox.com/">Dropbox</a>. </p>
<p>There has been a change in the way Dropbox URLs work and I just added some functionality to the <a href="http://christophergandrud.github.io/repmis/">repmis</a> R package. So I though that I'ld write a quick post on how to directly download data from Dropbox into R. </p>
<p>The download method is different depending on whether or not your <em>plain-text</em> data is in a Dropbox Public folder or not.</p>
<h2>Dropbox Public Folder</h2>
<p>Dropbox is trying to do away with its public folders. New users need to <a href="https://www.dropbox.com/help/16/en">actively create</a> a Public folder. Regardless, sometimes you may want to download data from one. It used to be that files in Public folders were accessible through non-secure (http) URLs. It's easy to download these into R, just use the <code>read.table</code> command, where the URL is the file name. Dropbox recently changed Public links to be secure (https) URLs. These cannot be accessed with <code>read.table</code>. </p>
<p>Instead you need can use the <code>source_data</code> command from <strong>repmis</strong>:</p>
<pre><code class="R">FinURL <-"https://dl.dropbox.com/u/12581470/code/Replicability_code/Fin_Trans_Replication_Journal/Data/public.fin.msm.model.csv"
# Download data
FinRegulatorData <- repmis::source_data(FinURL,
sep = ",",
header = TRUE)
</code></pre>
<h2>Non-Public Dropbox Folders</h2>
<p>Getting data from a non-Public folder into R was a trickier. When you click on a Dropbox-based file's <a href="https://www.dropbox.com/help/167/en">Share Link</a> button you are taken to a secure URL, but not for the file itself. The Dropbox webpage you're taken to is filled with lots of other Dropbox information. I used to think that accessing a plain-text data file embedded in one of these webpages would require some tricky web scrapping. Luckily, today I ran across this <a href="http://thebiobucket.blogspot.kr/2013/04/download-files-from-dropbox.html">blog post</a> by Kay Cichini. </p>
<p>With some modifications I was able to easily create a function that could download data from non-Public Dropbox folders. The <code>source_DropboxData</code> command is in the most recent version of <strong>repmis</strong> (v0.2.4) is the result. All you need to know is the name of the file you want to download and its Dropbox key. You can find both of these things in the URL for the webpage that appears when you click on <code>Share Link</code>. Here is an example:</p>
<blockquote>https://www.dropbox.com/s/exh4iobbm2p5p1v/fin_research_note.csv</blockquote>
<p>The file name is at the very end (<code>fin_research_note.csv</code>) and the key is the string of letters and numbers in the middle (<code>exh4iobbm2p5p1v</code>). Now we have all of the information we need for <code>source_DropboxData</code>:</p>
<pre><code class="R">FinDataFull <- repmis::source_DropboxData("fin_research_note.csv",
"exh4iobbm2p5p1v",
sep = ",",
header = TRUE)
</code></pre>Unknownnoreply@blogger.com51