Skip to main content

Posts

FillIn: a function for filling in missing data in one data frame with info from another

Update (10 March 2013): FillIn is now part of the budding DataCombine package. Sometimes I want to use R to fill in values that are missing in one data frame with values from another. For example, I have data from the World Bank on government deficits. However, there are some country-years with missing data. I gathered data from Eurostat on deficits and want to use this data to fill in some of the values that are missing from my World Bank data. Doing this is kind of a pain so I created a function that would do it for me. It's called FillIn . An Example Here is an example using some fake data. (This example and part of the function was inspired by a Stack Exchange conversation between JD Long and Josh O'Brien.) First let's make two data frames: one with missing values in a variable called fNA . And a data frame with a more complete variable called fFull . # Create data set with missing values naDF <- data.frame(a = sample(c(1,2), 100, rep=TRUE), ...

InstallOldPackages: a repmis command for installing old R package versions

A big problem in reproducible research is that software changes. The code you used to do a piece of research may depend on a specific version of software that has since been changed. This is an annoying problem in R because install.packages only installs the most recent version of a package. It can be tedious to collect the old versions. On Toby Dylan Hocking 's suggestion, I added tools to the repmis package so that you can install, load, and cite specific R package versions. It should work for any package version that is stored on the CRAN archive ( http://cran.r-project.org ). To only install old package versions use the new repmis command InstallOldPackages . For example: # Install old versions of the e1071 and gtools packages. # Create vectors of the package names and versions to install # Note the names and version numbers must be in the same order Names &lt- c("e1071", "gtools") Vers &lt- c("1.6", "2.6.1") # Install...

repmis: misc. tools for reproducible research in R

I've started to put together an R package called repmis . It has miscellaneous tools for reproducible research with R. The idea behind the package is to collate commands that simplify some of the common R code used within knitr -type reproducible research papers. It's still very much in the early stages of development and has two commands: LoadandCite : a command to load all of the R packages used in a paper and create a BibTeX file containing citation information for them. It can also install the packages if they are on CRAN . source_GitHubData : a command for downloading plain-text formatted data stored on GitHub or at any other secure (https) URL. I've written about why you might want to use source_GitHubData before (see here and here ). You can use LoadandCite in a code chunk near the beginning of a knitr reproducible research document to load all of the R packages you will use in the document and automatically generate a BibTeX file you can draw on to c...

source_GitHubData: a simple function for downloading data from GitHub into R

Update 31 January: I've folded source_GitHubData into the repmis packaged. See this post . Update 7 January 2012: I updated the internal workings of source_GitHubData so that it now relies on httr rather than RCurl . Also it is more directly descended from devtool 's source_url command. This has two advantages. Shortened URL's can be used instead of the data sets' full GitHub URL, The ssl.verifypeer issue is resolved. (Though please let me know if you have problems). The post has been rewritten to reflect these changes. In previous posts I've discussed how to download data stored in plain-text data files (e.g. CSV, TSV) on GitHub directly into R. Not sure why it took me so long to get around to this, but I've finally created a little function that simplifies the process of downloading plain-text data from GitHub. It's called source_GitHubData . (The name mimicks the devtools syntax for functions like source_gist and source_url...

Update to Graphing Non-Proportional Hazards in R

Update 31 July 2013: I've moved all of the functionality described in this post into an R package called simPH . Have a look. It is much easier to use. This is a quick update for a previous post on Graphing Non-Proportional Hazards in R . In the previous post I showed how to simulate and graph 1,000 non-proportional hazard ratios at roughly every point in time across an observation period. In the previous example I kept in simulation outliers. Some people have suggested dropping the top and bottom 2.5 percent of simulated values (i.e. keeping the middle 95 percent). Luckily this can be accomplished with Hadley Wickham 's plyr package and three lines of code. The trick is to use plyr's ddply command to subset the data frame at each point in Time where we simulated values. In the previous example the simulated values were in a variable called HRqmv . In each subset we use the quantile command from base R to create logical variables indicating if a simulation o...

Interesting: Scraply

Ran across a new R package in development on GitHub. It's called scraply . It claims to provide "error-proof scraping in R". This could be a better solution than the one I was working on last year (see HERE ). Haven't tried it yet, though.

Timeline Maps with googleVis & Twitter Bootstrap Carousel (& updated Slidify)

I've wanted to create timeline maps with interactive googleVis Geomaps for a while. These would be a nice way to quickly show the spatial distribution of some data over time. It turns out that it's pretty easy to do with a plugin for Twitter Bootstrap called Carousel . Carousel is probably intended for regular picture slide shows. But because it can hold iframes, it can pretty much include anything, even interactive maps. Here is a short slide show with examples and code for how to combine googleVis and Twitter Bootstrap Carousel to create interactive timeline maps. Note: I used the newest version (0.3.1) of Ramnath Vaidyanathan's Slidify to create the presentation. He is really putting a lot of good work into that package. I especially like the choice to set the default slide framework to Google's I/O 2012 style. It has many features you don't find in other HTML slide frameworks. Particularly useful here, it begins to load iframes when you are on the ...

Graphing Non-Proportional Hazards in R

Update 30 July 2013: I've moved all of the functionality described in this post into an R package called simPH . Have a look. It is much easier to use. Update 30 December 2012: I updated the code HERE so that it keeps only the middle 95 percent of the simulated values. I really like this article by Amanda Licht in Political Analysis. It gives a lot of information on how to interpret nonproportional hazards and includes some nice graphs. Her source code is really helpful for learning the nuts and bolts of how to simulate quantities of interests over time. However, it's in Stata code, which doesn't really fit into my R-based workflow at the moment. So I decided to port the code over. This post gives an example of what I did. What is a non-proportional hazard & why use them? Here is my motivation for being interested in non-proportional hazards: In a few papers I used Cox Proportional Hazard (PH) models to examine countries' policy adoption dec...