Skip to main content

Posts

Showing posts with the label text analysis

Simple Text Web Crawler

I put together a simple web crawler for R . It's useful if you are doing any text analysis and need to make .txt files from webpages. If you have a data frame of URLs it will cycle through them and grab all the websites. It strips out the HTML code. Then it saves each webpage as an individual text file. Thanks to Rex Douglass, also.  Enjoy (and please feel free to improve)