Skip to main content

A Link Between topicmodels LDA and LDAvis

Carson Sievert and Kenny Shirley have put together the really nice LDAvis R package. It provides a Shiny-based interactive interface for exploring the output from Latent Dirichlet Allocation topic models. If you've never used it, I highly recommend checking out their XKCD example (this paper also has some nice background).

LDAvis doesn't fit topic models, it just visualises the output. As such it is agnostic about what package you use to fit your LDA topic model. They have a useful example of how to use output from the lda package.

I wanted to use LDAvis with output from the topicmodels package. It works really nicely with texts preprocessed using the tm package. The trick is extracting the information LDAvis requires from the model and placing it into a specifically structured JSON formatted object.

To make the conversion from topicmodels output to LDAvis JSON input easier, I created a linking function called topicmodels_json_ldavis. The full function is below. To use it follow these steps:

  1. Create a VCorpus object using the tm package's Corpus function.

  2. Convert this to a document term matrix using DocumentTermMatrix, also from tm.

  3. Run your model using topicmodel's LDA function.

  4. Convert the output into JSON format using topicmodels_json_ldavis. The function requires the output from steps 1-3.

  5. Visualise with LDAvis' serVis.

Comments

titaniumtroop said…
At line 32, the method 'inspect' should instead by 'as.matrix', because inspect just provides a sample.
titaniumtroop said…
Nice post!
rmouniak said…
It was really a nice post and I was really impressed by reading this AWS Online Training
shivani said…
An astounding web diary I visit this blog, it's inconceivably magnificent. Strangely, in this current blog's substance made point of fact and sensible. The substance of information is instructive.
Oracle Fusion Financials Online Training
Oracle Fusion HCM Online Training
Oracle Fusion SCM Online Training
Taylor said…
This post was very helpful, but is there an easy way to assign the reordered topics that the LDAVis produces to the original LDA (i.e. 'fitted')?
Pankaj Singh said…
Thankful to you for this amazing information sharing with us. Get website designing and development services by Ogen Infosystem.
SEO Service in Delhi
shivani said…
A bewildering web journal I visit this blog, it's unfathomably heavenly. Oddly, in this present blog's substance made purpose of actuality and reasonable. The substance of data is informative
Oracle Fusion Financials Online Training
Oracle Fusion HCM Online Training
Oracle Fusion SCM Online Training
basha said…
Excellent blog I visit this blog it's really awesome. The important thing is that in this blog content written clearly and understandable. The content of information is very informative.
Magnificent blog I visit this blog it's extremely wonderful. Interestingly, in this blog content composed plainly and reasonable. The substance of data is useful.
Oracle Fusion HCM Online Training
Oracle Fusion SCM Online Training
Oracle Fusion Financials Online Training
Big Data and Hadoop Training In Hyderabad
oracle fusion financials classroom training
Oracle Fusion HCM Classroom Training
oracle cpq online training / Oracle CPQ Class Room Training
Oracle Taleo Online Training
jeewangarg said…
Wow!!! It was really an Informational Article which provide me with much Insightful Information. I would like to first thank the author for such an Insightful Content. If someone here is looking for a Website Designing Company in India, then look no further than Jeewangarg.com as we are the Best website designing in Delhi working in the arena for the last 8 years.
Best website designing company in Delhi NCR
website designing company in India
Chanson Water said…

Great website, continue the good work! Also visit my site – alkaline water machine
Servo said…
Thanks for the great article and post. Need a servo voltage stabilizer manufacturers in india? Don’t go beyond servo star “A popular 10 kva single phase voltage stabilizer which offers an extensive range of exceptional servo stabilizer. Feel Free to contact.
nexusups said…
Thanks for the great article and post. Need a Industrial UPS? Don’t go beyond nexusups. “A popular Industrial UPS which offers an extensive range of exceptional Industrial UPS, and Online UPS Free to contact.
Imperial Garden said…
Imperial Garden is the best wedding garden in Faridabad. The garden is capable of easily accommodating 2000 to 3000 people with easy connectivity and a free parking zone. Call now : 9990977115
I would first like to thank the author for coming up with the insightful information every time. People if you are also looking for the best shimla tour packages | Best Rajasthan Tour Packages then reach out to Indian Travel Store as we are the best Travel Partners of yours.
rightselects said…
Thanks for posting such a great post . Right Selects provides its users with the exact information of the “top 10” or “top 5” industry leader of the particular industry. For instance, if you are looking for selecting the best among the array of water purifiers in India, you can visit us for getting the most accurate information about “Top 10 water purifier for home”. Additionally, at right selects, you can have the information on lifestyle products, electronics, appliances, water purifiers, restaurants as well as about the best travel places.
phautecouture said…

Thank You Author!!! You are sharing the very informative and great post. If anyone is looking to lehenga indian wedding dresses | manish malhotra designer sarees then visit Panache Haute Couture, a leading Indian Designer House for Indian Dresses.
Ethan said…
Really Amazing Blog. In Case You Wish You Read Some Similar Blogs Or Find A Great Choice Of Everything Online, Use The Coupons Codes And Discounts Codes From Our Site.
Coupons Codes
Blue Marry said…
Your work is appreciable and you have amazing Blogging skills. I really recommended the readers to follow and share your idea in circles. I hope readers found it’s beneficial. Peoples should start blogging and try to forward information to others. Thanks and regards. Coupon and Discount Codes at LetCoupon
Olivia Lua said…
ftv is a Porn Training site. ftv is the largest Amateur porn training institute with the hottest selection Online and offline real life cam.
sex video
NICT CSP said…
Thanks for sharing such a great information, Here we are India's best service provider of SBI Kiosk Banking.
kiosk banking
online sbi kiosk
Charles said…
Thank you so much for this nice information. Hope so many people will get aware of this and useful as well. And please keep update like this.

Text Analytics Companies

Text Analytics Python
Debug said…
Wow!!Great post. Thanks for sharing this informative article. Keep it up.
I Need Trip - International Honeymoon Packages & Holiday Packages:-
Bali Packages
Singapore Packages
Mauritius Packages
Maldives Packages
Dubai Packages
sylvia said…
The information which you have provided is very good. It is very useful who is looking for Big data consulting services Singapore
Vipin said…
Thanks for provide great informatic and looking beautiful blog, really nice required information & the things i never imagined and i would request, wright more blog and blog post like that for us. Thanks you once agian

name change procedure in chandigarh
name change procedure in delhi
name change procedure mumbai
name change procedure in jaipur
name change in pune
name change procedure in gurgaon
name change in india
name change procedure in bangalore
name change procedure in rajasthan
name change procedure in maharashtra
Unknown said…
It was a great write up about Digital Marketing. Thank You for providing with such information. Connect to the best Flipkart Listing Services – ITDigiTech Solutions to avail the . Amazon SEO Services
Online Front said…
Really glad to say that it's an awesome post to read and learn something new, I always come to know about something interesting and unique. Good job and waiting for another one also. We also offers some services are:-
Digital marketing Company
SMM Services
PPC Ads Services
PPC Services in Delhi
Website Design & Development Packages
Web Development Packages
Web Development Package
Tender Gourmet Butchery is the Butcher shop in Australia. With over 30 years collective experience in the industry and over 200 awards, we pride ourselves on stocking and supplying only the highest-grade Australian meats in our award-winning shops.
Pankaj Singh said…
Nice Blog. Ogen Infosystem one of the best Website Designing Company in Delhi and also get digital marketing services like SEO, PPC, and SMO Services at best price for your business.
SEO Services in Delhi
Big Data 360 said…
Just saying thanks will not just be sufficient, for the fantasti c lucidity in your writing. I will instantly grab your rss feed to stay informed of any updates.

Data Science Course in Trichy

Popular posts from this blog

Set up R/Stan on Amazon EC2

A few months ago I posted the script that I use to set up my R/JAGS working environment on an Amazon EC2 instance. Since then I've largely transitioned to using R/ Stan to estimate my models. So, I've updated my setup script (see below). There are a few other changes: I don't install/use RStudio on Amazon EC2. Instead, I just use R from the terminal. Don't get me wrong, I love RStudio. But since what I'm doing on EC2 is just running simulations (I handle the results on my local machine), RStudio is overkill. I don't install git anymore. Instead I use source_url (from devtools) and source_data (from repmis) to source scripts from GitHub. Again all of the manipulation I'm doing to these scripts is on my local machine.

Slide: one function for lag/lead variables in data frames, including time-series cross-sectional data

I often want to quickly create a lag or lead variable in an R data frame. Sometimes I also want to create the lag or lead variable for different groups in a data frame, for example, if I want to lag GDP for each country in a data frame. I've found the various R methods for doing this hard to remember and usually need to look at old blog posts . Any time we find ourselves using the same series of codes over and over, it's probably time to put them into a function. So, I added a new command– slide –to the DataCombine R package (v0.1.5). Building on the shift function TszKin Julian posted on his blog , slide allows you to slide a variable up by any time unit to create a lead or down to create a lag. It returns the lag/lead variable to a new column in your data frame. It works with both data that has one observed unit and with time-series cross-sectional data. Note: your data needs to be in ascending time order with equally spaced time increments. For example 1995, 1996