Recently while doing some topic modelling, I encountered a few problems such as: How to use the topic coherence (TC) pipeline with other topic models (eg. HDP). How to find the optimal number of topics for LDA. LSI is brilliant since it ranks its topics. Can LDA do that too? If you face such problems, this blog might be able …
Validating gensim’s topic coherence pipeline
Sorry for not posting in such a long while. It had been a turbulent few weeks with some sharp twists and turns involving mails flying back and forth and a few pivots here and there. To validate the topic coherence pipeline in gensim, my plan was to work with the RTL-Wiki corpus and reproduce the results stated in the paper. …
What is Topic Coherence?
What exactly is this topic coherence pipeline thing? Why is it even important? Moreover, what is the advantage of having this pipeline at all? In this post I will look to answer those questions in an as non-technical language as possible. This is meant for the general reader as much as a technical one so I will try to engage …