What does the topic extraction task consist of?

The detection of themes, topics or topic extraction consists of the unsupervised automatic processing of texts with the objective of identifying the issues or reasons on which said texts deal . In other words, using this NLP technique it is possible to discover what specific topics are covered in a set of documents, for example. Apart from the themes underlying the data as a whole, it is possible to automatically know what each individual document is about or the frequency with which each theme appears . Furthermore, theme extraction can be applied to a set of documents to know how much they are similar or how much they differ from each other, that is, to know if the data sample is more heterogeneous or homogeneous thematically speaking. Topic detection consists of the unsupervised automatic processing of texts to identify topics or reasons that the texts address. The extraction of themes becomes especially relevant when it is framed in a specific context or in a particular area of ​​knowledge or business.



Thus, this technique is applicable to countless cases such as, for example, knowing what topics are covered in Spanish filmography of the last 20 years, aspects that are discussed in the opinions of website users who value the Germany Telemarketing Data satisfaction of a business or restaurant, trends and fashions (diet, physical exercise, habits, etc.) on social networks or issues that are covered by the legislation of a country (and those that are not) at a specific time. How to build themes step by step with NLP But how do you discover the themes underlying a data set? Topic extraction can be divided into several steps that start, like almost any NLP task, from text processing. Figure 2 summarizes the steps to follow in extracting themes. proceso topics Figure 2. Theme extraction process. The first step is data cleaning . The objective of this phase is to eliminate repeated texts or check their encoding, that is, to observe if there is any character that is not displayed correctly or that is not part of the language to which the data belongs. After cleaning the data, the second step is to apply a topic extraction algorithm.







This algorithm works, broadly speaking, in three phases: Software maintenance. The first phase consists of translating each of the texts into its numerical equivalent (vectors) . Thus, subsequent layers of processing can extract patterns from the content of the texts on which to establish comparisons. In the second phase, the theme extraction algorithm establishes these comparisons between the texts and forms groups with those texts that are most similar thematically . Furthermore, another group is generated with texts that do not specifically belong to any of the most salient themes detected, but rather constitute specific mentions of various aspects. The third phase consists of deciding which terms are most appropriate to represent the topic that each of the groups of texts talks about. These terms are the key to being able to interpret, always within the specific context of analysis, the information extracted from the data. Below is a practical case applied to the legal field. Topic extraction case: 'millennial' sentences vs. 'zentennial' sentences In accordance with Royal Decree 181/2008, of February 8, “the «Official State Gazette» (BOE), official newspaper of the Spanish State, is the means of publication of the laws, provisions and acts of mandatory insertion. The BOE contains different types of documents: laws and royal decrees, rulings, official bulletins of the Autonomous Communities or the Official Journal of the European Union in Spanish.