Sentiment analysis serves as a fundamental aspect of dealing with customers on online portals and websites for the companies. They do this all the time to classify a comment as a query, complaint, suggestion, opinion, or just love for a product. This way they can easily sort through the comments or questions and prioritize what they need to handle first and even order them in a way that looks better. Companies sometimes even try to delete content that has a negative sentiment attached to it.<\/p>\n
It is an easy way to understand and analyze public reception and perception of different ideas and concepts, or a newly launched product, maybe an event or a government policy.<\/p>\n Emotion understanding and sentiment analysis play a huge role in collaborative filtering based recommendation systems. Grouping together people who have similar reactions to a certain product and showing them related products. Like recommending movies to people by grouping them with others that have similar perceptions for a certain show or movie.<\/p>\n Lastly, they are also used for spam filtering and removing unwanted content.<\/p>\n NLP<\/strong> or natural language processing is the basic concept on which sentiment analysis is built upon. Natural language processing is a superclass of sentiment analysis that deals with understanding all kinds of things from a piece of text.<\/p>\n NLP is the branch of AI dealing with texts, giving machines the ability to understand and derive from the text. For tasks such as virtual assistant, query solving, creating and maintaining human-like conversations, summarizing texts, spam detection, sentiment analysis, etc. it includes everything from counting the number of words to a machine writing a story, indistinguishable from human texts.<\/p>\n Sentiment analysis can be classified into various categories based on various criteria. Depending upon the scope it can be classified into document-level sentiment analysis, sentence level sentiment analysis, and sub sentence level or phrase level sentiment analysis.<\/p>\n Also, a very common classification is based on what needs to be done with the data or the reason for sentiment analysis. Examples of which are<\/p>\n Based on what needs to be done and what kind of data we need to work with there are two major methods of tackling this problem.<\/p>\n In this machine learning project, we will use recurrent neural network for sentiment analysis in python.<\/strong><\/p>\n Recurrent neural networks or RNNs are the go-to method for sequential data. Recurrent neural networks have a memory of their own and remember the input that was given to each node. In a normal feed forward neural network data or information given in the form of input moves forward and never moves backward in any nodes, from the input layer to the hidden layer and out from the output layer. Because these kinds of networks have no memory they do not remember the last input or predict the next input.<\/p>\n When giving out an output an rnn considers what input it is given and also the input that it had received previously, because the information moves in a cycle in an rnn.<\/p>\n What a recurrent neural network does is creates a copy of the output and loops it back through the network. For example when a sentence is passed through a feedforward network and it takes it word by word, till it reaches the last word. It has no memory of what was fed to it before that. But rnns know the previous inputs as well and can thus also predict what can come next. They are widely used in sequential data.<\/p>\n Recurrent neural networks are not new, were first introduced in the 1980s, but have become a lot popular with the growth of deep learning and its use in sequential data. Still, rnn have their own set of problems, one of the major ones being the vanishing gradient problem. The answer to that being the lstm model.<\/p>\n LSTM or long short term memory models are used to solve these problems. Lstms are a special kind of rnn model that has the capability of learning long term dependencies. They are made to remember long term data. What makes lstm models special is the addition of a memory cell, which is an extra recurrent state and each cell has multiple gates that control the flow of information in and out of the memory cell.<\/p>\n There are three types of gates in an lstm: an input gate, an output gate and a forget gate. The input gate is used to update the cell status, the forget state decides which information is to be stored and which needs to be discarded. The output gate determines the values for the next hidden gates.<\/p>\n Now that we have a basic understanding of what we need to build we will try implementing it in an easy way.<\/p>\n We will be building a simple sentiment analysis classifier on top of movie reviews, that will classify if the user review of the movie was positive, negative or neutral. For this sentiment analysis python project, we are going to use the imdb movie review dataset.<\/p>\n Sentiment analysis is the process of finding users’ opinions towards a brand, company, or product. It defines the subject behind the social data, after launching a product we can find whether people are liking the product or not. There are many use-cases for sentiment analysis apart from opinion mining.<\/p>\n Please download the source code sentiment analysis in Python: Sentiment Analysis Python ML Code<\/strong><\/a><\/p>\n The dataset which we will use in sentiment analysis is the International Movie Database(IMDb) reviews for 50,000 reviews of movies from all over the world, its a binary classification dataset categorizing each review in a positive or negative. It has 25000 samples for training and 25000 for testing.<\/p>\n You don’t need to download it separately for this project but you can have a look at it on its official website<\/a>. Because it is a text dataset it is very lightweight around 80MB.<\/p>\n We are going to code all this up in a jupyter notebook on google colab to make use of the free gpu. If you follow along on your own system everything will be pretty much the same except for mounting the google drive for use as a persistent storage option.<\/p>\n We are going to use pytorch for this project and luckily it comes preinstalled with some functionalities for helping us speeding up our work<\/p>\n The torch.text library is a great tool for nlp projects. It has a loader for some common nlp datasets like the one we are going to use today, also complete pipeline for abstraction of vectorization of data, data loaders and iteration of data.<\/p>\n We are going to use the field method of the data class to decide how data needs to be preprocessed. The parameters we pass in there will determine the preprocessing, we are just going to tweak some parameters and leave the rest to default.<\/p>\n The first parameter, our tokenizer(that determines how the sentences are going to be broken down or tokenized in nlp standard) is spacy, which is a powerful tool for one line tokenization. We recommend using this, but the default is just tokenizing the string based on blank spaces. Also we need to tell the spacy tokenizer which language model to use for the task.<\/p>\n We also set the random seed to a certain number, which could be any but we mention it just for reproducibility purposes. You could change it or even omit it without any significant effect. We also want to use cuda and also the gpu available to us so we use swt that as well.<\/p>\n Here we have downloaded the imdb dataset for python sentiment analysis and divided it into train test and validation split. The dataset is already divided into a train and test set, we further create a validation set from it.<\/p>\n We further limit the number of words the model will learn to 25000, this will choose the most used 25000 words from the dataset and use them for training. Significantly reducing the work of the model without any real loss in accuracy.<\/p>\n We are now creating a training, testing and validation batch from the data that we have for preparing it to be fed to the model in the form of batches of 64 samples at a time. Reduce this if you get out of memory error.<\/p>\n We now prepare the model and define its architecture. For our task, we are using a lstm rnn. We will be using a multi layer bidirectional rnn. That means there will be multiple rnn layers stacked on top of each other.<\/p>\n Bidirectional RNNs have an advantage of having more context than a single directional network. For example, in a sentence flowing forward through a model, if a model has to guess the next word, it would do so on the basis of previous knowledge. But in a bidirectional network it will also have the knowledge of what’s next due to two networks flowing in opposite directions stacked on top of each other. The input flows in the opposite direction as well as sequence. A sentence \u201ci love DataFlair\u201d will flow in the first network as \u201ci\u201d , \u201clove\u201d , \u201cDataFlair\u201d but in the second it will flow like \u201cDataFlair\u201d , \u201clove\u201d , \u201ci\u201d. This provides better context and relation in the data to the network.<\/p>\n The hidden state output from the first layer will be the input to the next layer, also each layer has its independent hidden nodes.<\/p>\n We use nn.LSTM layers instead of normal nn.RNN layers<\/p>\n We pass everything through an embedding layer which is just a representation of words in lower-dimensional space. Simply put to place the words such that similar words are grouped together.<\/p>\n We define the parameters for python sentiment analysis model and pass it to an instance of the model class we just defined. The number of input parameters, hidden layer, and the output dimension along with throughput rate and bidirectionality boolean is defined. We also pass the pad token index from the vocabulary that we created earlier.<\/p>\n Now we print some details about our model. Getting the number of trainable parameters that are present there in the model.<\/p>\n We then get the pre-trained embedding weights and copy them to our model so that it does not need to learn the embeddings, and can directly focus on the job at hand that is learning the sentiments related to those embeddings.<\/p>\n Pretrained embedding weights are placed in place of the initial ones.<\/p>\n![\"sentiment\"](\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2021\/07\/sentiment.jpg\")
<\/a><\/p>\nHow does sentiment analysis work?<\/h2>\n
\n
\n
\nThis type of rule based sentiment analysis is easy to implement, but lacks flexibility and does not account for context.<\/li>\nUnderstanding the sentiment analysis system<\/h3>\n
<\/a><\/p>\n<\/a><\/p>\nBuilding Sentiment Analysis System<\/h2>\n
What is Sentiment Analysis<\/h3>\n
Download Sentiment Analysis Python Code<\/h3>\n
Sentiment Analysis Dataset<\/h3>\n
\n
import os\r\nfrom google.colab import drive\r\ndrive.mount('\/content\/drive')\r\n \r\nos.chdir('\/content\/drive\/My Drive\/DataFlair\/Sentiment')\r\n!ls<\/pre>\nPreparation of data<\/strong><\/h3>\n
import random\r\nimport torch\r\nfrom torchtext.legacy import data\r\nfrom torchtext.legacy import datasets\r\n \r\nseed = 42\r\n \r\ntorch.manual_seed(seed)\r\ntorch.backends.cudnn.deterministic = True\r\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\r\n \r\ntxt = data.Field(tokenize = 'spacy',\r\n tokenizer_language = 'en_core_web_sm',\r\n include_lengths = True)\r\n \r\nlabels = data.LabelField(dtype = torch.float)\r\n<\/pre>\ntrain_data, test_data = datasets.IMDB.splits(txt, labels)\r\ntrain_data, valid_data = train_data.split(random_state = random.seed(seed))\r\nnum_words = 25_000\r\n \r\ntxt.build_vocab(train_data, \r\n max_size = num_words, \r\n vectors = \"glove.6B.100d\", \r\n unk_init = torch.Tensor.normal_)\r\n \r\nlabels.build_vocab(train_data)\r\n<\/pre>\n
btch_size = 64\r\n \r\ntrain_itr, valid_itr, test_itr = data.BucketIterator.splits(\r\n (train_data, valid_data, test_data), \r\n batch_size = btch_size,\r\n sort_within_batch = True,\r\n device = device)\r\n<\/pre>\n
Defining python sentiment analysis model<\/h3>\n
import torch.nn as nn\r\n \r\nclass RNN(nn.Module):\r\n def __init__(self, word_limit, dimension_embedding, dimension_hidden, dimension_output, num_layers, \r\n bidirectional, dropout, pad_idx):\r\n \r\n super().__init__()\r\n \r\n self.embedding = nn.Embedding(word_limit, dimension_embedding, padding_idx = pad_idx)\r\n \r\n self.rnn = nn.LSTM(dimension_embedding, \r\n dimension_hidden, \r\n num_layers=num_layers, \r\n bidirectional=bidirectional, \r\n dropout=dropout)\r\n \r\n self.fc = nn.Linear(dimension_hidden * 2, dimension_output)\r\n \r\n self.dropout = nn.Dropout(dropout)\r\n \r\n def forward(self, text, len_txt):\r\n \r\n \r\n embedded = self.dropout(self.embedding(text))\r\n \r\n \r\n packed_embedded = nn.utils.rnn.pack_padded_sequence(embedded, len_txt.to('cpu'))\r\n \r\n packed_output, (hidden, cell) = self.rnn(packed_embedded)\r\n \r\n output, output_lengths = nn.utils.rnn.pad_packed_sequence(packed_output)\r\n \r\n \r\n hidden = self.dropout(torch.cat((hidden[-2,:,:], hidden[-1,:,:]), dim = 1))\r\n \r\n return self.fc(hidden)\r\n<\/pre>\ndimension_input = len(txt.vocab)\r\ndimension_embedding = 100\r\ndimension_hddn = 256\r\ndimension_out = 1\r\nlayers = 2\r\nbidirectional = True\r\ndropout = 0.5\r\nidx_pad = txt.vocab.stoi[txt.pad_token]\r\n \r\nmodel = RNN(dimension_input, \r\n dimension_embedding, \r\n dimension_hddn, \r\n dimension_out, \r\n layers, \r\n bidirectional, \r\n dropout, \r\n idx_pad)\r\n<\/pre>\n
def count_parameters(model):\r\n return sum(p.numel() for p in model.parameters() if p.requires_grad)\r\n \r\nprint(f'The model has {count_parameters(model):,} trainable parameters')\r\npretrained_embeddings = txt.vocab.vectors\r\n \r\nprint(pretrained_embeddings.shape)\r\nunique_id = txt.vocab.stoi[txt.unk_token]\r\n \r\nmodel.embedding.weight.data[unique_id] = torch.zeros(dimension_embedding)\r\nmodel.embedding.weight.data[idx_pad] = torch.zeros(dimension_embedding)\r\n \r\nprint(model.embedding.weight.data)\r\n<\/pre>\n