Required frameworks are<\/p>\n
- \n
- Numpy (version 1.16.5)<\/li>\n
- cv2 (openCV) (version 3.4.2)<\/li>\n
- Keras (version 2.3.1)<\/li>\n
- Tensorflow (Keras uses TensorFlow in backend and for some image preprocessing) (version 2.0.0)<\/li>\n
- Matplotlib (version 3.1.1)<\/li>\n
- Pandas (version 0.25.1)<\/li>\n<\/ol>\n
Download Dataset<\/h3>\n
The dataset for this project contains 372450 images of alphabets of 28×2, all present in the form of a CSV file:
\nHandwritten character recognition dataset<\/strong><\/a><\/p>\nSteps to develop handwritten character recognition<\/h3>\n
Download Project Code<\/h4>\n
Please download project source code: Handwritten Character Recognition with Neural Network<\/strong><\/a><\/p>\n
import matplotlib.pyplot as plt\r\nimport cv2\r\nimport numpy as np\r\nfrom keras.models import Sequential\r\nfrom keras.layers import Dense, Flatten, Conv2D, MaxPool2D, Dropout\r\nfrom keras.optimizers import SGD, Adam\r\nfrom keras.callbacks import ReduceLROnPlateau, EarlyStopping\r\nfrom keras.utils import to_categorical\r\nimport pandas as pd\r\nimport numpy as np\r\nfrom sklearn.model_selection import train_test_split\r\nfrom sklearn.utils import shuffle\r\n<\/pre>\n
- \n
- First of all, we do all the necessary imports as stated above. We will see the use of all the imports as we use them.<\/li>\n<\/ul>\n
Read the data:<\/strong><\/p>\n
data = pd.read_csv(r\"D:\\a-z alphabets\\A_Z Handwritten Data.csv\").astype('float32')\r\n\r\nprint(data.head(10))\r\n<\/pre>\n- \n
- Now we are reading the dataset using the pd.read_csv()<\/strong> and printing the first 10 images using data.head(10)<\/strong><\/li>\n<\/ul>\n
![\"dataframe](\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/dataframe-data-sample.png\")
<\/a><\/p>\n(The above image shows some of the rows of the dataframe data using the head() function of dataframe)<\/p>\n
Split data into images and their labels:<\/strong><\/p>\n
X = data.drop('0',axis = 1)\r\ny = data['0']\r\n<\/pre>\nSplitting the data read into the images & their corresponding labels. The \u20180\u2019 contains the labels, & so we drop the \u20180\u2019 column from the data dataframe read & use it in the y to form the labels.<\/p>\n
Reshaping the data in the csv file so that it can be displayed as an image<\/strong><\/p>\n
train_x, test_x, train_y, test_y = train_test_split(X, y, test_size = 0.2)\r\n\r\ntrain_x = np.reshape(train_x.values, (train_x.shape[0], 28,28))\r\ntest_x = np.reshape(test_x.values, (test_x.shape[0], 28,28))\r\n\r\nprint(\"Train data shape: \", train_x.shape)\r\nprint(\"Test data shape: \", test_x.shape)\r\n<\/pre>\n
- \n
- In the above segment, we are splitting the data into training & testing dataset using train_test_split().<\/li>\n
- Also, we are reshaping the train & test image data so that they can be displayed as an image, as initially in the CSV file they were present as 784 columns of pixel data. So we convert it to 28×28 pixels.<\/li>\n<\/ul>\n
word_dict = {0:'A',1:'B',2:'C',3:'D',4:'E',5:'F',6:'G',7:'H',8:'I',9:'J',10:'K',11:'L',12:'M',13:'N',14:'O',15:'P',16:'Q',17:'R',18:'S',19:'T',20:'U',21:'V',22:'W',23:'X', 24:'Y',25:'Z'}\r\n<\/pre>\n- \n
- All the labels are present in the form of floating point values, that we convert to integer values, & so we create a dictionary word_dict to map the integer values with the characters.<\/li>\n<\/ul>\n
Plotting the number of alphabets in the dataset<\/strong><\/p>\n
y_int = np.int0(y)\r\ncount = np.zeros(26, dtype='int')\r\nfor i in y_int:\r\n count[i] +=1\r\n\r\nalphabets = []\r\nfor i in word_dict.values():\r\n alphabets.append(i)\r\n\r\nfig, ax = plt.subplots(1,1, figsize=(10,10))\r\nax.barh(alphabets, count)\r\n\r\nplt.xlabel(\"Number of elements \")\r\nplt.ylabel(\"Alphabets\")\r\nplt.grid()\r\nplt.show()\r\n<\/pre>\n
- \n
- Here we are only describing the distribution of the alphabets.<\/li>\n
- Firstly we convert the labels into integer values and append into the count list according to the label. This count list has the number of images present in the dataset belonging to each alphabet.<\/li>\n
- Now we create a list – alphabets containing all the characters using the values() function of the dictionary.<\/li>\n
- Now using the count & alphabets lists we draw the horizontal bar plot.<\/li>\n<\/ul>\n
![\"plot](\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/plot-alphabets.png\")
<\/a><\/p>\nShuffling the data<\/strong><\/p>\n
shuff = shuffle(train_x[:100])\r\n\r\nfig, ax = plt.subplots(3,3, figsize = (10,10))\r\naxes = ax.flatten()\r\n\r\nfor i in range(9):\r\n _, shu = cv2.threshold(shuff[i], 30, 200, cv2.THRESH_BINARY)\r\n axes[i].imshow(np.reshape(shuff[i], (28,28)), cmap=\"Greys\")\r\nplt.show()\r\n<\/pre>\n
- \n
- Now we shuffle some of the images of the train set.<\/li>\n
- The shuffling is done using the shuffle() function so that we can display some random images.<\/li>\n
- We then create 9 plots in 3×3 shape & display the thresholded images of 9 alphabets.<\/li>\n<\/ul>\n
![\"character](\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/character-grayscale-images.png\")
<\/a><\/p>\n(The above image depicts the grayscale images that we got from the dataset)<\/p>\n
Data Reshaping<\/h4>\n
Reshaping the training & test dataset so that it can be put in the model<\/strong><\/p>\n
train_X = train_x.reshape(train_x.shape[0],train_x.shape[1],train_x.shape[2],1)\r\nprint(\"New shape of train data: \", train_X.shape)\r\n\r\ntest_X = test_x.reshape(test_x.shape[0], test_x.shape[1], test_x.shape[2],1)\r\nprint(\"New shape of train data: \", test_X.shape)\r\n\r\n\r\nNow we reshape the train & test image dataset so that they can be put in the model.\r\n\r\nNew shape of train data: (297960, 28, 28, 1)\r\nNew shape of train data: (74490, 28, 28, 1)\r\n<\/pre>\n
Now we reshape the train & test image dataset so that they can be put in the model.<\/p>\n
New shape of train data: (297960, 28, 28, 1)<\/p>\n
New shape of train data: (74490, 28, 28, 1)<\/p>\n
train_yOHE = to_categorical(train_y, num_classes = 26, dtype='int')\r\nprint(\"New shape of train labels: \", train_yOHE.shape)\r\n\r\ntest_yOHE = to_categorical(test_y, num_classes = 26, dtype='int')\r\nprint(\"New shape of test labels: \", test_yOHE.shape)\r\n<\/pre>\n
Here we convert the single float values to categorical values. This is done as the CNN model takes input of labels & generates the output as a vector of probabilities.<\/p>\n
Now we define the CNN.<\/p>\n
What is CNN?<\/h3>\n
CNN stands for Convolutional Neural Networks that are used to extract the features of the images using several layers of filters.<\/p>\n
![\"cnn](\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/cnn-convolutional-neural-network.jpg\")
<\/a><\/p>\n(Example of how a CNN looks logically)<\/p>\n
The convolution layers are generally followed by maxpool layers that are used to reduce the number of features extracted and ultimately the output of the maxpool and layers and convolution layers are flattened into a vector of single dimension and are given as an input to the Dense layer (The fully connected network).<\/p>\n
The model created is as follows:<\/p>\n
model = Sequential()\r\n\r\nmodel.add(Conv2D(filters=32, kernel_size=(3, 3), activation='relu', input_shape=(28,28,1)))\r\nmodel.add(MaxPool2D(pool_size=(2, 2), strides=2))\r\n\r\nmodel.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding = 'same'))\r\nmodel.add(MaxPool2D(pool_size=(2, 2), strides=2))\r\n\r\nmodel.add(Conv2D(filters=128, kernel_size=(3, 3), activation='relu', padding = 'valid'))\r\nmodel.add(MaxPool2D(pool_size=(2, 2), strides=2))\r\n\r\nmodel.add(Flatten())\r\n\r\nmodel.add(Dense(64,activation =\"relu\"))\r\nmodel.add(Dense(128,activation =\"relu\"))\r\n\r\nmodel.add(Dense(26,activation =\"softmax\"))\r\n<\/pre>\n
Above we have the CNN model that we designed for training the model over the training dataset.<\/p>\n
Compiling & Fitting Model<\/h4>\n
model.compile(optimizer = Adam(learning_rate=0.001), loss='categorical_crossentropy', metrics=['accuracy'])\r\n\r\nhistory = model.fit(train_X, train_yOHE, epochs=1, validation_data = (test_X,test_yOHE))\r\n<\/pre>\n
- \n
- Here we are compiling the model, where we define the optimizing function & the loss function to be used for fitting.<\/li>\n
- The optimizing function used is Adam, that is a combination of RMSprop & Adagram optimizing algorithms.<\/li>\n
- The dataset is very large so we are training for only a single epoch, however, as required we can even train it for multiple epochs (which is recommended for character recognition for better accuracy).<\/li>\n<\/ul>\n
model.summary()\r\nmodel.save(r'model_hand.h5')\r\n<\/pre>\n
Now we are getting the model summary that tells us what were the different layers defined in the model & also we save the model using model.save()<\/strong> function.<\/p>\n
![\"text](\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/text-recognition-model.png\")
<\/a><\/p>\n(Summary of the defined model)<\/p>\n
Getting the Train & Validation Accuracies & Losses<\/h4>\n
print(\"The validation accuracy is :\", history.history['val_accuracy'])\r\nprint(\"The training accuracy is :\", history.history['accuracy'])\r\nprint(\"The validation loss is :\", history.history['val_loss'])\r\nprint(\"The training loss is :\", history.history['loss'])\r\n<\/pre>\n
In the above code segment, we print out the training & validation accuracies along with the training & validation losses for character recognition.<\/p>\n
- All the labels are present in the form of floating point values, that we convert to integer values, & so we create a dictionary word_dict to map the integer values with the characters.<\/li>\n<\/ul>\n
- Now we are reading the dataset using the pd.read_csv()<\/strong> and printing the first 10 images using data.head(10)<\/strong><\/li>\n<\/ul>\n
- First of all, we do all the necessary imports as stated above. We will see the use of all the imports as we use them.<\/li>\n<\/ul>\n
![\"train](\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/train-validation-accuracies.png\")
<\/a><\/p>\n![\"text](\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/text-prediction-on-test-data.png\")
<\/a><\/p>\n![\"Text](\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/Text-recognition-output.png\")
<\/a><\/p>\n