{"id":16969,"date":"2018-06-08T04:00:22","date_gmt":"2018-06-08T04:00:22","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=16969"},"modified":"2021-05-14T11:00:10","modified_gmt":"2021-05-14T05:30:10","slug":"embedding-in-tensorflow","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/embedding-in-tensorflow\/","title":{"rendered":"Embedding in TensorFlow | TensorBoard Embedding Projector"},"content":{"rendered":"

In this TensorFlow<\/strong> Embedding tutorial, we will be learning about the Embedding in TensorFlow & also TensorFlow Embedding example. Moreover, we will look at TensorFlow Embedding Visualization example. <\/span><\/p>\n

Along with this, we will discuss TensorFlow Embedding Projector and metadata for Embedding in TensorFlow. At last. we will see how to create Embeddings in TensorFlow.<\/span><\/p>\n

So, let’s start embeddings in TensorFlow.<\/p>\n

What is Embedding in TensorFlow?<\/span><\/h2>\n

An E<\/span>mbedding in TensorFlow\u00a0defines as the mapping like the word to vector (word2vec<\/strong>) of real numbers. A TensorFlow embedding example below where a list of colors represents as vectors:<\/span><\/p>\n

Black: \u00a0(0.01359, 0.00075997, 0.24608, …, -0.2524, 1.0048, 0.06259)<\/span>
\n<\/span>Blues: \u00a0(0.01396, 0.11887, -0.48963, …, 0.033483, -0.10007, 0.1158)<\/span>
\n<\/span>Yellow: \u00a0(-0.24776, -0.12359, 0.20986, …, 0.079717, 0.23865, -0.014213)<\/span>
\n<\/span>Oranges: \u00a0(-0.35609, 0.21854, 0.080944, …, -0.35413, 0.38511, -0.070976)<\/span><\/p>\n

The dimensions in these kinds of vectors usually don\u2019t have any meaning, but the pattern of the matrix, as well as the location and distance between the vectors, contains some significant information that can be taken an advantage of.<\/span><\/p>\n

As you are already aware, the classifiers and neural networks use these vectors of real numbers in their regular computation dense vectors are used to train them the best way. But many times the data involved such as words of text, don\u2019t have a vector to be represented by. <\/span><\/p>\n

Therefore, you could make use of standard embedding functions that prove to be an effective way to transform such inputs into useful vectors.<\/span><\/p>\n

Another example for embeddings in TensorFlow can be assumed to be the representation of Euclidean distance or the angle between vectors which are commonly used to find the nearest neighbors as shown below, the word and the respective angles:<\/span><\/p>\n

Black:<\/strong> \u00a0(red, 47.6\u00b0), (yellow, 51.9\u00b0), (purple, 52.4\u00b0)<\/span>
\n<\/span>Blues:<\/strong> \u00a0(jazz, 53.3\u00b0), (folk, 59.1\u00b0), (bluegrass, 60.6\u00b0)<\/span>
\n<\/span>Yellow:<\/strong> \u00a0(yellow, 53.5\u00b0), (colored, 58.0\u00b0), (bright, 59.9\u00b0)<\/span>
\n<\/span>Oranges:<\/strong> \u00a0(apples, 45.3\u00b0), (lemons, 48.3\u00b0), (mangoes, 50.4\u00b0)<\/span><\/p>\n

Creating Embedding in TensorFlow\u00a0<\/span><\/h2>\n

To create word embedding in TensorFlow, you start off by splitting the input text into words and then assigning an integer to every word. After that has been done, the <\/span>word_id<\/span><\/i> become a vector of these integers. <\/span><\/p>\n

Let us take an example for embedding in TensorFlow, \u201cI love the dog.\u201d This could split into [\u201cI\u201d, \u201clove\u201d, \u201cthe\u201d, \u201cdog\u201d, \u201c.\u201d]. The <\/span>word_ids<\/span><\/i> vector will now be of size [5] and will have 5 integers. <\/span><\/p>\n

Now, for mapping the word_ids into vectors, you will create an embedding variable and should use the <\/span>tf.nn.embedding_lookup<\/i> function:<\/strong><\/p>\n

word_embeddings = tf.get_variable(\u201cword_embeddings\u201d,\r\n    [vocabulary_size, embedding_size])\r\nembedded_word_ids = tf.nn.embedding_lookup(word_embeddings, word_ids)<\/pre>\n
After which the word_ids will be of size [5, embedding_size] and will contain the representation for every word. <\/span><\/p>\n
Visualizing TensorFlow Embeddings<\/span><\/h2>\n
For visualization of embeddings in TensorFlow, TensorBoard offers an <\/span>embedding projector<\/span><\/i>, a tool which lets you interactively visualize embeddings. The TensorFlow embedding projector consists of three panels:<\/span><\/p>\n