{"id":13941,"date":"2018-04-18T12:33:09","date_gmt":"2018-04-18T12:33:09","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=13941"},"modified":"2021-05-14T11:00:29","modified_gmt":"2021-05-14T05:30:29","slug":"tensorflow-architecture","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/","title":{"rendered":"TensorFlow Architecture, Important Terms and Functionalities"},"content":{"rendered":"<p>We saw previously why we should use <strong>TensorFlow<\/strong>. Today, we\u2019ll be looking at the Tensorflow Architecture.\u00a0To understand the Tensorflow Architecture, there are some terms that need to be understood first.<\/p>\n<p>These terms are Tensorflow Servables, Servable Streams, Tensorflow Models, Loaders, Sources, Manager, Core. The terms and their functionality in the Architecture of TensorFlow are described below.<\/p>\n<p>So, let&#8217;s discuss each term related to TensorFlow Architecture.<\/p>\n<h2>TensorFlow Servables<\/h2>\n<p>These are the central rudimentary units in TensorFlow Serving. TensorFlow\u00a0Servables are the objects that clients use to perform the computation.<\/p>\n<p>The size of a servable is flexible. A single servable might include anything from a lookup table to a single model to a tuple of inference models. Servables can be of any type and interface, enabling flexibility and future improvements such as:<\/p>\n<ul>\n<li>Streaming results<\/li>\n<li>Experimental APIs<\/li>\n<li>Asynchronous modes of operation.<\/li>\n<\/ul>\n<h2>Servable Versions<\/h2>\n<p>TensorFlow Serving can handle one or more versions of a servable, over the lifetime of a single server instance. This opens the door for fresh algorithm configurations, weights, and other data to be loaded over time.<\/p>\n<p>They also enable more than one version of a servable to be loaded concurrently, supporting gradual roll-out and experimentation. At serving time, clients may request either the latest version or a specific version id, for a particular model.<\/p>\n<h2>Servable Streams<\/h2>\n<p>A sequence of versions of a servable sorted by increasing version numbers.<\/p>\n<h2>TensorFlow Models<\/h2>\n<p>A Serving represents a model as one or more servables. A machine-learned model may include one or more algorithms (including learned weights) and lookup or embedding tables. A servable can also serve as a fraction of a model, for example, a large lookup table can be served as many instances.<\/p>\n<h2>TensorFlow Loaders<\/h2>\n<p>Loaders manage a servable&#8217;s life cycle. The Loader API enables common infrastructure independent from specific learning algorithms, data or product use-cases involved. Specifically, Loaders standardize the APIs for loading and unloading a servable.<\/p>\n<h2>Sources in TensorFlow Architecture<\/h2>\n<p>Sources are in simple terms, modules that find and provide servables. Each Source provides zero or more servable streams. For each servable stream, a Source supplies one Loader instance for each version it makes available to be loaded.<\/p>\n<h2>TensorFlow Managers<\/h2>\n<p>Tensorflow Managers handle the full lifecycle of Servables, including:<\/p>\n<ul>\n<li>Loading Servables<\/li>\n<li>Serving Servables<\/li>\n<li>Unloading Servables<\/li>\n<\/ul>\n<p>Managers listen to sources and track all versions. The Manager tries to fulfil Sources&#8217; requests but, may refuse to load an aspired version. Managers may also postpone an &#8220;unload&#8221;.<\/p>\n<p>For example, a Manager may wait to unload until a newer version finishes loading, based on a policy to guarantee that at least one version is loaded at all times. For example, GetServableHandle(), for clients to access loaded servable instances.<\/p>\n<h2>TensorFlow Core<\/h2>\n<p>This manages (via standard TensorFlow Serving APIs) the following aspects of servables:<\/p>\n<div id=\"attachment_14628\" style=\"width: 1210px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/04\/Libraries-01.jpg\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-14628\" class=\"wp-image-14628 size-full\" src=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/04\/Libraries-01.jpg\" alt=\"Tensorflow Architecture\" width=\"1200\" height=\"628\" srcset=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/04\/Libraries-01.jpg 1200w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/04\/Libraries-01-150x79.jpg 150w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/04\/Libraries-01-300x157.jpg 300w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/04\/Libraries-01-768x402.jpg 768w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/04\/Libraries-01-1024x536.jpg 1024w\" sizes=\"auto, (max-width: 1200px) 100vw, 1200px\" \/><\/a><p id=\"caption-attachment-14628\" class=\"wp-caption-text\">TensorFlow Architecture: TensorFlow Core<\/p><\/div>\n<ul>\n<li>Lifecycle<\/li>\n<li>Metrics<\/li>\n<li>TensorFlow Serving Core treats servables and loaders as opaque objects.<\/li>\n<\/ul>\n<h2>Life of a Servable<\/h2>\n<div id=\"attachment_14810\" style=\"width: 1210px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Technical-Architecture.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-14810\" class=\"wp-image-14810 size-full\" src=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Technical-Architecture.png\" alt=\"Tensorflow Technical Architecture\" width=\"1200\" height=\"628\" srcset=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Technical-Architecture.png 1200w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Technical-Architecture-150x79.png 150w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Technical-Architecture-300x157.png 300w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Technical-Architecture-768x402.png 768w, https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Technical-Architecture-1024x536.png 1024w\" sizes=\"auto, (max-width: 1200px) 100vw, 1200px\" \/><\/a><p id=\"caption-attachment-14810\" class=\"wp-caption-text\">Tensorflow Technical Architecture<\/p><\/div>\n<p>TensorFlow Technical Architecture:<\/p>\n<ul>\n<li>Sources create Loaders for Servable Versions, then Loaders are sent as Aspired Versions to the Manager, which loads and serves them to client requests.<\/li>\n<li>The Loader contains whatever metadata it needs to load the Servable.<\/li>\n<li>The Source uses a callback to notify the manager of the Aspired Version.<\/li>\n<li>The manager applies the configured Version Policy to determine the next action to take.<\/li>\n<li>If the manager determines that it&#8217;s safe, it gives the Loader the required resources and tells the Loader to load the new version.<\/li>\n<li>Clients ask the manager for the Servable, either specifying a version explicitly or just requesting the latest version. The manager returns a handle for the Servable. The Dynamic Manager applies the Version Policy and decides to load the new version.<\/li>\n<li>The Dynamic Manager tells the Loader that there is enough memory. The Loader instantiates the TensorFlow graph with the new weights.<\/li>\n<li>A client requests a handle to the latest version of the model, and the Dynamic Manager returns a handle to the new version of the Servable.<\/li>\n<\/ul>\n<h2>TensorFlow Loaders<\/h2>\n<p>These Loaders are the extension point for adding algorithm and data backends. TensorFlow is one such algorithm backend. For example, you would implement a new Loader in order to load, provide access to, and unload an instance of a new type of servable machine learning model.<\/p>\n<h2>Batcher in TensorFlow Architecture<\/h2>\n<p>Batching of multiple requests into a single request can significantly reduce the cost of performing inference, especially in the presence of hardware accelerators such as GPUs.<\/p>\n<p>TensorFlow Serving includes a request batching widget that let clients easily batch their type-specific inferences across requests into batch requests that algorithm systems can more efficiently process.<\/p>\n<p>So, this was all about Tensorflow Architecture. Hope you like our explanation.<\/p>\n<h2>Conclusion &#8211; TensorFlow Architecture<\/h2>\n<p>Hence, Tensorflow architecture as described above is an overview of how the different components serve their roles in the decentralized system and make the operation of serving a client\u2019s request more optimized as there are now a number of sub-parts that do their operation in parallel and make the architecture pipelined.<\/p>\n<p>In addition, we discuss various terms related to TensorFlow Architecture, such as Tensorflow Servables, Servable Streams, Tensorflow Models, Loaders, Sources, Manager, Core\u00a0 Next, we\u2019ll be looking at the installation guide of TensorFlow.<\/p>\n<p>Furthermore, if you have any query, feel free to ask in the comment section.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We saw previously why we should use TensorFlow. Today, we\u2019ll be looking at the Tensorflow Architecture.\u00a0To understand the Tensorflow Architecture, there are some terms that need to be understood first. These terms are Tensorflow&#46;&#46;&#46;<\/p>\n","protected":false},"author":6,"featured_media":21485,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[73],"tags":[1092,1671,8223,12748,12749,13017,14526,14534,14571,14572,14580,14604,14605,14606],"class_list":["post-13941","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tensorflow","tag-architecture-of-tensorflow","tag-batcher","tag-life-of-a-servable","tag-servable-streams","tag-servable-versions","tag-sources","tag-tensorflow-architecture","tag-tensorflow-core","tag-tensorflow-loader","tag-tensorflow-managers","tag-tensorflow-models","tag-tensorflow-servables","tag-tensorflow-serving","tag-tensorflow-sources"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>TensorFlow Architecture, Important Terms and Functionalities - DataFlair<\/title>\n<meta name=\"description\" content=\"Tensorflow Architecture,Tensorflow Servables,Servable StreamsTensorflow Models,loaders, Sources, Manager, core, Life of a Servable,Batcher,Servable versions\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"TensorFlow Architecture, Important Terms and Functionalities - DataFlair\" \/>\n<meta property=\"og:description\" content=\"Tensorflow Architecture,Tensorflow Servables,Servable StreamsTensorflow Models,loaders, Sources, Manager, core, Life of a Servable,Batcher,Servable versions\" \/>\n<meta property=\"og:url\" content=\"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/\" \/>\n<meta property=\"og:site_name\" content=\"DataFlair\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataFlairWS\/\" \/>\n<meta property=\"article:published_time\" content=\"2018-04-18T12:33:09+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-05-14T05:30:29+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/07\/Tensorflow-Architecture-01-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"DataFlair Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:site\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DataFlair Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"TensorFlow Architecture, Important Terms and Functionalities - DataFlair","description":"Tensorflow Architecture,Tensorflow Servables,Servable StreamsTensorflow Models,loaders, Sources, Manager, core, Life of a Servable,Batcher,Servable versions","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/","og_locale":"en_US","og_type":"article","og_title":"TensorFlow Architecture, Important Terms and Functionalities - DataFlair","og_description":"Tensorflow Architecture,Tensorflow Servables,Servable StreamsTensorflow Models,loaders, Sources, Manager, core, Life of a Servable,Batcher,Servable versions","og_url":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/","og_site_name":"DataFlair","article_publisher":"https:\/\/www.facebook.com\/DataFlairWS\/","article_published_time":"2018-04-18T12:33:09+00:00","article_modified_time":"2021-05-14T05:30:29+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/07\/Tensorflow-Architecture-01-1.jpg","type":"image\/jpeg"}],"author":"DataFlair Team","twitter_card":"summary_large_image","twitter_creator":"@DataFlairWS","twitter_site":"@DataFlairWS","twitter_misc":{"Written by":"DataFlair Team","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/#article","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/"},"author":{"name":"DataFlair Team","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89"},"headline":"TensorFlow Architecture, Important Terms and Functionalities","datePublished":"2018-04-18T12:33:09+00:00","dateModified":"2021-05-14T05:30:29+00:00","mainEntityOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/"},"wordCount":900,"commentCount":0,"publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/07\/Tensorflow-Architecture-01-1.jpg","keywords":["Architecture of TensorFlow","Batcher","life of a servable","Servable Streams","Servable Versions","sources","Tensorflow Architecture","tensorflow core","Tensorflow loader","TensorFlow Managers","tensorflow models","tensorflow servables","tensorflow serving","Tensorflow sources"],"articleSection":["Tensorflow Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/","url":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/","name":"TensorFlow Architecture, Important Terms and Functionalities - DataFlair","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/#primaryimage"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/07\/Tensorflow-Architecture-01-1.jpg","datePublished":"2018-04-18T12:33:09+00:00","dateModified":"2021-05-14T05:30:29+00:00","description":"Tensorflow Architecture,Tensorflow Servables,Servable StreamsTensorflow Models,loaders, Sources, Manager, core, Life of a Servable,Batcher,Servable versions","breadcrumb":{"@id":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/#primaryimage","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/07\/Tensorflow-Architecture-01-1.jpg","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/07\/Tensorflow-Architecture-01-1.jpg","width":1200,"height":628,"caption":"TensorFlow Architecture, Important Terms and Functionalities"},{"@type":"BreadcrumbList","@id":"https:\/\/data-flair.training\/blogs\/tensorflow-architecture\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog Home","item":"https:\/\/data-flair.training\/blogs\/"},{"@type":"ListItem","position":2,"name":"Tensorflow Tutorials","item":"https:\/\/data-flair.training\/blogs\/category\/tensorflow\/"},{"@type":"ListItem","position":3,"name":"TensorFlow Architecture, Important Terms and Functionalities"}]},{"@type":"WebSite","@id":"https:\/\/data-flair.training\/blogs\/#website","url":"https:\/\/data-flair.training\/blogs\/","name":"DataFlair","description":"Learn Today. Lead Tomorrow.","publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/data-flair.training\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/data-flair.training\/blogs\/#organization","name":"DataFlair","url":"https:\/\/data-flair.training\/blogs\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","width":106,"height":48,"caption":"DataFlair"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataFlairWS\/","https:\/\/x.com\/DataFlairWS","https:\/\/www.linkedin.com\/company\/dataflair-web-services-pvt-ltd\/","https:\/\/www.youtube.com\/user\/DataFlairWS"]},{"@type":"Person","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89","name":"DataFlair Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","caption":"DataFlair Team"},"description":"The DataFlair Team provides industry-driven content on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our expert educators focus on delivering value-packed, easy-to-follow resources for tech enthusiasts and professionals.","url":"https:\/\/data-flair.training\/blogs\/author\/dfteam2\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/13941","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/comments?post=13941"}],"version-history":[{"count":6,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/13941\/revisions"}],"predecessor-version":[{"id":95013,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/13941\/revisions\/95013"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media\/21485"}],"wp:attachment":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media?parent=13941"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/categories?post=13941"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/tags?post=13941"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}