{"id":15793,"date":"2018-06-03T06:10:29","date_gmt":"2018-06-03T06:10:29","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=15793"},"modified":"2018-06-03T06:10:29","modified_gmt":"2018-06-03T06:10:29","slug":"kafka-terminologies","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/","title":{"rendered":"Top Apache Kafka Terminologies and Concepts"},"content":{"rendered":"

Basically, Kafka architecture<\/strong>\u00a0contains few key terms, like topics, producers, consumers,<\/span> brokers and many more. To understand Apache Kafka<\/strong><\/em> in detail, we must understand these key terms first. <\/span><\/p>\n

So, in this article, “Kafka Terminologies” we will learn all these Kafka Terminologies which will help us to build the strong foundation of Kafka Knowledge. <\/span><\/p>\n

So, let’s begin with Apache Kafka Terminology.<\/p>\n

\"Kafka<\/a>

Apache Kafka Terminologies and Concepts<\/p><\/div>\n

List of Kafka Terminologies<\/span><\/h2>\n

Below is the list of most prominent Kafka terminologies which\u00a0may help you to build the strong foundation of Kafka knowledge. Get yourslef aware with these terminologies and then ready to implement the concept through Kafka Online Training with Industry Experts<\/strong><\/em><\/p>\n

i. Kafka Broker<\/h4>\n

There\u00a0are one or more servers available in Apache Kafka cluster, basically, these servers (each) are what we call a broker<\/strong>.<\/span><\/p>\n

ii. Kafka Topics<\/h4>\n

Basically, Kafka maintains feeds of messages in categories. And, messages are stored as well as published in a category\/feed name that is what we call a topic. In addition, all Kafka messages are generally organized into Kafka topics.<\/span><\/p>\n

iii. Kafka Partitions<\/h4>\n

In each broker in Kafka, there is some number of partitions. These Kafka partitions in Kafka can be both a leader or a replica of a topic. So, on defining a Leader, it is responsible for all writes and reads to a topic whereas if somehow the leader fails, replica takes over as the new leader.<\/p>\n

iv. Kafka Producers<\/h4>\n

In simple words,\u00a0 the processes\u00a0which publish messages to Kafka is what we call Producers<\/strong>. In addition,\u00a0it publishes data on the topics of their choice.<\/span><\/p>\n

v. Kafka Consumers<\/h4>\n

The processes that subscribe to topics and process as well as read the feed of published messages, is what we call Consumers<\/strong>.<\/p>\n

vi. Offset in Kafka<\/h4>\n

The position of the consumer in the log and which is retained on a per-consumer basis is what we call Offset. Moreover, we can say it is the only metadata retained on a per-consumer basis.<\/span><\/p>\n

vii. Kafka Consumer Group<\/h4>\n

Basically, a consumer abstraction\u00a0offered by Kafka\u00a0which generalizes both traditional messaging models of queuing and also publish-subscribe is what we call the consumer group. However,\u00a0with a consumer group name, Consumers can label themselves.<\/span><\/p>\n

viii. Kafka Log Anatomy<\/h4>\n

A\u00a0log is nothing different but another way to view a partition. Basically, a data source writes messages to the log. Further, one or more consumers read that data from the log at\u00a0any time they want.\u00a0<\/span><\/p>\n

Let’s understand it with a diagram, here consumers A and B are reading a data source which is writing to the log and from the log at different offsets.<\/span><\/p>\n

\"Kafka<\/a>

Log Anatomy in Kafka<\/p><\/div>\n

ix.\u00a0Kafka Message Ordering and Client Acknowledgments<\/h4>\n

In Kafka, the order of the messages delivered from a certain partition and messages received by the partition is same.\u00a0<\/span><\/p>\n

x. Node in Kafka<\/h4>\n

In the Apache Kafka cluster, a node is a single computer.<\/span><\/p>\n

xi. Kafka Cluster<\/h4>\n

A\u00a0 group of computers which are acting together in order to achieve a common purpose is what we call a cluster<\/strong>. In Kafka also, it has the same meaning i.e. a group of computers, each having one instance of Kafka broker.<\/span><\/p>\n

xii.<\/strong>\u00a0<\/span>Kafka Replicas<\/h4>\n

Here, the word replica refers to a backup. That means a replica of a partition is a “backup” of a partition. Basically, we use replicas in order to prevent data loss, they never read or write data.<\/span><\/p>\n

xiii. Kafka Message<\/h4>\n

In one line, Message in Kafka is an information\u00a0which travels from the producer to a consumer through Apache Kafka.<\/span><\/p>\n

xiv. Kafka Leader<\/h4>\n

A\u00a0node which is responsible for all reads and writes for the given partition is what we call a Kafka Leader. So, every partition\u00a0consists of one server, which acts as a leader.<\/span><\/p>\n

xv. Follower in Kafka<\/h4>\n

Simply putting, a node that follows leader instructions is what we call a follower. The basic usage of a follower is, if any leader fails, any of these followers will automatically become the new leader. However, it plays as the normal consumer, which pulls messages and also updates its own data store.<\/span><\/p>\n

xvi. Kafka Data Log<\/h4>\n

Messages are\u00a0preserved through Kafka, especially for a considerable amount of time. That means consumers can read as per their convenience.\u00a0<\/span><\/p>\n

Since Kafka is configured to keep messages for 24 hours\u00a0but somehow consumer is down for time greater than 24 hours, in that case, the consumer will lose messages. Still, it is possible to read that message from last known offset, only if the downtime on part of the consumer is just 60 minutes. <\/span><\/p>\n

xvii.\u00a0<\/strong>Kafka Connector API<\/h4>\n

The API which permits to build as well as run reusable consumers or producers that connects\u00a0existing applications or data systems to Kafka topics, we use the Connector API.\u00a0<\/span><\/p>\n

So, this was all about Apache Kafka Terminologies. Hope you like our explanation.<\/p>\n

Summary<\/span><\/h2>\n

Hence, this Apache Kafka tutorial sums up most of the important Apache Kafka Terminologies. Moreover, this article says how these terms play an important role in Apache Kafka computations. <\/span><\/p>\n

Also, helps us to understand Kafka in more depth. Thus, as a result, this blog will help us to learn the concept of Apache Kafka more efficiently.<\/span><\/p>\n

Furthermore, if you have any query in Kafka Terminologies, feel free to ask through the comment section.<\/span><\/p>\n

After exploring all these Kafka Terminologies, don’t you think you should master this technology?<\/p>\n","protected":false},"excerpt":{"rendered":"

Basically, Kafka architecture\u00a0contains few key terms, like topics, producers, consumers, brokers and many more. To understand Apache Kafka in detail, we must understand these key terms first. So, in this article, “Kafka Terminologies” we...<\/p>\n","protected":false},"author":5,"featured_media":16452,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[849,852,7858,7861,7869,7875,7876,7879,7892,7896,7898,7914,7915,7922,7929,7956,7967,8332,9101,9206,9432,15799],"class_list":["post-15793","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-kafka","tag-apache-kafka-terminologies","tag-apache-kafka-tutorial","tag-kafka-cluster","tag-kafka-concepts","tag-kafka-connector-api","tag-kafka-consumer-group","tag-kafka-consumers","tag-kafka-data-log","tag-kafka-leader","tag-kafka-log-anatomy","tag-kafka-message","tag-kafka-partition-key","tag-kafka-partitions","tag-kafka-producers","tag-kafka-replicas","tag-kafka-terminologies","tag-kafka-topics","tag-list-of-kafka-terminologies","tag-node-in-kafka","tag-offset-in-kafka","tag-partitions-in-kafka","tag-what-is-kafka-broker"],"yoast_head":"\nTop Apache Kafka Terminologies and Concepts - DataFlair<\/title>\n<meta name=\"description\" content=\"Apache Kafka Terminologies- Kafka Broker, Partitions in Kafka, Kafka Consumer,log anatomy,Kafka follower,kafka offset,kafka cluster,Kafka replica,Kafka API\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Top Apache Kafka Terminologies and Concepts - DataFlair\" \/>\n<meta property=\"og:description\" content=\"Apache Kafka Terminologies- Kafka Broker, Partitions in Kafka, Kafka Consumer,log anatomy,Kafka follower,kafka offset,kafka cluster,Kafka replica,Kafka API\" \/>\n<meta property=\"og:url\" content=\"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/\" \/>\n<meta property=\"og:site_name\" content=\"DataFlair\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataFlairWS\/\" \/>\n<meta property=\"article:published_time\" content=\"2018-06-03T06:10:29+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Apache-Kafka-Terminologies-01.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"DataFlair Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:site\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DataFlair Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Top Apache Kafka Terminologies and Concepts - DataFlair","description":"Apache Kafka Terminologies- Kafka Broker, Partitions in Kafka, Kafka Consumer,log anatomy,Kafka follower,kafka offset,kafka cluster,Kafka replica,Kafka API","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/","og_locale":"en_US","og_type":"article","og_title":"Top Apache Kafka Terminologies and Concepts - DataFlair","og_description":"Apache Kafka Terminologies- Kafka Broker, Partitions in Kafka, Kafka Consumer,log anatomy,Kafka follower,kafka offset,kafka cluster,Kafka replica,Kafka API","og_url":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/","og_site_name":"DataFlair","article_publisher":"https:\/\/www.facebook.com\/DataFlairWS\/","article_published_time":"2018-06-03T06:10:29+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Apache-Kafka-Terminologies-01.jpg","type":"image\/jpeg"}],"author":"DataFlair Team","twitter_card":"summary_large_image","twitter_creator":"@DataFlairWS","twitter_site":"@DataFlairWS","twitter_misc":{"Written by":"DataFlair Team","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/#article","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/"},"author":{"name":"DataFlair Team","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/7f83c342f5d1632d6f7b4b0b0f447823"},"headline":"Top Apache Kafka Terminologies and Concepts","datePublished":"2018-06-03T06:10:29+00:00","mainEntityOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/"},"wordCount":916,"commentCount":2,"publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Apache-Kafka-Terminologies-01.jpg","keywords":["Apache Kafka Terminologies","apache Kafka tutorial","Kafka Cluster","Kafka concepts","Kafka Connector API","Kafka Consumer Group","Kafka Consumers","Kafka Data Log","Kafka Leader","Kafka Log Anatomy","Kafka Message","Kafka Partition Key","Kafka Partitions","Kafka Producers","Kafka Replicas","Kafka terminologies","Kafka Topics","List of Kafka Terminologies","Node in Kafka","Offset in Kafka","Partitions in kafka","what is Kafka Broker"],"articleSection":["Apache Kafka Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/data-flair.training\/blogs\/kafka-terminologies\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/","url":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/","name":"Top Apache Kafka Terminologies and Concepts - DataFlair","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/#primaryimage"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Apache-Kafka-Terminologies-01.jpg","datePublished":"2018-06-03T06:10:29+00:00","description":"Apache Kafka Terminologies- Kafka Broker, Partitions in Kafka, Kafka Consumer,log anatomy,Kafka follower,kafka offset,kafka cluster,Kafka replica,Kafka API","breadcrumb":{"@id":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/data-flair.training\/blogs\/kafka-terminologies\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/#primaryimage","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Apache-Kafka-Terminologies-01.jpg","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/05\/Apache-Kafka-Terminologies-01.jpg","width":1200,"height":628,"caption":"k"},{"@type":"BreadcrumbList","@id":"https:\/\/data-flair.training\/blogs\/kafka-terminologies\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog Home","item":"https:\/\/data-flair.training\/blogs\/"},{"@type":"ListItem","position":2,"name":"Apache Kafka Tutorials","item":"https:\/\/data-flair.training\/blogs\/category\/kafka\/"},{"@type":"ListItem","position":3,"name":"Top Apache Kafka Terminologies and Concepts"}]},{"@type":"WebSite","@id":"https:\/\/data-flair.training\/blogs\/#website","url":"https:\/\/data-flair.training\/blogs\/","name":"DataFlair","description":"Learn Today. Lead Tomorrow.","publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/data-flair.training\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/data-flair.training\/blogs\/#organization","name":"DataFlair","url":"https:\/\/data-flair.training\/blogs\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","width":106,"height":48,"caption":"DataFlair"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataFlairWS\/","https:\/\/x.com\/DataFlairWS","https:\/\/www.linkedin.com\/company\/dataflair-web-services-pvt-ltd\/","https:\/\/www.youtube.com\/user\/DataFlairWS"]},{"@type":"Person","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/7f83c342f5d1632d6f7b4b0b0f447823","name":"DataFlair Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/4cf3a74600d131330b8c481d519afd1574093ed89f6d3396a95393ad223eb7cd?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/4cf3a74600d131330b8c481d519afd1574093ed89f6d3396a95393ad223eb7cd?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/4cf3a74600d131330b8c481d519afd1574093ed89f6d3396a95393ad223eb7cd?s=96&d=mm&r=g","caption":"DataFlair Team"},"description":"DataFlair Team creates expert-level guides on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our goal is to empower learners with easy-to-understand content. Explore our resources for career growth and practical learning.","url":"https:\/\/data-flair.training\/blogs\/author\/dfteam1\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/15793","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/comments?post=15793"}],"version-history":[{"count":0,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/15793\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media\/16452"}],"wp:attachment":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media?parent=15793"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/categories?post=15793"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/tags?post=15793"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}