{"id":8789,"date":"2018-02-20T13:35:34","date_gmt":"2018-02-20T13:35:34","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=8789"},"modified":"2021-05-09T13:08:31","modified_gmt":"2021-05-09T07:38:31","slug":"flume-channel-selectors","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/","title":{"rendered":"Flume Channel Selectors – Apache Flume"},"content":{"rendered":"

Flume Channel Selectors is the component of the Flume agent responsible for choosing the channel which transfers an event in case of multiple channels.<\/p>\n

In this article, we will explore the Flume Channel Selector in detail. It explains the different types of Channel selector. It also describe the properties associated with different channel selectors along with examples.<\/p>\n

Flume Channel Selectors<\/h2>\n

Flume Channel Selector is the component of the Flume agent which determines which channel should be selected for transferring an event when a group of channels exists. They work in between source and channel. The mechanism used is the internal mechanism.<\/p>\n

The Channel Selector are categorized into three types:
\n1.<\/strong> Replicating channel selectors
\n2.<\/strong> Multiplexing channel selector
\n3.<\/strong> Custom Channel Selector<\/p>\n

For any channel, the selector is specified using the selector.type<\/strong> property.
\nIf the type is not specified, then the Replicating channel selector is used by default.<\/strong><\/p>\n

Let us now explore each type in detail.<\/p>\n

1. Replicating Channel Selector in Flume<\/h3>\n

It is the default Channel Selector. If nothing is configured respective to the channel selector, then the Replicating channel selector decides the channel through which the event gets transferred.<\/p>\n

Replicating channel selector works by replicating events into each channel by assuming that there is more than one channel.<\/p>\n

Properties for Replicating Channel Selector:<\/strong><\/p>\n\n\n\n\n\n
Property Name<\/strong><\/td>\nDefault value<\/strong><\/td>\nDescription<\/strong><\/td>\n<\/tr>\n
selector.type<\/p>\n

(Required)<\/span><\/td>\n

replicating<\/span><\/td>\nThe component type name must be <\/span>replicating.<\/span><\/td>\n<\/tr>\n
\u00a0 \u00a0 selector.optional<\/p>\n

\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0(Optional)<\/span><\/td>\n

 –<\/span><\/td>\nIt specifies the channels marked as optional.<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Replicating channel selectors:<\/strong><\/p>\n

<Agent_name>.sources = <Source-name>\r\n<Agent_name>.channels = <Channel1> <Channel2>\u2026\u2026<Channeln>\r\n<Agent_name>.sources.<source-name>.selector.type = replicating\r\n<Agent_name>.sources.<source-name>.channels = <Channel1> <Channel2>\u2026\u2026<Channeln>\r\n<Agent_name>.sources.<source-name>.selector.optional = <Optional channel-number>\r\n<\/pre>\n
Example for agent named agent1 and it\u2019s source called src , and channel be ch1, ch2, ch3:<\/strong><\/p>\n
agent1.sources = src\r\nagent1.channels = ch1 ch2 ch3\r\nagt1.sources.src.selector.type = replicating\r\nagent1.sources.src.channels = ch1 ch2 ch3\r\nagent1.sources.src.selector.optional = ch2\r\n<\/pre>\n
In the above configuration, ch2 is an optional channel. Failure for writing to ch2 is simply ignored. Failure to write to channel ch1, and ch3 would cause the transaction to fail as they are not marked as optional.<\/p>\n
2. Multiplexing Channel Selector in Flume<\/h3>\n
Multiplexing Channel Selector writes an event to different channels on the basis of the information in the header.<\/p>\n
Properties for Multiplexing Channel Selector:<\/strong><\/p>\n\n\n\n\n\n\n\n
Property Name<\/strong><\/td>\nDefault value<\/strong><\/td>\nDescription<\/strong><\/td>\n<\/tr>\n
\u00a0 \u00a0 <\/span>selector.type<\/p>\n
\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0(Required)<\/span><\/td>\n
replicating<\/span><\/td>\nThe component type name must be <\/span>multiplexing.<\/span><\/td>\n<\/tr>\n
\u00a0 selector.header<\/p>\n
\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0(Required)<\/span><\/td>\n
flume.selector.header<\/span><\/td>\n<\/td>\n<\/tr>\n
\u00a0 selector.default<\/p>\n
\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0(Optional)<\/span><\/td>\n
–<\/span><\/td>\n<\/td>\n<\/tr>\n
\u00a0 selector.mapping.*<\/p>\n
\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0(Optional)<\/span><\/td>\n
–<\/span><\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
Multiplexing Channel Selectors:<\/strong><\/p>\n
<Agent_name>.sources = <Source-name>\r\n<Agent_name>.channels = <Channel1> <Channel2>\u2026\u2026<Channeln>\r\n<Agent_name>.sources.<source-name>.selector.type = multiplexing\r\n<Agent_name>.sources.<source-name>.selector.header = <header-name>\r\n<Agent_name>.sources.<source-name>.selector.mapping.<header-category> = <Channel1> <Channel2>\u2026\u2026<Channeln>\r\n\u2026\r\n<Agent_name>.sources.<source-name>.selector.mapping.<header-category> = <Channel1> <Channel2>\u2026\u2026<Channeln>\r\n<Agent_name>.sources.<source-name>.selector.default = <Channel1> <Channel2>\u2026\u2026<Channeln>\r\n<\/pre>\n
Example for an agent named agent1 and source called src, and channel be ch1, ch2, ch3:<\/strong><\/p>\n
agent1.sources = src\r\nagent1.channels = ch1 ch2 ch3 ch4\r\nagent1.sources.src.selector.type = multiplexing\r\nagent1.sources.src.selector.header = subject\r\nagent1.sources.src.selector.mapping.subject1 = ch1\r\nagent1.sources.src.selector.mapping.subject2 = ch2\r\nagent1.sources.src.selector.default = ch4\r\n<\/pre>\n
In the above configuration, channels were separated by the header subject. subject1 are routed to ch1, subject2 are routed to ch2.
\nch4 is a default channel.<\/p>\n
3. Custom Channel Selector<\/h3>\n
The custom channel selector is our implementation of the ChannelSelector interface.
\nWhen starting the Flume agent, the custom channel selector\u2019s class and its dependencies must be included in the agent\u2019s classpath.<\/p>\n
The Custom channel selector type is FQCN.<\/p>\n
Property for Custom Channel Selector:<\/strong><\/p>\n\n\n\n\n
Property Name<\/strong><\/td>\nDefault Value<\/strong><\/td>\nDescription<\/strong><\/td>\n<\/tr>\n
\u00a0 \u00a0 \u00a0 Selector.type<\/p>\n
\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0(Required) \u00a0 \u00a0 \u00a0 <\/span>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/b><\/td>\n
<\/td>\nThe component type must be our FQCN<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
Custom Channel Selector:<\/strong><\/p>\n
<Agent_name>.sources = <Source-name>\r\n<Agent_name>.channels = <Channel1>\r\n<Agent_name>.sources.<source-name>.selector.type = custom selector type\r\n\r\n<\/pre>\n
Example for agent named agent1 and its source called src, and channel be ch1:<\/strong><\/p>\n
agent1.sources = src\r\nagent1.channels = ch1\r\nagent1.sources.src.selector.type = org.example.MyChannelSelector\r\n<\/pre>\n
Summary<\/h2>\n
I hope after reading this article you clearly understand the Flume Channel Selector. Channel Selector determines the channel through which the event gets transferred. The work in between source and channel.<\/p>\n
The article explained the three different types of Channel selector that are replicating channel selector, multiplexing channel selector, and custom channel selector. The article also explains how to set the particular type of channel selector by using the selector.type property.<\/p>\n","protected":false},"excerpt":{"rendered":"
Flume Channel Selectors is the component of the Flume agent responsible for choosing the channel which transfers an event in case of multiple channels. In this article, we will explore the Flume Channel Selector...<\/p>\n","protected":false},"author":6,"featured_media":78416,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21],"tags":[2459,4807,4808],"class_list":["post-8789","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-flume","tag-channel-selectors-in-flume","tag-flume-channel-selector","tag-flume-channel-selectors"],"yoast_head":"\nFlume Channel Selectors - Apache Flume - DataFlair<\/title>\n<meta name=\"description\" content=\"Flume Channel Selectors-types of Channel selectors in flume,Custom Channel Selector,Multiplexing Channel Selector,Replicating Channel Selector with examples\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Flume Channel Selectors - Apache Flume - DataFlair\" \/>\n<meta property=\"og:description\" content=\"Flume Channel Selectors-types of Channel selectors in flume,Custom Channel Selector,Multiplexing Channel Selector,Replicating Channel Selector with examples\" \/>\n<meta property=\"og:url\" content=\"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/\" \/>\n<meta property=\"og:site_name\" content=\"DataFlair\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataFlairWS\/\" \/>\n<meta property=\"article:published_time\" content=\"2018-02-20T13:35:34+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-05-09T07:38:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/flume-channel-selector.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"802\" \/>\n\t<meta property=\"og:image:height\" content=\"420\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"DataFlair Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:site\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DataFlair Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Flume Channel Selectors - Apache Flume - DataFlair","description":"Flume Channel Selectors-types of Channel selectors in flume,Custom Channel Selector,Multiplexing Channel Selector,Replicating Channel Selector with examples","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/","og_locale":"en_US","og_type":"article","og_title":"Flume Channel Selectors - Apache Flume - DataFlair","og_description":"Flume Channel Selectors-types of Channel selectors in flume,Custom Channel Selector,Multiplexing Channel Selector,Replicating Channel Selector with examples","og_url":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/","og_site_name":"DataFlair","article_publisher":"https:\/\/www.facebook.com\/DataFlairWS\/","article_published_time":"2018-02-20T13:35:34+00:00","article_modified_time":"2021-05-09T07:38:31+00:00","og_image":[{"width":802,"height":420,"url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/flume-channel-selector.jpg","type":"image\/jpeg"}],"author":"DataFlair Team","twitter_card":"summary_large_image","twitter_creator":"@DataFlairWS","twitter_site":"@DataFlairWS","twitter_misc":{"Written by":"DataFlair Team","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/#article","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/"},"author":{"name":"DataFlair Team","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89"},"headline":"Flume Channel Selectors – Apache Flume","datePublished":"2018-02-20T13:35:34+00:00","dateModified":"2021-05-09T07:38:31+00:00","mainEntityOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/"},"wordCount":558,"commentCount":0,"publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/flume-channel-selector.jpg","keywords":["Channel Selectors in Flume","Flume channel selector","Flume Channel Selectors"],"articleSection":["Flume Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/","url":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/","name":"Flume Channel Selectors - Apache Flume - DataFlair","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/#primaryimage"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/flume-channel-selector.jpg","datePublished":"2018-02-20T13:35:34+00:00","dateModified":"2021-05-09T07:38:31+00:00","description":"Flume Channel Selectors-types of Channel selectors in flume,Custom Channel Selector,Multiplexing Channel Selector,Replicating Channel Selector with examples","breadcrumb":{"@id":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/#primaryimage","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/flume-channel-selector.jpg","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/02\/flume-channel-selector.jpg","width":802,"height":420,"caption":"flume channel selectors"},{"@type":"BreadcrumbList","@id":"https:\/\/data-flair.training\/blogs\/flume-channel-selectors\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog Home","item":"https:\/\/data-flair.training\/blogs\/"},{"@type":"ListItem","position":2,"name":"Flume Tutorials","item":"https:\/\/data-flair.training\/blogs\/category\/flume\/"},{"@type":"ListItem","position":3,"name":"Flume Channel Selectors – Apache Flume"}]},{"@type":"WebSite","@id":"https:\/\/data-flair.training\/blogs\/#website","url":"https:\/\/data-flair.training\/blogs\/","name":"DataFlair","description":"Learn Today. Lead Tomorrow.","publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/data-flair.training\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/data-flair.training\/blogs\/#organization","name":"DataFlair","url":"https:\/\/data-flair.training\/blogs\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","width":106,"height":48,"caption":"DataFlair"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataFlairWS\/","https:\/\/x.com\/DataFlairWS","https:\/\/www.linkedin.com\/company\/dataflair-web-services-pvt-ltd\/","https:\/\/www.youtube.com\/user\/DataFlairWS"]},{"@type":"Person","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89","name":"DataFlair Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","caption":"DataFlair Team"},"description":"The DataFlair Team provides industry-driven content on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our expert educators focus on delivering value-packed, easy-to-follow resources for tech enthusiasts and professionals.","url":"https:\/\/data-flair.training\/blogs\/author\/dfteam2\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/8789","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/comments?post=8789"}],"version-history":[{"count":3,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/8789\/revisions"}],"predecessor-version":[{"id":92682,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/8789\/revisions\/92682"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media\/78416"}],"wp:attachment":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media?parent=8789"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/categories?post=8789"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/tags?post=8789"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}