Apache Flume

aid: apache-flume
name: Apache Flume
description: Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving
  large amounts of log and event data. It provides a simple and flexible architecture based on streaming data flows with pluggable
  sources, channels, and sinks, plus a REST monitoring API for agent metrics.
type: Index
position: Consumer
access: 3rd-Party
image: https://kinlane-productions2.s3.amazonaws.com/apis-json/apis-json-logo.jpg
tags:
- Apache
- Data Collection
- ETL
- Log Aggregation
- Open Source
- Streaming
created: '2026-03-16'
modified: '2026-05-19'
url: https://raw.githubusercontent.com/api-evangelist/apache-flume/refs/heads/main/apis.yml
specificationVersion: '0.19'
apis:
- aid: apache-flume:apache-flume-monitoring-api
  name: Apache Flume Monitoring API
  description: REST API for monitoring Apache Flume agents, retrieving component metrics for sources, channels, and sinks,
    and accessing agent health information.
  humanURL: https://flume.apache.org/FlumeUserGuide.html
  baseURL: http://localhost:41414
  tags:
  - Monitoring
  - Metrics
  - REST API
  properties:
  - type: Documentation
    url: https://flume.apache.org/FlumeUserGuide.html
  - type: OpenAPI
    url: openapi/apache-flume-monitoring-openapi.yml
  - type: JSONSchema
    url: json-schema/flume-monitoring-agent-metrics-schema.json
  - type: JSONLD
    url: json-ld/apache-flume-monitoring-context.jsonld
  - type: NaftikoCapability
    url: capabilities/monitoring-monitoring.yaml
- aid: apache-flume:apache-flume-java-api
  name: Apache Flume Java API
  description: Java API for building custom Flume sources, channels, sinks, and interceptors. Provides interfaces for developing
    pluggable data ingestion components.
  humanURL: https://flume.apache.org/FlumeDeveloperGuide.html
  tags:
  - Java
  - SDK
  - Extension
  properties:
  - type: Documentation
    url: https://flume.apache.org/FlumeDeveloperGuide.html
  - type: SDK
    url: https://search.maven.org/artifact/org.apache.flume/flume-ng-core
    title: Java SDK (Maven Central)
common:
- type: Documentation
  url: https://flume.apache.org/documentation.html
- type: GettingStarted
  url: https://flume.apache.org/FlumeUserGuide.html
- type: GitHubOrganization
  url: https://github.com/apache
- type: GitHubRepository
  url: https://github.com/apache/flume
- type: SpectralRules
  url: rules/apache-flume-spectral-rules.yml
- type: Vocabulary
  url: vocabulary/apache-flume-vocabulary.yaml
- type: Features
  data:
  - name: Pluggable Sources
    description: Extensible source architecture supporting Avro, Thrift, Exec, Taildir, Kafka, HTTP, Syslog, and custom sources.
  - name: Durable Channels
    description: Multiple channel implementations including memory, file-backed, and Kafka-backed channels for different durability
      requirements.
  - name: Multi-Destination Sinks
    description: Write events to HDFS, HBase, Solr, Elasticsearch, Kafka, and custom sink destinations.
  - name: Fan-In Consolidation
    description: Aggregate events from multiple agent sources into a single destination for centralized log collection.
  - name: Fan-Out Distribution
    description: Route events from a single source to multiple channel/sink combinations for parallel processing.
  - name: Interceptors
    description: Event transformation interceptors for filtering, enrichment, and routing based on event content.
  - name: SSL/TLS Security
    description: TLS encryption support across Avro, Thrift, Kafka, HTTP, and Syslog components.
  - name: Monitoring REST API
    description: HTTP monitoring endpoint exposing source, channel, and sink metrics for agent health monitoring.
  - name: Multi-Hop Flows
    description: Chain multiple Flume agents via Avro/Thrift RPC for tiered log aggregation architectures.
- type: UseCases
  data:
  - name: Centralized Log Aggregation
    description: Collect application logs from hundreds of servers and aggregate them into HDFS, Kafka, or Elasticsearch.
  - name: Real-Time Log Tailing
    description: Tail application log files in real time using Taildir source for immediate event processing.
  - name: Syslog Ingestion
    description: Ingest RFC-3164 and RFC-5424 syslog events from network devices into centralized storage.
  - name: Kafka Event Ingestion
    description: Bridge Kafka topics to HDFS or other storage for batch analytics on streaming event data.
  - name: Multi-Tier Architectures
    description: Build tiered data collection with edge collectors forwarding to aggregation agents and final destinations.
- type: Integrations
  data:
  - name: Apache Kafka
    description: Kafka source and channel for consuming events, and Kafka sink for writing events to topics.
  - name: Apache HDFS
    description: Primary sink for writing log data to Hadoop Distributed File System for batch analytics.
  - name: Apache HBase
    description: HBase sink for writing events directly to HBase tables for random-access analytics.
  - name: Apache Solr
    description: Solr sink for indexing log events for full-text search capabilities.
  - name: Elasticsearch
    description: Elasticsearch sink for indexing and searching aggregated log data.
maintainers:
- FN: Kin Lane
  email: [email protected]