Apache Kafka
Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. It provides a REST Proxy API, Kafka Connect REST API, and AsyncAPI for event streaming.
APIs
Kafka REST Proxy API
The Kafka REST Proxy provides a RESTful interface to a Kafka cluster for producing and consuming messages, managing topics, partitions, consumer groups, and viewing cluster stat...
Kafka Connect REST API
Kafka Connect REST API for managing connectors, their configurations, tasks, and offsets for integrating Kafka with external data systems including databases, object stores, and...
Apache Kafka Messaging API
The core Kafka messaging protocol for producing and consuming records to/from topics using the native Kafka binary protocol, supporting exactly-once semantics, compaction, and p...
Capabilities
Apache Kafka Event Streaming
Workflow capability for data engineers and platform architects to manage Kafka topics, produce and consume messages, and manage connectors via Kafka REST Proxy and Connect APIs.
Run with NaftikoFeatures
Handle millions of messages per second with low latency at massive scale.
Guarantee exactly-once message delivery with idempotent producers and transactional APIs.
Automatic replication across brokers for fault tolerance and high availability.
Real-time stream processing via Kafka Streams library and KSQL.
200+ pre-built Kafka Connect connectors for databases, clouds, and SaaS.
Retain the latest value for each key with topic log compaction.
Horizontally scalable consumers with automatic partition rebalancing.
Use Cases
Build event-driven microservices with reliable message delivery.
Move data between systems at scale with exactly-once delivery guarantees.
Process and analyze event streams in real time with Kafka Streams.
Centralize application and infrastructure logs for analysis and alerting.
Capture database changes and stream them to data warehouses and caches.
Integrations
Spark Structured Streaming integration for batch and streaming analytics.
Native Flink Kafka connector for low-latency stream processing.
CDC platform using Kafka Connect to capture database change events.
Kafka Connect Elasticsearch sink for indexing event data.
Kafka Connect S3 sink for archiving event streams to object storage.
HDFS sink connector for streaming data into Hadoop data lake.
Event Specifications
Apache Kafka Messaging API
Apache Kafka is a distributed event streaming platform capable of handling trillions of events a day. This spec describes the core messaging protocol for producing and consuming...
ASYNCAPI