Kafka Deduplicator
命令行 已验证 活跃🦔 PostHog is an all-in-one developer platform for building successful products. We offer product analytics, web analytics, session replay, error tracking, feature flags, experimentation, surveys, data warehouse, a CDP, and an AI product assistant to help debug your code, ship features faster, and keep all your usage and customer data in one stack.
To provide reliable, partition-aware deduplication of Kafka events, ensuring data integrity and operational stability in streaming pipelines.
功能
- Partition-aware Kafka event deduplication
- State persistence using RocksDB
- Checkpointing for recovery and reassignment
- Support for multiple pipeline types (ingestion_events, clickhouse_events)
- Configurable fail-open mode for operational bypass
使用场景
- Ensuring unique event processing from Kafka topics
- Recovering from consumer rebalances or failures with minimal data loss
- Handling high-throughput event streams with deduplication requirements
- Integrating with downstream Kafka topics or other data sinks after deduplication
非目标
- Providing a generic Kafka consumer without deduplication logic
- Acting as a Kafka producer or broker
- Handling event transformations beyond deduplication
Trust
- info:Issues AttentionThere were 544 issues opened and 163 closed in the last 90 days, indicating a closure rate below 50% and a high volume of open issues.
Compliance
- info:GDPRThe service processes Kafka events which may contain personal data. While not submitting to a third party, the data is processed and potentially logged, with no explicit sanitization mentioned before LLM interaction.
质量评分
已验证类似扩展
Netdata Field Encoder CLI
99The fastest path to AI-powered full stack observability, even for lean teams.
Personhog Writer
75🦔 PostHog is an all-in-one developer platform for building successful products. We offer product analytics, web analytics, session replay, error tracking, feature flags, experimentation, surveys, data warehouse, a CDP, and an AI product assistant to help debug your code, ship features faster, and keep all your usage and customer data in one stack.
Personhog Leader
75🦔 PostHog is an all-in-one developer platform for building successful products. We offer product analytics, web analytics, session replay, error tracking, feature flags, experimentation, surveys, data warehouse, a CDP, and an AI product assistant to help debug your code, ship features faster, and keep all your usage and customer data in one stack.
Batch Import Worker
75🦔 PostHog is an all-in-one developer platform for building successful products. We offer product analytics, web analytics, session replay, error tracking, feature flags, experimentation, surveys, data warehouse, a CDP, and an AI product assistant to help debug your code, ship features faster, and keep all your usage and customer data in one stack.
Livestream
68🦔 PostHog is an all-in-one developer platform for building successful products. We offer product analytics, web analytics, session replay, error tracking, feature flags, experimentation, surveys, data warehouse, a CDP, and an AI product assistant to help debug your code, ship features faster, and keep all your usage and customer data in one stack.
Property Defs Rs
65🦔 PostHog is an all-in-one developer platform for building successful products. We offer product analytics, web analytics, session replay, error tracking, feature flags, experimentation, surveys, data warehouse, a CDP, and an AI product assistant to help debug your code, ship features faster, and keep all your usage and customer data in one stack.