This project contains a set of Terraform configurations and Ansible playbooks that will deploy to AWS a complete Kafka cluster (including Zookeeper), as well as a set of producer and consumer applications running in a Docker Swarm cluster. It is fully configured and instrumented with New Relic.
You'll see how New Relic provides observability into Kafka clusters as well as producer and consumer applications in a simulation of a real-world asynchronous messaging scenario. It is completely configurable and extensible, allowing you to scale your cluster size and to modify/redeploy the applications to experiment with the features of the New Relic platform and how they relate to messages passed via Kafka streams.
Features
- Instruments Kafka with the New Relic Kafka On-Host Integration
- All hosts and containers are instrumented with the New Relic Infrastructure Agent
- Logs collected from both the Kafka brokers as well as the producer and consumer applications.
- The applications are instrumented with New Relic APM
- Distributed Tracing has been enabled between producers and consumers
- Logs in Context for all logs emitted by the producers and consumers.
Getting Started
Go to the Project's README for setup and usage details.
Contributions
New Relic Kafka Playground accepts contributions under our open source Code of Conduct.
Issues / Enhancement Requests
Issues and enhancement requests can be submitted in the Issues tab of this repository. Please search for and review the existing open issues before submitting a new issue.
Code Contributions
Code contributions are welcome. Please review our Contributors Guide and review the Issues list. New Relic open source projects follow a fork-and-pull-request code contribution workflow.
Top Contributors
Thank you to the following and all contributors.