Streaming & Data Analytics

What is Amazon MSK?

Amazon MSK is a fully managed Apache Kafka solution provided by AWS. It runs open-source versions of Apache Kafka enabling you to run your existing applications, tooling, and plugins from partners as well as the Apache Kafka community on the AWS infrastructure without requiring changes to application code.

Fully Managed Service

Amazon Managed Streaming for Apache Kafka (Amazon MSK) enables highly available, secure streaming and processing of your real-time data analytics pipelines created with Apache Kafka. With Amazon MSK, you can use native APIs to populate data lakes, stream database changes, and power machine learning and analytics.

Distributed Architecture

Amazon MSK is a distributed publish-subscribe messaging system that help you to implement fast, scalable and durable solutions. Amazon MSK keeps up with feeds of messages within topics. Producers create the data within the topics and consumers read from those topics. Amazon MSK is distributed, therefore, topics are separated by partitions and replicated across various nodes in different availability zones.

Download our free Amazon MSK eBook today

Download now

Why migrate to Amazon MSK?

Build a highly available and secure MSK cluster in minutes.
Use the applications and tools you know and love – built on open source Kafka platform enabling seamless migration for your streaming applications.
Keep data secure – deploy your MSK cluster in a VPC with encryption at rest and in-transit out of the box.
Lower costs – pay as you go with the ability to scale as your business grows and utilization increases.

Apache Kafka use cases

Real-time web and log analytics

Transaction and event sourcing

Messaging

Decoupled microservices

Streaming ETL

Metrics and log aggregation

MM2 is an open-source Kafka replication engine, but it isn’t straightforward to set-up and configure in your environment. In fact, we estimate that enabling replication between your self-managed Kafka environment and Amazon MSK, takes days, and even weeks of time from your Kafka specialists.

We recognize that deep Kafka expertise is not readily available in every organization, so our engineers decided to do something about this and have developed a plugin which enables replication to be established in minutes.

The plugin is deployed on Amazon MSK Connect so there is no need for additional infrastructure to manage, and we provide Amazon CloudWatch monitoring and alerting for seamless integration.

Why Cornerstone is the perfect partner for MSK migrations

Cornerstone has a breadth of migration planning experience with specific attention on zero service disruption.

All solutions developed using the Well Architected Framework with Security at the core of our engagement.

We implement best in class logging and monitoring solutions so that you have full control over your systems.

We are specialized in helping customers migrate self-managed Kafka workload to Amazon MSK.

TCO analysis embedded into our approach.

We provide Operational Support during the period of migration and documented runbooks for transitioning day-to-day tasks to your Operations and Engineering teams.