Install with Confluent Cloud

On this page

Before you Begin

Steps
Next Steps
Learn More

You can configure Relational Migrator with Kafka using Confluent Cloud. Configuring Relational Migrator with Confluent Cloud gives your deployment the reliability of Kafka without requiring that you manage your own Kafka cluster.

Before you Begin

You must have Relational Migrator installed on your machine. To download Relational Migrator, see the Download Center.
Because of Confluent's restrictions, you can only use Basic, Standard, and Dedicated clusters. For most use cases, the basic and standard cluster are enough to handle migrating large volumes of data.

Steps

Create a Confluent Cloud environment

Go to Confluent Cloud and create a new environment.

Enable schema registry

Enable schema registry and select a region.

If you have more than 1000 tables to migrate, use the Advanced schema registry tier. Otherwise, the Essentials tier is enough.

Create a cluster

Navigate to your Environment page. Click Add Cluster.

The Basic and Standard tier clusters are enough for most use cases. If you are migrating large volumes of data, consider the Dedicated cluster tier.

Configure permissions and gather API keys

To configure Relational Migrator, you must configure and gather the Cloud API key and Kafka Cluster API key. You'll need to include them in your user.properties file. The location of your user.properties file varies according to your operating system. For more information, see Relational Migrator File Locations.

The following API keys are found in the Confluent Cloud console. For more information, see Use API Keys.

Configure Confluent Cloud API keys

In the top right corner of the Confluent Console, click the icon. Under Administration, select Cloud API Keys.

Create Confluent Cloud API keys and configure permissions for them. You have two permission options:

Grant global access.
Create a service account and grant granular access.
If you create a service account, you must add the Operator role for the environment and the CloudClusterAdmin role to the cluster.

Once created, save the API key and secret. These values are added to your user.properties file as migrator.confluent.cloud-credentials.api-key and migrator.confluent.cloud-credentials.api-secret.

For more information, see Use API Keys to Control Access in Confluent Cloud.

Configure Kafka cluster API keys

To access your Kafka cluster, you must configure your Kafka cluster API keys. Navigate to your Cluster Overview. In the left-side panel, click API Keys to create your Kafka cluster API keys.

Save the API key and secret. These values are added to your user.properties file as migrator.confluent.kafka-credentials.api-key and migrator.confluent.kafka-credentials.api-secret.

For more information, see Use API Keys to Control Access in Confluent Cloud.

Gather more configuration information

You'll find your environment ID, cluster ID, Bootstrap server, and cluster REST endpoint in the Confluent Cloud console. For more information, see Confluent Cloud Documentation.

Find and save your environment ID

From your Environments overview page, choose your cloud environment. Your environment ID is below the environment name in the right-side panel.

The general environment ID form is:

env-ab1cde

Find and save your cluster ID

Navigate to your Cluster Overview page and find your cluster ID in the right-side information panel.

The general cluster ID form is:

abc-1test2

Find and save your cluster's Bootstrap server

From your Cluster Overview page, use the left-side navigation and click Cluster Settings. On the Cluster Settings page, locate your cluster's Bootstrap server.

The general Bootstrap server form is:

xxx-xxxxx.region.provider.confluent.cloud:9092

Find and save your cluster REST endpoint

On the Cluster Settings page, find your cluster's REST endpoint.

The general REST endpoint form is:

https://xxx-xxxxx.region.provider.confluent.cloud:443

Add the configuration properties to your `user.properties` file

The location of your user.properties file varies according to your operating system. For more information, see Relational Migrator File Locations.

Use the following example to add the configuration properties to your user.properties file:

spring.profiles.active: confluent
migrator.confluent.environment.environment-id: <Environment ID>
migrator.confluent.environment.cluster-id: <Cluster ID>
migrator.confluent.environment.bootstrap-server: <xx.region.provider.confluent.cloud:9092>
migrator.confluent.environment.rest-endpoint: <https://xxx.region.provider.confluent.cloud:443>
migrator.confluent.cloud-credentials.api-key: <Cloud API key>
migrator.confluent.cloud-credentials.api-secret: <Cloud API Secret>
migrator.confluent.kafka-credentials.api-key: <Cluster API key>
migrator.confluent.kafka-credentials.api-secret: <Cluster API Secret>

Restart Relational Migrator

Exit Relational Migrator and relaunch it.

Confirm successful installation

Start Relational Migrator. In your project, navigate to the Data Migration tab. If the installation is successful, on the top right corner, a Confluent Deployment model symbol is present.

If errors appear, check your log file. Your log file location varies by OS. For details, see Relational Migrator File Locations.

click to enlarge

Next Steps

After you verify a successful installation, you can create a new project.

Learn More

← Install with a Containerized Cluster Using Docker

Advanced Kafka Settings →