Export from Yandex Cloud to ClickHouse

Step 1. Install the CloudQuery CLI

The CloudQuery CLI is a command-line tool that runs the sync. It supports MacOS, Linux and Windows.

brew install cloudquery/tap/cloudquery

Step 2. Configure the Yandex Cloud source plugin

Create a configuration file for the Yandex Cloud plugin and set up authentication.

Yandex Cloud is a partner plugin, which means that it is maintained by the Yandex Cloud team. Create a file called yandexcloud.yaml, then copy the example and follow the instructions in the Yandex Cloud Plugin Documentation ↗ to fit your needs.

Step 3. Configure the ClickHouse destination plugin

Create a configuration file for the ClickHouse plugin and set up authentication.

Configuration

Create a file called clickhouse.yaml and add the following contents:

Fine-tune this configuration to match your needs. For more information, see the ClickHouse Plugin ↗ page in the docs.

Step 4. Start the Sync

Run the following command in your terminal to start the sync

And away we go! 🚀 The sync will run until completion, fetching all selected tables from Yandex Cloud. Any errors will be logged to a file called cloudquery.log.

Further Reading

Now that you've seen the basics of syncing Yandex Cloud to ClickHouse, you should know that there's a lot more you can do. Check out the CloudQuery Documentation, Source Code and How-to Guides for more details.