Migrations and History Deprecation
May 23, 2022
- Name
- Yevgeny Pats
- @yevgenypats
Update (February 2023): While TimescaleDB is no longer supported, CloudQuery v1 supports append write mode, which can be used to create a historical view of your data. See the write_mode documentation for more information.
This is a short note that v0.24.0 is deprecating migrations from cloudquery together with history.
Late last year we introduced experimental support of historical data with TimescaleDB (See the blog post) (opens in a new tab). Giving it a try we learned that maintaining full migrations for every single table is impossible and also affects developer experience of both third-party providers, ours internally and contributors.
What will change
- We removed both the migration and history support with timescale which should help with developer experience but also we will be looking into implementing history support either for specific tables or views such as a global inventory view together with potential support for data warehouses such as BigQuery, Redshift, Snowflake.
- When upgrading provider version if tables schema was changed, they will be recreated and data will be dropped (re-populated on the next fetch)
We really appreciate everyone who gave feedback and we are excited to continue building and focusing on what you ask and what we can deliver!
