包阅导读总结
1. 关键词:Redis Insight、RDI、数据集成、数据管道、无代码
2. 总结:Redis Insight 使 Redis Data Integration(RDI)更简单,RDI 实现无缝数据集成和更快访问,支持流行数据库且无需代码。通过 YAML 文件配置,在 Redis Insight 中可轻松创建、测试、部署和监控 RDI 数据管道,还能进行数据转换和结果验证。
3. 主要内容:
– Redis Data Integration(RDI)
– 建立数据流式管道,近实时同步数据库到 Redis
– 企业级,高可用,保证至少一次交付
– 支持流行数据库,无需代码,灵活易用
– 使用 YAML 文件管理数据管道
– 包括主配置文件和可选的作业文件
– 可配置连接、数据集、目标 Redis 数据库等
– 在 Redis Insight 中管理 RDI 数据管道
– 创建、测试和部署 RDI 管道
– 简化模板和代码补全
– 可进行数据转换测试
– 一键部署并验证结果
– 流水线创建
– 可下载、上传或新建管道
– 管道配置和测试
– 自动完成配置文件,测试目标数据库连接
– 数据转换
– 包括默认和声明式转换
– Redis Insight 协助创建作业
– 监控和管理
– 查看引擎状态、数据库配置和处理统计
– 可添加 Redis 数据库进行数据可视化和追踪
思维导图:
文章地址:https://redis.io/blog/redis-insight-makes-rdi-even-simpler/
文章来源:redis.io
作者:kevin.wiley@redis.com
发布时间:2024/8/6 15:26
语言:英文
总字数:1017字
预计阅读时间:5分钟
评分:85分
标签:Redis 数据集成,数据同步,Redis Insight,无代码开发,数据管道
以下为原文内容
本内容来源于用户推荐转载,旨在分享知识与观点,如有侵权请联系删除 联系邮箱 media@ilingban.com
Get seamless data integration and faster data access with Redis Data Integration (RDI)— no-code required. Seamlessly sync your database to Redis in near real-time and build apps at scale, faster than ever before.
RDI establishes a data streaming pipeline that mirrors data from your existing database to Redis, so if a record is added or updated, those changes automatically flow into Redis. Plus, it’s fully integrated into Redis Insight, making it easy to create, validate, deploy, and monitor your data pipelines directly from the most powerful, most used Redis GUI.
Enjoy effortless and dependable data syncing with RDI
We built an end-to-end solution using a Capture Data Change (CDC) mechanism that mirrors your apps’s primary database to Redis and a Stream Processor to map and transform source data such as relational tables into fast Redis data structures that match your requirements.
And with RDI, you get an enterprise-grade pipeline that’s highly available, guarantees at-least-once delivery, and ticks all the essential enterprise boxes.
Flexible and easy to use, RDI supports the most popular databases and requires no code. RDI pipelines are declarative, using prebuilt transformations that can be easily configured using expressions and built-in functions. Focus on your application code, create your pipelines with YAML files, and let RDI handle the rest.
Manage your data pipeline easily using YAML files
Get the flexibility you need to design your pipeline with RDI using a series of YAML files. Configure your RDI pipeline easily with human-readable YAML files:
- config.yaml: This is the main configuration file – and the only mandatory one. This file configures the connection to the source, the dataset to replicate (tables, columns), and then the connection to the target Redis database.
- Job files: These are optional configuration files per source table. This lets you massage the data, starting from simple things like changing field names all the way to calculated values and nesting into JSON of one-to-many relationships.
Even the most simple configuration file needs tooling to make you productive. Enter Redis Insight.
Deploy, manage, and visualize your RDI data pipelines in Redis Insight
Redis Insight is the most popular Redis developer tool. Now it’s more useful than ever, equipped with new features to help you build fast apps faster.
Create, test & deploy RDI pipelines
Redis Insight makes pipeline creation simple. With Redis Insight, you can seamlessly:
- Simplify templates & code completion: Redis Insight gets you started with essential files and guides you through code completion, so you don’t have to switch back and forth to reference documentation.
- Dry-run your transformations: Test a job with transformations anytime and compare the results with input mock data.
- Ready, set, deploy: So you’ve dry-run your pipeline? Why not test it with some real data? One click in Redis Insight, and your pipeline is deployed to RDI so you can watch it in action.
- Verify the results: Redis Insight provides pipeline counters and performance statistics to see your pipeline health and performance. Want to go deeper? Connect to the target Redis database and have a closer look at the data.
Streamlined pipeline creation
First, add your RDI endpoint in Redis Insight via the new “Redis Data Integration” tab. From there, you can manage your pipeline by:
- Downloading it from the RDI server if it’s already deployed.
- Uploading an existing RDI pipeline from a file.
- Creating a new pipeline from scratch using RDI templates for easy configuration.

Pipeline configuration and testing
When creating a new RDI pipeline, Redis Insight auto-completes your configuration file. After specifying the configuration, you can test target database connections to verify your RDI instance writes data successfully.

Data transformation
RDI’s other main function is to map data from a source database to a Redis key type. There are two types of data transformations in RDI:
- Default transformation: Each source row is automatically converted into a hash or JSON key in Redis, with the schema used to convert source columns to Redis hash fields or JSON types.
- Declarative transformation: This advanced transformation is specified in YAML files, with each job containing transformations for source tables. These built-in transformations can be customized to manipulate the data and map to additional Redis data structures (strings, sets, streams).
Redis Insight helps you create jobs. Here’s how:
- Templates for easy job creation.
- Auto-completion and syntax highlighting for YAML files.
- Auto-completion and syntax highlighting for JMESPath and SQL functions in a dedicated editor.

Once your data transformation job is ready, you can perform a dry run to view the transformation results without affecting your actual data. After finalizing the transformation jobs, you can download the pipeline to a file or deploy it to your RDI server using a dedicated deployment button.
Monitoring and management
After deployment, navigate to the Statistics page in Redis Insight to view the current RDI engine status, target database configuration, and processing statistics broken down by stream. This overview lets you easily monitor your data integration pipeline.

Plus, add your Redis database in Redis Insight to filter and visualize Redis keys, allowing you to track the data transformation results.
Redis Insight also gives you tutorials on querying your data directly in Redis in real-time, using Redis for common use cases, and many other tutorials to help you get the most out of Redis.
Want to start building faster apps using RDI?
Talk to sales to get started with RDI.
Download Redis Insight for free from our website.
Related resources
Redis Data Integration, now GA
We don’t just help you build fast apps. We help you do it fast. That’s why we’re excited to announce that Redis Data Integration (RDI) has arrived for…
RedisInsight
Get the best in GUI Redis Insight is a powerful desktop user interface that helps you visualize and optimize your data for Redis and Redis Stack. Plus it…
Meeting
Talk to our team Build better apps with our self-managed, on-prem software designed for enterprise compliance, reliability, and resilience. Work with our solutions architects to understand your requirements and…