4 posts tagged with "Release"

DataSQRL is an open-source data engineering harness that provides guardrails and feedback for AI coding agents to develop and operate data pipelines, data products, and data APIs autonomously. You can customize DataSQRL as the foundation of your agentic data platform. Our goal is to develop DataSQRL into a comprehensive data engineering harness for data platform automation.

DataSQRL 0.10 has been released and the headline feature is supporting mutations for Iceberg tables. DataSQRL can now manage Apache Iceberg tables as sources and sinks.

Why is that a big deal? Up to this point, DataSQRL could read and write to Apache Iceberg tables, but you had to manage them explicitly. This new release makes it easy to share data through Apache Iceberg between DataSQRL pipelines.

DataSQRL 0.7 marks a major milestone in our journey to automate data pipelines, thanks to significant improvements to the serving layer:

• Support for the Model Context Protocol (MCP) for tooling and resource access
• REST API support
• JWT-based authentication and authorization

These features enable developers to build a wide range of production-ready data interfaces. This release also includes performance and configuration improvements to the serving layer of DataSQRL-generated pipelines.

You can find the full release notes and source code on our GitHub release page. To update your local installation of DataSQRL, simply pull the latest Docker image:

docker pull datasqrl/cmd:0.7.0

The Last Mile: Data Delivery​

Data delivery is the final and most visible stage of any data pipeline. It's how users, applications, and AI agents actually access and consume data. Most enterprise data interactions happen through APIs, making the delivery interface a critical component. At DataSQRL, we've invested heavily in automating the upstream parts of the pipeline: from Flink-powered data processing to Postgres-backed storage. With version 0.7, we turn our focus to the serving layer: introducing support for the Model Context Protocol (MCP) and REST APIs, as well as JWT-based authentication and authorization. These additions ensure seamless integration with most authentication providers and enable secure, token-based data access, with fine-grained authorization logic enforced directly in the SQRL script. This completes our vision of end-to-end pipeline automation, where consumption patterns inform data storage and processing—closing the loop between data production and usage.

Check out the interface documentation for more information.

The DataSQRL community is proud to announce the release of DataSQRL 0.6. This release marks a major milestone in the evolution of our open-source project, bringing enhanced alignment with Flink SQL and powerful new capabilities to the real-time serving layer.

You can find the full release notes and source code on our GitHub release page. To get started with the latest compiler, simply pull the latest Docker image:

docker pull datasqrl/cmd:0.6.0

A New Chapter: Flink SQL Integration​

With DataSQRL 0.6, we are embracing the Flink ecosystem more deeply than ever before. This release introduces a complete re-architecture of the DataSQRL compiler to build directly on top of Flink SQL's parser and planner. By aligning our internal model with Flink SQL semantics, we unlock a host of new capabilities and bring DataSQRL users closer to the vibrant Flink ecosystem.

This architectural shift allows DataSQRL to:

• Use Flink SQL syntax as the foundation, enabling more intuitive query definitions and easier onboarding for users familiar with Flink.
• Extend Flink SQL with domain-specific features, such as declarative relationship definitions and functions to define the data interface.
• Transpile FlinkSQL to database dialects for query execution.