DataSQRL 0.7 Release: The Data Delivery Interface

DataSQRL 0.7 marks a major milestone in our journey to automate data pipelines, thanks to significant improvements to the serving layer:

• Support for the Model Context Protocol (MCP) for tooling and resource access
• REST API support
• JWT-based authentication and authorization

These features enable developers to build a wide range of production-ready data interfaces. This release also includes performance and configuration improvements to the serving layer of DataSQRL-generated pipelines.

You can find the full release notes and source code on our GitHub release page. To update your local installation of DataSQRL, simply pull the latest Docker image:

docker pull datasqrl/cmd:0.7.0

The Last Mile: Data Delivery

Data delivery is the final and most visible stage of any data pipeline. It's how users, applications, and AI agents actually access and consume data. Most enterprise data interactions happen through APIs, making the delivery interface a critical component. At DataSQRL, we've invested heavily in automating the upstream parts of the pipeline: from Flink-powered data processing to Postgres-backed storage. With version 0.7, we turn our focus to the serving layer: introducing support for the Model Context Protocol (MCP) and REST APIs, as well as JWT-based authentication and authorization. These additions ensure seamless integration with most authentication providers and enable secure, token-based data access, with fine-grained authorization logic enforced directly in the SQRL script. This completes our vision of end-to-end pipeline automation, where consumption patterns inform data storage and processing—closing the loop between data production and usage.

Check out the interface documentation for more information.

Major Contributions

In addition to the flagship features—MCP, REST, and JWT support—which we’ll discuss in more detail in future blog posts, the 0.7.0 release contains a number of additional features and improvements.

Configuration & CLI Improvements

• Refactored CLI and Flink configuration logic.
• Improved error messages during package config validation.
• Enforced predictable ordering and sorting of config keys.
• Migrated deprecated config key naming (-dir → -folder), now with compile-time warnings.
• Standardized configuration schema and structured logging to /build/logs.

Testing Infrastructure Enhancements

• Added new sqrl-container-testing module.
• Converted tests to use AssertJ.
• Increased test coverage for SqrlConfig and dependency mapping.
• Fixed test runner error reporting and exit code handling.
• Reworked dependent service startup to trigger post-compilation.

Flink-SQL Runner Integration

• Integrated flink-sql-runner into DatasqrlRun.
• Temporarily merged sqrl-test module into sqrl-run.
• Simplified Docker image setup (public ghcr.io images).
• Updated submodule paths and version to 0.7.0.

Authentication & API Enhancements

• Added initial JWT-based authentication support.
• Published documentation for JWT and Swagger-based OpenAPI specs for REST endpoints.
• Added batch GraphQL mutation support with transactional semantics.
• Replaced GraphQLBigInteger with native Long handling.

Kafka & Runtime Improvements

• Kafka topic names now support templating.
• Added an async OpenAI test use case and resolved snapshot issues.
• Fixed intermittent WebSocket failures in SubscriptionClient.

Project Structure and CI Pipeline

• Simplified project structure and removed outdated dependency declarations.
• Refactored CI pipeline and added automated GitHub package cleanup workflow.

Updated Dependencies

Upgraded versions for the following dependencies and patched critical vulnerabilities:

• Apache Flink and Flink connectors (e.g., Postgres CDC)
• Vert.x and related plugins
• Apache Iceberg, DuckDB, PostgreSQL JDBC
• AWS SDK BOM
• JSON Schema Validator, Netty, OpenCSV
• Micrometer, Log4j, Reactor, Immutables, Testcontainers
• Maven plugins: Enforcer, GPG, Build-helper