Skip to main content

PostgreSQL Engine Configuration

PostgreSQL is a realtime database that stores the materialized views and tables generated by your DataSQRL pipeline for low-latency querying.

Configuration Options​

No mandatory configuration keys are required. Physical DDL (tables, indexes, views) is produced automatically by the DataSQRL compiler.

Basic Configuration​

{
  "engines": {
    "postgres": {
      "config": {
        // Optional PostgreSQL-specific settings
      }
    }
  }
}

Deployment Configuration​

PostgreSQL supports deployment-specific configuration for database scaling and high availability:

KeyTypeDefaultDescription
instance-sizestring-Database instance size for compute and memory
replica-countinteger-Number of read replicas (minimum: 0, maximum varies)

Instance Size Options​

Available instance-size options:

  • dev - Development/testing size with minimal resources
  • small - Small production workloads
  • medium - Medium production workloads
  • large - Large production workloads
  • xlarge - Extra large production workloads

Deployment Example​

{
  "engines": {
    "postgres": {
      "deployment": {
        "instance-size": "large",
        "replica-count": 2
      }
    }
  }
}

Usage Notes​

  • Database schema is automatically generated from your SQRL script
  • Tables, indexes, and views are created based on the compiled data pipeline
  • Connection parameters are typically provided via environment variables
  • The engine handles both real-time data ingestion and query serving
  • Optimized for low-latency reads of materialized data
  • Read replicas improve query performance and provide redundancy

Internal Environment Variables​

When running pipelines with the DataSQRL run command, the following environment variables are used in the configuration:

  • POSTGRES_VERSION
  • POSTGRES_HOST
  • POSTGRES_PORT
  • POSTGRES_DATABASE
  • POSTGRES_AUTHORITY
  • POSTGRES_JDBC_URL
  • POSTGRES_USERNAME
  • POSTGRES_PASSWORD