Under the Hood

What is
Data Product.

An authentic data product goes beyond a mere table; it is a self-sufficient, deployable structure that includes code, data, and the necessary infrastructure.

View Components Explore Interfaces

The Three Pillars

Each autonomous Data Product consists of three separate components that are deployed and managed collectively as a unified entity.

Data & Metadata

The historical and current datasets are enriched with detailed semantics, schemas, and cataloging information to ensure the data is both comprehensible and practical.

Code & Logic

The necessary pipelines, transformations, API endpoints, access control policies, and testing scripts to securely ingest, process, and serve the data.

Infrastructure

The product is run using Infrastructure-as-Code to provision storage buckets, compute clusters, orchestrators, and other physical or cloud resources.

Standardized Interfaces (Ports)

In order to preserve autonomy and enable interoperability within the organization, Data Products use clearly defined 'ports' to communicate with external systems.

Output Ports

The main interfaces available to consumers are highly dependable REST APIs, GraphQL, traditional SQL views, and event streams optimized for seamless integration.

Input Ports

The methods employed to safely transfer operational data or data from other data products into the current product's storage.

Control & Discovery Ports

Interfaces enable central governance platforms to automatically monitor SLA metrics, audit logs, schema registries, and enforce global security policies.

// Sample Output Port config

{
  "product_id": "customer_360",
  "version": "v1.2.0",
  "output_ports": [
    {
      "type": "SQL",
      "endpoint": "snowflake://db/schema/vw_customers",
      "sla": "99.9%",
      "refresh_rate": "real-time"
    },
    {
      "type": "REST_API",
      "endpoint": "https://api.data.inc/v1/customers",
      "auth": "OAuth2"
    }
  ]
}

What is Data Product.