Data Service API Endpoint Reference

Complete reference for all Data Service API endpoints with request/response examples.

Authentication

All endpoints require JWT authentication. Include the access token in the Authorization header:

Authorization: Bearer YOUR_ACCESS_TOKEN

Apps Endpoints

List Apps

GET /api/apps/

Response:

[
  {
    "id": 1,
    "name": "My Application",
    "slug": "my-application",
    "description": "Application description",
    "datasource_count": 2,
    "created_at": "2024-10-30T12:00:00Z",
    "updated_at": "2024-10-30T12:00:00Z"
  }
]

Create App

POST /api/apps/
Content-Type: application/json

{
  "name": "New Application",
  "description": "Optional description"
}

Response: 201 Created

{
  "id": 2,
  "name": "New Application",
  "slug": "new-application",
  "description": "Optional description",
  "datasource_count": 0,
  "table_count": 0,
  "datasources": [],
  "created_at": "2024-10-30T12:00:00Z",
  "updated_at": "2024-10-30T12:00:00Z",
  "created_by": 1,
  "modified_by": null
}

Get App Details

GET /api/apps/{app_slug}/

Response:

{
  "id": 1,
  "name": "My Application",
  "slug": "my-application",
  "description": "Application description",
  "datasource_count": 2,
  "table_count": 5,
  "datasources": [
    {
      "id": 1,
      "name": "Users DB",
      "slug": "users-db",
      "provider_type": "flat_table",
      "table_count": 3
    },
    {
      "id": 2,
      "name": "Analytics DB",
      "slug": "analytics-db",
      "provider_type": "jsonb",
      "table_count": 2
    }
  ],
  "created_at": "2024-10-30T12:00:00Z",
  "updated_at": "2024-10-30T12:00:00Z",
  "created_by": 1,
  "modified_by": null
}

Update App

PUT /api/apps/{app_slug}/
Content-Type: application/json

{
  "name": "Updated Name",
  "description": "Updated description"
}

PATCH /api/apps/{app_slug}/
Content-Type: application/json

{
  "description": "Just update description"
}

Delete App (Soft Delete)

DELETE /api/apps/{app_slug}/

Response: 204 No Content

Note: This marks the app, all its datasources, and all datatables as inactive.

Get App Statistics

GET /api/apps/{app_slug}/stats/

Response:

{
  "app_name": "My Application",
  "app_slug": "my-application",
  "total_datasources": 2,
  "total_tables": 5,
  "materialized_tables": 3,
  "unmaterialized_tables": 2,
  "datasources": [
    {
      "name": "Users DB",
      "slug": "users-db",
      "provider_type": "flat_table",
      "total_tables": 3,
      "materialized_tables": 2
    },
    {
      "name": "Analytics DB",
      "slug": "analytics-db",
      "provider_type": "jsonb",
      "total_tables": 2,
      "materialized_tables": 1
    }
  ]
}

Restore App

POST /api/apps/{app_slug}/restore/

Response:

{
  "message": "App restored successfully"
}

DataSource Endpoints

List DataSources

GET /api/apps/{app_slug}/datasources/

Response:

[
  {
    "id": 1,
    "name": "Users DB",
    "slug": "users-db",
    "app_name": "My Application",
    "app_slug": "my-application",
    "provider_type": "flat_table",
    "description": "User data storage",
    "is_active": true,
    "table_count": 3,
    "created_at": "2024-10-30T12:00:00Z",
    "updated_at": "2024-10-30T12:00:00Z"
  }
]

Create DataSource

POST /api/apps/{app_slug}/datasources/
Content-Type: application/json

{
  "name": "Analytics DB",
  "slug": "analytics-db",
  "provider_type": "jsonb",
  "provider_config": {
    "compression": "lz4",
    "indexing": "gin"
  },
  "description": "Analytics data storage"
}

Provider Types:

flat_table: Traditional PostgreSQL tables
jsonb: PostgreSQL JSONB storage
mongodb: MongoDB collections

Response: 201 Created

{
  "id": 2,
  "app": 1,
  "app_name": "My Application",
  "app_slug": "my-application",
  "name": "Analytics DB",
  "slug": "analytics-db",
  "provider_type": "jsonb",
  "provider_config": {
    "compression": "lz4",
    "indexing": "gin"
  },
  "description": "Analytics data storage",
  "is_active": true,
  "table_count": 0,
  "materialized_count": 0,
  "created_at": "2024-10-30T12:00:00Z",
  "updated_at": "2024-10-30T12:00:00Z",
  "created_by": 1,
  "modified_by": null
}

Get DataSource Details

GET /api/apps/{app_slug}/datasources/{datasource_slug}/

Update DataSource

PUT /api/apps/{app_slug}/datasources/{datasource_slug}/
PATCH /api/apps/{app_slug}/datasources/{datasource_slug}/

Delete DataSource

DELETE /api/apps/{app_slug}/datasources/{datasource_slug}/

Get DataSource Tables

GET /api/apps/{app_slug}/datasources/{datasource_slug}/tables/

Get DataSource Statistics

GET /api/apps/{app_slug}/datasources/{datasource_slug}/stats/

Response:

{
  "total_tables": 3,
  "materialized_tables": 2,
  "unmaterialized_tables": 1,
  "provider_type": "flat_table",
  "is_active": true
}

DataTable Endpoints

List DataTables

GET /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/

Response:

[
  {
    "id": 1,
    "name": "users",
    "datasource_name": "Users DB",
    "datasource_slug": "users-db",
    "app_name": "My Application",
    "app_slug": "my-application",
    "physical_table_name": "myapp_users",
    "is_materialized": true,
    "is_active": true,
    "field_count": 5,
    "created_at": "2024-10-30T12:00:00Z",
    "updated_at": "2024-10-30T12:00:00Z"
  }
]

Create DataTable

POST /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/
Content-Type: application/json

{
  "name": "users",
  "description": "User accounts table",
  "json_schema": {
    "fields": [
      {
        "name": "id",
        "type": "integer",
        "constraints": {
          "required": true,
          "unique": true
        }
      },
      {
        "name": "email",
        "type": "string",
        "format": "email",
        "constraints": {
          "required": true,
          "unique": true
        }
      },
      {
        "name": "name",
        "type": "string",
        "constraints": {
          "required": true
        }
      },
      {
        "name": "is_active",
        "type": "boolean"
      },
      {
        "name": "created_at",
        "type": "datetime"
      }
    ],
    "primaryKey": ["id"]
  }
}

Response: 201 Created

{
  "id": 1,
  "data_source": 1,
  "datasource_name": "Users DB",
  "datasource_slug": "users-db",
  "app_name": "My Application",
  "app_slug": "my-application",
  "name": "users",
  "json_schema": {
    "fields": [...],
    "primaryKey": ["id"]
  },
  "physical_table_name": "myapp_users",
  "is_materialized": false,
  "is_active": true,
  "description": "User accounts table",
  "schema_hash": "a7f3b8c...",
  "field_count": 5,
  "field_names": ["id", "email", "name", "is_active", "created_at"],
  "primary_key": ["id"],
  "created_at": "2024-10-30T12:00:00Z",
  "updated_at": "2024-10-30T12:00:00Z",
  "created_by": 1,
  "modified_by": null
}

Get DataTable Details

GET /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/{name}/

Update DataTable

PUT /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/{name}/
PATCH /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/{name}/
Content-Type: application/json

{
  "description": "Updated description",
  "json_schema": {
    // Updated schema
  }
}

Note: Schema updates automatically recalculate the schema_hash.

Delete DataTable

DELETE /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/{name}/

Get Schema Only

GET /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/{name}/schema/

Response:

{
  "name": "users",
  "schema": {
    "fields": [...],
    "primaryKey": ["id"]
  },
  "schema_hash": "a7f3b8c...",
  "field_count": 5
}

Validate Data Against Schema

POST /api/apps/{app_slug}/datasources/{datasource_slug}/datatables/{name}/validate_data/
Content-Type: application/json

{
  "data": [
    {
      "id": 1,
      "email": "[email protected]",
      "name": "John Doe",
      "is_active": true,
      "created_at": "2024-10-30T12:00:00Z"
    },
    {
      "id": 2,
      "email": "invalid-email",
      "name": "Jane Smith"
    }
  ]
}

Response:

{
  "is_valid": false,
  "errors": [
    "Row 1: Missing required field 'email'"
  ],
  "validated_rows": 2
}

Data CRUD Endpoints

The following endpoints operate on the actual data stored in materialized tables. See CRUD Operations Guide for detailed examples.

Base URL: /api/apps/{app_slug}/datatables/{name}/data/

List Records

GET /api/apps/{app_slug}/datatables/{name}/data/

Query Parameters:

Pagination: ?page=1&page_size=20 or ?limit=20&offset=0
Filtering: ?field=value, ?field__gt=100, ?field__contains=text (see all operators)
Sorting: ?ordering=-created_at,title
Populate: ?populate=author,comments or ?populate=*
Aggregation: ?_aggregate=sum(price)&_group_by=category (see aggregations)
Format: ?format=flat|tree|graph
Search: ?search=keyword

Response:

{
  "data": [
    {
      "id": 1,
      "title": "First Post",
      "author_id": 5,
      "status": "published",
      "created_at": "2024-01-15T10:00:00Z"
    }
  ],
  "total": 150,
  "page": 1,
  "page_size": 20
}

Get Single Record

GET /api/apps/{app_slug}/datatables/{name}/data/{id}/

Query Parameters: Same as List (populate, format, etc.)

Response:

{
  "data": {
    "id": 1,
    "title": "First Post",
    "content": "Post content...",
    "author_id": 5,
    "created_at": "2024-01-15T10:00:00Z"
  }
}

Create Record

POST /api/apps/{app_slug}/datatables/{name}/data/
Content-Type: application/json

{
  "title": "New Post",
  "content": "Content here",
  "author_id": 5,
  "status": "draft"
}

Response: 201 Created

{
  "data": {
    "id": 42,
    "title": "New Post",
    "content": "Content here",
    "author_id": 5,
    "status": "draft",
    "created_at": "2024-01-15T12:00:00Z",
    "updated_at": "2024-01-15T12:00:00Z"
  }
}

Bulk Create

POST /api/apps/{app_slug}/datatables/{name}/data/
Content-Type: application/json

[
  {"title": "Post 1", "author_id": 5},
  {"title": "Post 2", "author_id": 5},
  {"title": "Post 3", "author_id": 6}
]

Response: 201 Created

{
  "data": [
    {"id": 42, "title": "Post 1", ...},
    {"id": 43, "title": "Post 2", ...},
    {"id": 44, "title": "Post 3", ...}
  ]
}

Upsert (Insert or Update)

POST /api/apps/{app_slug}/datatables/{name}/data/upsert/?unique_fields=email
Content-Type: application/json

{
  "email": "[email protected]",
  "name": "Alice Smith",
  "bio": "Software engineer"
}

Response: 200 OK (if updated) or 201 Created (if inserted)

{
  "data": {
    "id": 5,
    "email": "[email protected]",
    "name": "Alice Smith",
    "bio": "Software engineer",
    "created_at": "2024-01-10T08:00:00Z",
    "updated_at": "2024-01-15T12:00:00Z"
  },
  "created": false
}

Update Record

PATCH /api/apps/{app_slug}/datatables/{name}/data/{id}/
Content-Type: application/json

{
  "status": "published",
  "published_at": "2024-01-15T12:00:00Z"
}

Response: 200 OK

{
  "data": {
    "id": 42,
    "title": "My Post",
    "status": "published",
    "published_at": "2024-01-15T12:00:00Z",
    "updated_at": "2024-01-15T12:00:00Z"
  }
}

Bulk Update

PATCH /api/apps/{app_slug}/datatables/{name}/data/
Content-Type: application/json

[
  {"id": 42, "status": "published"},
  {"id": 43, "status": "archived"}
]

Response: 200 OK

{
  "data": [
    {"id": 42, "status": "published", "updated_at": "2024-01-15T12:00:00Z"},
    {"id": 43, "status": "archived", "updated_at": "2024-01-15T12:00:00Z"}
  ]
}

Delete Record

DELETE /api/apps/{app_slug}/datatables/{name}/data/{id}/

Response: 204 No Content

Bulk Delete by IDs

DELETE /api/apps/{app_slug}/datatables/{name}/data/?ids=42,43,44

Response: 200 OK

{
  "deleted": 3,
  "ids": [42, 43, 44]
}

Bulk Delete by Filter

DELETE /api/apps/{app_slug}/datatables/{name}/data/?filter={"status":"draft","created_at__lt":"2023-01-01"}

Response: 200 OK

{
  "deleted": 15
}

Graph/Relationship Endpoints

Manage edges for graph traversal and hierarchical data. See Hierarchy Guide for details.

Base URL: /api/apps/{app_slug}/datatables/{name}/edges/

List Edges

GET /api/apps/{app_slug}/datatables/{name}/edges/?from_id=5&to_id=10&types=manager,dotted_line&limit=100

Response:

{
  "data": [
    {
      "id": 1,
      "from_id": 5,
      "to_id": 10,
      "type": "manager",
      "metadata": {"title": "Engineering Manager"},
      "created_at": "2024-01-15T10:00:00Z"
    }
  ],
  "total": 50
}

Bulk Create Edges

POST /api/apps/{app_slug}/datatables/{name}/edges/
Content-Type: application/json

[
  {"from": 1, "to": 2, "type": "manager", "metadata": {"department": "Engineering"}},
  {"from": 1, "to": 3, "type": "mentor", "metadata": {"since": "2024-01-01"}}
]

Response: 201 Created

{
  "data": [
    {"id": 10, "from_id": 1, "to_id": 2, "type": "manager", ...},
    {"id": 11, "from_id": 1, "to_id": 3, "type": "mentor", ...}
  ]
}

Bulk Delete Edges

DELETE /api/apps/{app_slug}/datatables/{name}/edges/
Content-Type: application/json

{
  "edge_ids": [10, 11, 12]
}

Response: 200 OK

{
  "deleted": 3
}

Operation History

View migration history for a datatable. See Migrations Guide.

GET /api/apps/{app_slug}/datatables/{name}/operation-history/

Query Parameters:

?status=success|failed|rolled_back
?page=1&page_size=20

Response:

{
  "data": [
    {
      "id": 1,
      "operation_type": "ALTER",
      "description": "Add column 'email'",
      "sql": "ALTER TABLE posts ADD COLUMN email VARCHAR(255);",
      "rollback_sql": "ALTER TABLE posts DROP COLUMN email;",
      "status": "success",
      "execution_time_ms": 125,
      "created_at": "2024-01-15T10:00:00Z"
    }
  ]
}

Cross-DataSource Endpoints

List All DataTables for App

GET /api/apps/{app_slug}/datatables/

Lists all datatables across all datasources for the app.

Query Parameters:

datasource: Filter by datasource slug
is_materialized: Filter by materialization status (true/false)

Examples:

# All tables
GET /api/apps/my-app/datatables/

# Tables in specific datasource
GET /api/apps/my-app/datatables/?datasource=users-db

# Only materialized tables
GET /api/apps/my-app/datatables/?is_materialized=true

Get Cross-DataSource Statistics

GET /api/apps/{app_slug}/datatables/stats/

Response:

{
  "total_tables": 5,
  "materialized_tables": 3,
  "unmaterialized_tables": 2,
  "datasource_count": 2,
  "datasources": {
    "users-db": {
      "name": "Users DB",
      "provider_type": "flat_table",
      "table_count": 3,
      "materialized_count": 2
    },
    "analytics-db": {
      "name": "Analytics DB",
      "provider_type": "jsonb",
      "table_count": 2,
      "materialized_count": 1
    }
  }
}

Error Responses

Validation Errors

{
  "json_schema": [
    "Invalid Frictionless schema: Field 'age' has invalid constraints"
  ]
}

Not Found

HTTP/1.1 404 Not Found

{
  "detail": "Not found."
}

Permission Denied

HTTP/1.1 403 Forbidden

{
  "detail": "You do not have permission to perform this action."
}

Rate Limit Exceeded

HTTP/1.1 429 Too Many Requests

{
  "detail": "Request was throttled. Expected available in 45 seconds."
}

Pagination

List endpoints support pagination:

GET /api/apps/?page=2&page_size=20

Response:

{
  "count": 50,
  "next": "http://api.example.com/api/apps/?page=3",
  "previous": "http://api.example.com/api/apps/?page=1",
  "results": [...]
}

Complete Workflow Example

1. Create App

curl -X POST https://api.example.com/api/apps/ \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "E-Commerce Platform",
    "description": "Main e-commerce application"
  }'

2. Create DataSource

curl -X POST https://api.example.com/api/apps/e-commerce-platform/datasources/ \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Product Database",
    "slug": "products-db",
    "provider_type": "flat_table"
  }'

3. Create DataTable

curl -X POST https://api.example.com/api/apps/e-commerce-platform/datasources/products-db/datatables/ \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "products",
    "json_schema": {
      "fields": [
        {"name": "id", "type": "integer", "constraints": {"required": true, "unique": true}},
        {"name": "name", "type": "string", "constraints": {"required": true}},
        {"name": "price", "type": "number", "constraints": {"required": true, "minimum": 0}},
        {"name": "in_stock", "type": "boolean"}
      ],
      "primaryKey": ["id"]
    }
  }'

4. Get Statistics

curl https://api.example.com/api/apps/e-commerce-platform/stats/ \
  -H "Authorization: Bearer YOUR_TOKEN"

Best Practices

Always validate schemas before deployment
Use slug-based lookups for stable URLs
Implement pagination for large result sets
Handle rate limits gracefully with exponential backoff
Monitor schema_hash for cache invalidation
Use soft deletes to maintain data history
Document your schemas with title and description fields

Rate Limiting Strategy

import time
import requests

def api_call_with_retry(url, headers, data, max_retries=3):
    for attempt in range(max_retries):
        response = requests.post(url, headers=headers, json=data)

        if response.status_code == 429:  # Rate limited
            retry_after = int(response.headers.get('Retry-After', 60))
            time.sleep(retry_after)
            continue

        return response

    raise Exception("Max retries exceeded")

Authentication​

Apps Endpoints​

List Apps​

Create App​

Get App Details​

Update App​

Delete App (Soft Delete)​

Get App Statistics​

Restore App​

DataSource Endpoints​

List DataSources​

Create DataSource​

Get DataSource Details​

Update DataSource​

Delete DataSource​

Get DataSource Tables​

Get DataSource Statistics​

DataTable Endpoints​

List DataTables​

Create DataTable​

Get DataTable Details​

Update DataTable​

Delete DataTable​

Get Schema Only​

Validate Data Against Schema​

Data CRUD Endpoints​

List Records​

Get Single Record​

Create Record​

Bulk Create​

Upsert (Insert or Update)​

Update Record​

Bulk Update​

Delete Record​

Bulk Delete by IDs​

Bulk Delete by Filter​

Graph/Relationship Endpoints​

List Edges​

Bulk Create Edges​

Bulk Delete Edges​

Operation History​

Cross-DataSource Endpoints​

List All DataTables for App​

Get Cross-DataSource Statistics​

Error Responses​

Validation Errors​

Not Found​

Permission Denied​

Rate Limit Exceeded​

Pagination​

Complete Workflow Example​

1. Create App​

2. Create DataSource​

3. Create DataTable​

4. Get Statistics​

Best Practices​

Rate Limiting Strategy​

Authentication

Apps Endpoints

List Apps

Create App

Get App Details

Update App

Delete App (Soft Delete)

Get App Statistics

Restore App

DataSource Endpoints

List DataSources

Create DataSource

Get DataSource Details

Update DataSource

Delete DataSource

Get DataSource Tables

Get DataSource Statistics

DataTable Endpoints

List DataTables

Create DataTable

Get DataTable Details

Update DataTable

Delete DataTable

Get Schema Only

Validate Data Against Schema

Data CRUD Endpoints

List Records

Get Single Record

Create Record

Bulk Create

Upsert (Insert or Update)

Update Record

Bulk Update

Delete Record

Bulk Delete by IDs

Bulk Delete by Filter

Graph/Relationship Endpoints

List Edges

Bulk Create Edges

Bulk Delete Edges

Operation History

Cross-DataSource Endpoints

List All DataTables for App

Get Cross-DataSource Statistics

Error Responses

Validation Errors

Not Found

Permission Denied

Rate Limit Exceeded

Pagination

Complete Workflow Example

1. Create App

2. Create DataSource

3. Create DataTable

4. Get Statistics

Best Practices

Rate Limiting Strategy