Skip to main content

Introduction to stagDB

Welcome to stagDB - an open-source tool that converts basic Ubuntu VMs into staging database servers with instant database provisioning and branching capabilities.

What is stagDB?​

stagDB transforms your infrastructure by providing:

  • Instant Database Provisioning: Deploy PostgreSQL, MySQL, and MongoDB instances in seconds using Docker containers
  • Database Branching: Create instant database branches using ZFS snapshots and clones for space-efficient testing
  • Isolation: Each database runs in its own Docker container with complete environment isolation
  • Self-Hosted: Full control over your staging infrastructure with no vendor lock-in

Key Features​

🚀 Multi-Database Support​

stagDB supports multiple database engines with various versions:

  • PostgreSQL: Versions 12, 13, 14, 15, 16 with extension support
  • MySQL: Versions 5.7, 8.0, 8.1 with full compatibility
  • MongoDB: Versions 5.0, 6.0, 7.0 with replica set support

Each database runs in its own isolated Docker container with configurable resources.

🌳 Zero-Copy Database Branching​

Create instant database branches using ZFS copy-on-write technology:

  • Instant Snapshots: Point-in-time captures created in sub-seconds regardless of database size
  • Space-Efficient Clones: Branches share storage with parent database, only storing changes
  • No Data Copying: Create writable database copies without time-consuming dump/restore cycles
  • Branch from Snapshots: Create multiple branches from any snapshot for parallel testing

🔧 RESTful API​

Complete programmatic control via HTTP API:

  • Database Lifecycle: Create, start, stop, restart, and delete databases
  • Snapshot Management: Create, list, and delete snapshots programmatically
  • Branch Operations: Create branches from databases or snapshots
  • Monitoring: Access database statistics and logs via API endpoints
  • Webhook Support: Subscribe to database events (created, deleted, snapshot created)
  • Authentication: API key-based authentication with rate limiting

⚙️ Flexible Resource Configuration​

Control resource allocation per database:

  • Memory Limits: 256MB to 4GB per database
  • CPU Allocation: 0.5 to 4 CPU cores
  • Storage Pools: 1GB to 50GB per database
  • Environment Variables: Custom configuration and initialization scripts
  • Port Management: Automatic or custom port assignment

🔒 Container Isolation​

Complete isolation between database instances:

  • Each database runs in its own Docker container
  • Network isolation with configurable access
  • No interference between different database instances
  • Clean resource cleanup on database deletion
  • Separate ZFS datasets for storage isolation

Getting Started​

Ready to get started? Check out our Installation Guide to set up stagDB on your infrastructure.

Architecture Overview​

stagDB uses a modern containerized architecture:

  • Docker Engine: Provides database isolation and lifecycle management
  • ZFS Filesystem: Enables instant snapshots and copy-on-write cloning
  • RESTful API: HTTP-based programmatic access to all functionality
  • API Key Authentication: Secure access control with rate limiting
  • Event System: Webhook notifications for database lifecycle events

Use Cases​

Development Teams​

Create isolated database environments for each feature branch:

# Developer creates a branch for their feature
curl -X POST https://api.stagdb.com/v1/databases/main-db/branches \
  -d '{"name": "feature-user-auth"}'

# Work with isolated data without affecting team members
# Delete when feature is complete
curl -X DELETE https://api.stagdb.com/v1/databases/feature-user-auth

Test migrations safely before applying to production:

# Snapshot production state
curl -X POST https://api.stagdb.com/v1/databases/prod-db/snapshots \
  -d '{"name": "pre-migration-v2.0"}'

# Create branch and test migration
curl -X POST https://api.stagdb.com/v1/snapshots/pre-migration-v2.0/branches \
  -d '{"name": "migration-test"}'

QA Testing​

Quickly provision fresh test databases with production-like data:

# Create baseline snapshot of production data
curl -X POST https://api.stagdb.com/v1/databases/prod-db/snapshots \
  -d '{"name": "qa-baseline"}'

# Create multiple test environments from same baseline
for env in qa1 qa2 qa3; do
  curl -X POST https://api.stagdb.com/v1/snapshots/qa-baseline/branches \
    -d "{\"name\": \"$env\"}"
done

Reset databases to known states between test runs without time-consuming restores.

CI/CD Pipelines​

Integrate database provisioning into automated testing workflows:

# In your CI pipeline
# 1. Create test database from baseline
curl -X POST https://api.stagdb.com/v1/databases/baseline/branches \
  -d '{"name": "ci-build-'$BUILD_ID'"}'

# 2. Run tests against database
export DB_URL="postgresql://...ci-build-$BUILD_ID..."
npm test

# 3. Clean up automatically
curl -X DELETE https://api.stagdb.com/v1/databases/ci-build-$BUILD_ID

Ensure consistent test environments with instant provisioning and automatic cleanup.

How Database Branching Works​

stagDB leverages ZFS copy-on-write technology to enable instant database branching:

Traditional Approach (Without stagDB)​

# Dump database (slow for large databases)
pg_dump production > dump.sql  # Takes minutes to hours

# Create new database
createdb test-db

# Restore dump (equally slow)
psql test-db < dump.sql  # Takes minutes to hours

With stagDB​

# Create instant branch (seconds regardless of size)
curl -X POST https://api.stagdb.com/v1/databases/production/branches \
  -d '{"name": "test-db"}'
# Completes in seconds, uses minimal storage

Performance Characteristics​

  • Snapshot Creation: Sub-second for most databases
  • Branch Creation: Seconds regardless of database size (10MB or 100GB)
  • Storage Efficiency: Initial branches use near-zero additional storage
  • Copy-on-Write: Only modified data blocks consume additional space

Real-World Example​

A 50GB PostgreSQL database:

  • Traditional clone: 30+ minutes, 50GB additional storage
  • stagDB branch: 3 seconds, ~0GB initial storage (grows as data changes)

Technical Requirements​

System Requirements​

  • Ubuntu 20.04 or 22.04 LTS
  • ZFS support (kernel module or built-in)
  • Docker Engine 20.10 or later
  • Minimum 4GB RAM, 8GB recommended
  • ZFS pool with adequate storage

Network Requirements​

  • Open ports for database access (configurable)
  • API endpoint accessible to clients
  • Optional: webhook endpoint for event notifications

API Integration​

stagDB provides a comprehensive REST API. Example workflow:

# Create PostgreSQL database
curl -X POST https://api.stagdb.com/v1/databases \
  -H "Authorization: Bearer your-api-key" \
  -d '{
    "name": "my-app-db",
    "type": "postgresql",
    "version": "15",
    "config": {
      "memory": "1g",
      "storage": "10g"
    }
  }'

# Response includes connection details
{
  "id": "db-12345",
  "connection": {
    "host": "stagdb.example.com",
    "port": 5432,
    "username": "postgres",
    "password": "generated-password"
  },
  "url": "postgresql://postgres:password@stagdb.example.com:5432/postgres"
}

# Create snapshot
curl -X POST https://api.stagdb.com/v1/databases/my-app-db/snapshots \
  -H "Authorization: Bearer your-api-key" \
  -d '{"name": "before-migration"}'

# Create branch from snapshot
curl -X POST https://api.stagdb.com/v1/snapshots/before-migration/branches \
  -H "Authorization: Bearer your-api-key" \
  -d '{"name": "test-migration"}'

# Monitor database
curl https://api.stagdb.com/v1/databases/my-app-db/stats \
  -H "Authorization: Bearer your-api-key"

See the API Overview for complete endpoint documentation.

Community & Support​

stagDB is an open-source project. Get involved:

License​

stagDB is released under the MIT License.