Skip to content

Sandbox Data Management System (DMS) Operators & Admins Manual

The ./tests/sandbox_dms directory provides a self-contained, containerized FermiHDI Data Management System (DMS) tailored for local development, testing, and benchmarking. It deploys the entire ecosystem securely, mimicking a production-like Data Plane and Control Plane without requiring external infrastructure.

The sandbox is also the reference deployment the installer is built from and checked against: compose.yml is every service with its full environment, and generate_certs_and_license.sh is the clearest statement of the PKI an install needs. install/ADMIN_GUIDE.md ยง8 reads the two side by side, including where the sandbox is deliberately not production.

Architecture

The sandbox provisions the following components via Docker Compose: * DMS-Core: The central control plane component responsible for orchestrating HD instances. * DMS-CA: The internal Certificate Authority responsible for provisioning mutual TLS (mTLS) identities. * DMS-WebUX: The user interface for interacting with the cluster. * HD Proxy: The high-performance entry point evaluating PBAC policies via OPA. * Storage Cluster Controllers (SCC): Nodes managing horizontal data synchronization across Storage Nodes. * Storage Nodes (SN): The data layer managing NVMe/file-backed Data Plane retrieval.

Networking Rules

The sandbox strictly enforces FermiHDI network segmentation: * 6985: Internal DMS Plane (DMS-Core <-> DMS-CA, WebUX) * 6986: Control Plane (HTTPS mTLS) for queries and orchestration. * 6987: Data Plane (FHDWP) for unidirectional, high-speed query response. * 6988: Auxiliary Network (NORM Multicast) * 6989: DNS CA Network * 8080: Observability (Prometheus Metrics & Health)

All containers are completely isolated on an internal bridge network (isolated_network), avoiding host port conflicts unless explicitly mapped.

Operating the Sandbox

Bootstrapping

To start the Sandbox in an automated manner: 

cd ./tests/sandbox_dms
./start_sandbox.sh
The script performs the following actions: 1. Cleans up any existing sandbox state to prevent cross-contamination. 2. Regenerates cryptographic key material (Root CA, Intermediate CA). 3. Evaluates CPU architecture and builds the relevant Docker Compose topology. 4. Executes docker compose up -d.

Customizing Configuration

Administrators can modify the deployment via environment overrides. * CPU Set Pinning: Edit the docker-compose.yml to define explicit CPU affinities for high-throughput testing: 

cpuset: "0-3" # Pin SCC to first 4 logical cores
* Dataset Schema Injection: The proxy schema can be pre-loaded by modifying the environment variable FERMIHDI_DATASET_SCHEMA_FILE.

Diagnostics & Troubleshooting

  • Validating Logs: 
    docker compose -f ./tests/sandbox_dms/docker-compose.yml logs -f dms-core
  • Checking Data Plane Connectivity: Ensure port 6987 is accessible natively if querying from outside the isolated bridge network.
  • Verifying Observability: Navigate to http://localhost:8080/metrics on any given node container.

Shutdown & Cleanup

Always ensure proper teardown to release isolated networks: 

cd ./tests/sandbox_dms
docker compose down -v --remove-orphans
Note: Never issue blanket commands like docker rm -f $(docker ps -aq).