name: fermihdi-hd-plan description: Probe one or more servers intended to support an HD instance and recommend a concrete deployment — topology, storage-node count and storage type, hugepage sizing, CPU/NUMA placement, whether to run a BloomRF filter node, and NIC/jumbo-frame choices — from real hardware facts and the workload's expected shape. Runs before fermihdi-hd-install or fermihdi-hd-sandbox, which then carry out the plan this skill produces.

Plan an HD deployment from real hardware

This is advisory — it probes hosts and does the sizing math, then hands the result to fermihdi-hd-install (production) or fermihdi-hd-sandbox (single-host eval) to actually execute. It never applies anything to a host itself.

Step 1: probe the host(s)

install/fermihdi-discover is the real per-host probe — reads /sys//proc only, no dependencies (no lspci, numactl, or Python), safe on a bare host before anything is installed:

./fermihdi-discover                 # YAML facts to stdout
./fermihdi-discover -o facts.yaml   # write to a file
./fermihdi-discover --format=tsv    # for shell consumers
./fermihdi-discover --check-deps    # report tool availability and exit

Run it once per host in a multi-host deployment (the Ansible layer does this automatically in a real install). It reports CPU topology, NUMA layout, IOMMU groups, candidate storage devices, and NICs — the raw material every recommendation below is built from. Don't ask the user to hand-transcribe lscpu/lsblk output when this exists.

Step 2: get the workload's expected shape

Ask, in one batch: expected record size (bytes), records per batch, sustained records/sec (or a target ingestion rate in MB/s directly), how many Storage Nodes the deployment should run, how many NVMe drives are active simultaneously per SN, and the expected batch-complete notify RTT if known (default 50ms is reasonable for a same-datacenter deployment).

Step 3: run the real sizing calculator

tests/sandbox_dms/fermihdi_sizing.py is the actual, non-interactive sizing engine — use it rather than hand-deriving the formulas below:

python3 fermihdi_sizing.py --type both \
  --record-size 150 --batch-records 50000 --records-per-sec 1000 \
  --sn-instances 1 --sn-active-drives 1 --scc-num-sns 3 --notify-rtt-ms 50

It reports cores, RAM (rounded to standard DIMM sizes), hugepage counts, and NIC bandwidth per role, and flags two things worth surfacing to the user directly: when the out-of-band hash table would exceed 4GB (suggests a streaming hash variant instead), and when the record size forces multipart framing (jumbo frames would help). --help lists every flag; run --type sn, --type scc, or --type both depending on what's being sized.

What it computes, for context when explaining the numbers: SN cores ≈ 1 (host) + instances × (6 for ingest + 3× active drives for query); SN RAM ≈ 4GB host + per instance (8GB hugepages + 8 bytes × batch-records for the out-record-hash table + 2GB write pool + 4GB OS/app). SCC cores ≈ 7 base + 1 per SN; SCC RAM ≈ 3GB hugepages + 0.5GB × num-SNs for fanout rings + 2GB OS/app.

Step 4: hugepages — two different numbers, don't conflate them

• A quick eval-only default (what the sandbox uses): a flat 4×1GB + 512×2MB per instance (SN or SCC), regardless of workload. Fine for a demo, not sized to anything real.
• The real production formula, from a worked cluster.yaml example: 2MB pages — SCC 300, SN 4140 (this one scales with configuration, not flat — sn_hot_path sizes its engine budget by transmit-shard count), BloomRF 300, ingester 300, GraphQL 300, Proxy/DMS 0. 1GB pages — 4 per NVMe device (SPDK DMA) + 2 per SCC; roles with no SCC and no NVMe need none. At the host level, when several NUMA nodes each run instances, the 1GB total is the hungriest node's requirement × the number of nodes, not a flat sum across nodes.

Use fermihdi_sizing.py's output (Step 3) as the authoritative number for a real deployment; the flat default above is only for a quick "will this host even boot the sandbox" check.

Step 5: NUMA and CPU placement

Hard rule: an SN or SCC instance's cores must all come from one NUMA node — never straddle. On a multi-NUMA host, decide confine (put everything on the node with the most drives — best memory locality, fewer usable cores) or spread (distribute SNs across nodes — more total cores, some cross-node traffic). DPDK's interrupt thread hardcodes affinity to physical CPUs 0/1 on top of whatever's allocated to services — the launcher always strips CPU 0/1 out of the isolated set before assigning it, so budget for that when counting available cores.

Quick host-viability floor (non-production, but a fast sanity check before running the real sizing math): OS 2 physical cores, SCC 10, SN 9 each, ingester 3, DMS 2, everything else 2. Real per-service CPU isolation needs roughly 28/37/46 physical cores for 1/2/3 SNs respectively; below that, isolation falls back to a shared, unpinned pool rather than refusing to start.

Step 6: NIC and jumbo frames

fermihdi_sizing.py's own analysis (part of the SN sizing output) states directly whether jumbo frames would help for the given record size and warns when the record size forces multipart framing — use its answer rather than a rule of thumb. A dedicated DPDK data-plane NIC only matters when the role is explicitly configured for it; POSIX sockets over the normal NIC are the default and are fine unless the sizing output says otherwise. Required bandwidth scales roughly with 2× the SN's inbound rate (headroom) and, on the SCC, the inbound rate × number of SNs (it fans out to all of them).

Step 7: BloomRF filter node — worth it or not

Fixed cost regardless of workload: 2 cores, 300×2MB hugepages, no 1GB pages, no storage devices. Memory scales with retained batches (512 by default) × bytes per batch. The actual decision rule: it earns its keep on selective predicates over many batches — an address, a port, a small value on a wide column, filtered across a wide time window. It does not help: predicates on the batch-id column itself (already the time window), != predicates, wide scans or aggregate-everything queries, or values present in nearly every batch. If the workload's queries don't look like the first pattern, recommend leaving it off or passive rather than defaulting it on.

Step 8: pick a topology

There's no documented host-count or throughput threshold beyond this table — the call from here is this skill's own judgment from the sizing numbers (does everything from Step 3 fit on one available host, does the customer already run k8s), not a cited platform rule. Say so if asked.

Step 9: write the plan

install/fermihdi-configure turns one host's probe facts into a cluster.yaml host block — interactive, or non-interactive from saved facts:

./fermihdi-configure --facts facts.yaml -o host.yaml
./fermihdi-configure --facts facts.yaml --append cluster.yaml   # add another host to an existing file

It changes nothing on the host — only writes the file. One run per host; concatenate (or --append) for a multi-host cluster.yaml.

Handoff

Hand the resulting cluster.yaml (or the single-host answers) to fermihdi-hd-install to actually provision, or fermihdi-hd-sandbox if the recommendation was a single-host eval deployment. Neither of those skills re-derives sizing — this one is where those numbers get decided.