# Studio storage upgrade

**Decision: build a separate NVMe render tier (option 2). Planning hardware total: $29,503.34, including ten 25GbE client connections and existing-NAS repairs. Verified installed purchase total: unverified.** Several component prices remain indexed rather than live-confirmed; do not purchase against this subtotal without requoting them.

Inspected **November 4, 2026** through read-only SSH/middleware, ZFS counters, DMI, PCIe and SMART attribute queries. No production fio, file read benchmark, test write, scrub, restart, configuration or application change. The temporary network alias was local and removed. Hardware identifiers, addresses and account names are omitted.

## Immediate finding: repair before scaling

The data pool is **DEGRADED**, not merely the known dead-drive condition: an additional 12TB member is FAULTED with **62 ZFS read errors**, **776 reallocated sectors** and **15 reported uncorrectable errors**. ZFS reports no known data errors. The boot pool now has **295 write errors and 1 read error**, versus the supplied baseline's 19 write errors. Its SMART test status is still PASSED; 24 SATA CRC errors also justify checking cabling/controller connectivity. SMART PASSED does not override ZFS faults.

The old dead 12TB drive remains failed and a spare is already INUSE; that alone is pre-existing, not a new alert. Another original member has 752 reallocated sectors versus the baseline's 696; a third has 624. Device letters are not permanent identities, so those baseline comparisons are provisional without serial matching. Budget three 18TB replacements: faulted member first, then the two high-reallocation members, one completed resilver at a time. Restore an idle spare after completing the existing spare replacement. Replace/diagnose boot storage and establish a mirrored boot pair. These are deployment instructions, **not actions performed**.

## Observed hardware and storage

| Item | Read-only observation |
|---|---|
| CPU / board | AMD Ryzen 5 3600, 6 cores / 12 threads; MSI B450 TOMAHAWK MAX |
| RAM | 64GiB installed, 62.73GiB OS-visible; 4 × 16GB G.Skill F4-3200C16-16GVK DDR4-3200 non-ECC UDIMM; all DIMM slots occupied |
| OS | TrueNAS SCALE **24.04.0** |
| HDD inventory | 9 × Seagate ST12000NM0127 12TB; 2 × ST18000VN000-3J3101 18TB; 1 × ST14000NM0408-2H7131 14TB. Includes dead drive and in-use spare |
| Data layout | **One 10-position RAIDZ2 vdev**; an in-use 18TB spare sits alongside its original 12TB member in one logical position. Ten effective columns remain approximately 12TB each; larger replacements do not presently expand the vdev |
| Pool capacity | zpool physical-accounting size **109.125TiB**, allocated **61.94TiB**, free **47.19TiB**, 56% allocated. These are not parity-adjusted user space |
| Dataset capacity | zfs reports **49.83TiB used**, **34.68TiB available** at the data-pool root. Used includes descendants/snapshots and is not equivalent to the supplied ~34TB Nextcloud-project estimate |
| Boot / SLOG | Single PNY 2TB SATA boot SSD, degraded; separate single PNY CS900 1TB SATA **SLOG**. No L2ARC; no special vdev |
| HBA / PCIe | LSI 9207-8i / SAS2308 at PCIe 3.0 x8. Existing X540 NIC negotiates **PCIe 2.0 x4** |
| Network | Intel X540-AT2 dual 10GBASE-T: one port actually **1,000Mb/s**, full duplex, MTU 9000; other down. Onboard Realtek gigabit port down. No 25GbE NIC |
| Shares | SMB running, **3 enabled read-write shares**; NFS disabled/stopped, **0 exports**; iSCSI stopped |
| ARC | **41.32GiB**; lifetime hit ratio **99.764%**, short observation **99.744%** (1,557 hits / 4 misses). Historical hits are predominantly metadata; not a scene-load benchmark |

**Sequential-read capability: unverified.** Passive `zpool iostat` showed zero data-pool read bytes/s in three one-second intervals. Its first, since-import mixed-workload average was 20.86MB/s; that is neither sequential speed nor a throughput ceiling. No production benchmark was substituted. The observed 1GbE link alone caps payload below its calculated 0.125GB/s line rate.

## Architecture decision

Use an **ordinary all-NVMe pool**, not cache or a special allocation class: **4 × 7.68TB Solidigm D7-P5520**, two striped two-way mirror vdevs. Nominal mirrored capacity is **15.36TB / 13.97TiB** before ZFS overhead. Limit combined live data and retained snapshots to about **12.29TB**; this is a selected active-project tier, not a migration of the entire archive. Enterprise drives provide power-loss protection; individual vendor speed figures are not ZFS/application measurements.[21]

Do **not** buy a special vdev plus L2ARC for the existing HDD pool. A special vdev holds the only copies of selected blocks; losing it can lose the pool, and RAIDZ constrains removal. It does not automatically migrate old metadata/data. Matching RAIDZ2's two-device-loss tolerance would require at least a three-way special mirror, not a two-way mirror.[17] L2ARC is expendable but consumes ARC-header RAM and helps repeated cacheable reads, not a guaranteed first read of a fresh scene; ordinary streaming data also depends on cache eligibility and fill policy.[18] Increasing `special_small_blocks` until EXR/model records land there merely creates a poorly bounded flash allocation tier. No separate SLOG is needed on the new mirrored PLP flash pool; SLOG accelerates synchronous-write acknowledgement, not scene reads.[18]

Install **256GB ECC RAM** with an initial dedicated-tier ARC ceiling of **192GiB**, leaving room for services; on a shared upgraded box initially reserve more for apps and cap ARC at **160GiB**. These are proposed deployment settings, not inspected values. Sixteen CPU cores are chosen for filesystem, checksum and protocol work, not rendering. Use the server board's native PCIe slots; four direct U.2 carrier cards plus a 100GbE NIC consume five slots, six when retaining the existing HBA. No PCIe bifurcation, SAS-to-NVMe compatibility assumption or undocumented NVMe backplane is required. The board supplies seven PCIe slots and eight ECC DIMM channels.[1]

Use **100GbE aggregate uplink** from storage to the switch and **25GbE to every client**. One 25GbE storage uplink would remain the same approximately 2.5GB/s practical bottleneck shared by everyone. The 16 × 25GbE / 2 × 100GbE switch accommodates the ten budgeted clients, one old-NAS replication link and growth.[22] Start with MTU 1500; change to jumbo only after verifying the entire path. Keep management separate. No RDMA or LACP dependency.

## Two builds and exact planning BOM

**Every listed price was seen November 4, 2026, in USD.** V = fetched live page or manufacturer JSON; U = **unverified current offer**, observed only through an indexed retailer listing or quote-only page. Live checkout, warranty eligibility and delivery are not guaranteed. Manufacturer suggested price is labeled in JSON. Taxes, shipping, labor, UPS, spare NVMe and an independent offsite-backup target are excluded and unverified. Published source URLs are below and repeated per part in JSON.

### Platform shared by both builds

| Part | Qty | Unit USD | Extended USD | Evidence |
|---|---:|---:|---:|---|
| Supermicro ATX SP3 motherboard — `MBD-H12SSL-I-O` | 1 | $649.99 | $649.99 | U [2] |
| AMD EPYC 7302P, 16 cores, vendor unlocked, new bulk — `100-000000049 / EPYC 7302P` | 1 | $1,012.50 | $1,012.50 | V [3] |
| NEMIX 256GB, 8 x 32GB DDR4-3200 ECC RDIMM — `MR25600-324K08-G` | 1 | $3,426.99 | $3,426.99 | V [4] |
| Solidigm D7-P5520 7.68TB U.2 NVMe — `SSDPF2KX076T11Z` | 4 | $3,950.00 | $15,800.00 | U [5] |
| StarTech U.2 drive-on-card, PCIe 3.0 x4 — `PEX4SFF8639` | 4 | $27.99 | $111.96 | U [6] |
| NVIDIA ConnectX-5, single 100GbE QSFP28, PCIe 3.0 x16 — `MCX515A-CCAT` | 1 | $749.00 | $749.00 | V [7] |
| Noctua SP3 dual-92mm tower cooler — `NH-U9 TR4-SP3` | 1 | $109.99 | $109.99 | U [8] |
| Noctua 140mm PWM fan — `NF-A14 PWM` | 3 | $24.95 | $74.85 | U [11] |

**Platform subtotal: $21,935.28.** Four carrier cards include the U.2 sockets; the SSDs mount directly onto cards. Add three PWM fans to direct airflow across SSDs and NIC. No U.2 data cables are required. Carrier PCIe 3.0 x4 bandwidth limits each otherwise Gen4 SSD; aggregate performance, not the drives' headline speed, is the planning basis.[6]

### Shared fabric: maximum planned fleet

| Part | Qty | Unit USD | Extended USD | Evidence |
|---|---:|---:|---:|---|
| MikroTik 16 x 25GbE + 2 x 100GbE switch — `CRS518-16XS-2XQ-RM` | 1 | $1,325.00 | $1,325.00 | V [14] |
| NVIDIA ConnectX-4 Lx dual 25GbE, PCIe 3.0 x8 — `MCX4121A-ACAT` | 10 | $259.00 | $2,590.00 | V [15] |
| MikroTik 3m SFP28 25GbE DAC — `XS+DA0003` | 10 | $58.00 | $580.00 | V [14] |
| MikroTik 1m QSFP28 100GbE DAC — `XQ+DA0001` | 1 | $39.00 | $39.00 | V [16] |

**Fabric subtotal: $4,534.00.** Count assumes **4 GPU nodes + 4 CPU nodes + 2 workstations**. Each gets one 25GbE connection; second NIC ports are unused, not double-counted. Cabling assumes clients within 3m of the switch and storage within 1m; actual room distances and endpoint free PCIe slots are **unverified**. Longer workstation runs need a separately quoted optical link, not an assertion that short DACs will reach.

### Existing-NAS repairs required in either build

| Part | Qty | Unit USD | Extended USD | Evidence |
|---|---:|---:|---:|---|
| Seagate IronWolf Pro 18TB CMR SATA repair disk — `ST18000NT001` | 3 | $435.78 | $1,307.34 | V [13] |
| Samsung 870 EVO 500GB SATA boot SSD — `MZ-77E500B/AM` | 2 | $249.99 | $499.98 | U [12] |

**Repair subtotal: $1,807.32.** HDD replacements are not a capacity expansion. The boot pair replaces the current single boot device; remove the already-failed unpartitioned HDD during physical maintenance to free a SATA port for the second boot SSD. Data-pool SLOG changes require a separate planned maintenance decision, not changes made during this audit.

### Option 1 — upgrade the existing box

Replace its motherboard, CPU and RAM with the platform above, repair HDD/boot storage, and add the independent mirrored NVMe pool. Reuse chassis, PSU, SATA wiring and HBA. The existing B450 platform's occupied slots, RAM capacity and observed NIC bus restriction make it a poor foundation for a 100GbE/flash retrofit; this option therefore explicitly replaces the platform rather than pretending a NIC and cache solve it. MSI's indexed board specification is supporting context; live access to its page was blocked.[23]

**Total: $28,276.60.** Shared Nextcloud/apps, HDD replication and flash serving still share an OS, PSU and failure domain. Existing PSU rating, available power connectors, chassis airflow and 125mm CPU-cooler clearance are **unverified**; do not buy until checked. Plan **6GB/s aggregate reads**, reserve **3GB/s** for two artists, leaving **3GB/s** for renders. Throughput is an unverified engineering budget. This is not the recommendation.

### Option 2 — separate NVMe render tier — recommended

Use the same platform in the following new chassis/power configuration. Keep the old NAS's CPU/RAM/HBA intact, repair its disks/boot pair, and add one 25GbE adapter for replication. Its existing PCIe 2.0 x4 slot prevents full 25GbE throughput, but does not need to match the hot tier's serving bandwidth. The boot SSD pair below is for the **new** server; the earlier repair pair remains in the old NAS.

| Part | Qty | Unit USD | Extended USD | Evidence |
|---|---:|---:|---:|---|
| Fractal Define 7 XL solid black ATX tower — `FD-C-DEF7X-01` | 1 | $294.79 | $294.79 | V [9] |
| Corsair RM850e 2025, 850W modular PSU — `CP-9020296-NA` | 1 | $114.97 | $114.97 | U [10] |
| Samsung 870 EVO 500GB SATA boot SSD — `MZ-77E500B/AM` | 2 | $249.99 | $499.98 | U [12] |
| NVIDIA ConnectX-4 Lx dual 25GbE, PCIe 3.0 x8 — `MCX4121A-ACAT` | 1 | $259.00 | $259.00 | V [15] |
| MikroTik 3m SFP28 25GbE DAC — `XS+DA0003` | 1 | $58.00 | $58.00 | V [14] |

**Additional subtotal: $1,226.74. Total: $29,503.34.** The extra cost over option 1 buys an independent render-storage failure/maintenance domain. Use only cables supplied with the new modular PSU. Expected planning budget: **8GB/s aggregate file reads**, **2.5GB/s per 25GbE client**, with **3GB/s reserved for artists** and **5GB/s for scene loads**. All throughput remains **unverified** until tested on the completed build. Do not treat this as redundant/HA storage: one chassis/PSU/NIC failure interrupts rendering, even though mirror copies protect against some SSD failures.

## Concurrent scene loads and comp budget

Assume decimal **30GB transferred per render node**, no compression savings, fair sharing, client capacity 2.5GB/s. Two 24fps EXR streams at 60MB/frame require **2.88GB/s**, hence the 3GB/s artist reservation. At 20MB/frame those two streams require **0.96GB/s**. ProRes and model transfers consume the same budget; stage 10–60GB model weights and immutable scene bundles to node-local NVMe once per version rather than rereading on every task. Client disks must sustain staging writes; speed is unverified.

| Render nodes loading together | Total scene bytes | Option 1, artists active | Option 2, artists active |
|---:|---:|---:|---:|
| 1 | 30GB | 12s | 12s |
| 2 | 60GB | 20s | 12s |
| 4 | 120GB | 40s | 24s |
| 8 | 240GB | 80s | 48s |

Calculated transfer time is `max(30 / 2.5, nodes × 30 / remaining_GB_per_second)`. These are steady-throughput planning times, **not measured render-ready times**. USD composition, texture opens, EXR decode, CPU setup and VRAM uploads add time. A freshly published scene does not benefit from a previous warm-cache result. Distinct 30GB scene versions should be used for acceptance testing so repeated reads do not hide pool limits.

## Protocol, publishing and backup rules

- **Linux renders: NFSv4.1 over TCP. Workstations: SMB3.** This is a workflow choice, not a claim that NFS is inherently faster. Benchmark real clients; keep inputs immutable and NFS exports read-only. Use consistent identity mapping and a restricted storage network. Put writable render outputs on separate NFS datasets; ingest completed outputs into SMB artist work areas. Avoid concurrent cross-protocol writes to the same files. TrueNAS explicitly warns of multiprotocol conflicts/corruption; do not apply newer documentation's setup screens blindly to installed 24.04.0.[19]
- For new bulk datasets use **1MiB recordsize, lz4, atime off, sync standard, dedup off**; initially retain 128KiB for small/metadata-heavy assets. These are proposed settings. Never disable sync as a benchmark shortcut. Read-publish versioned assets only after a manifest/checksum validation; keep stable render paths independent of protocol-specific drive mappings.
- **Nextcloud remains sync/delivery, not render storage.** Do not mount its WebDAV/sync tree as the render data plane or directly mutate its managed storage. Export/import through its normal client/API flow; external changes otherwise need explicit discovery/scanning.[20]
- After repair, use the HDD NAS as the **hourly snapshot-replication target**: retain 48 hourly, 30 daily and 8 weekly versions, under a **12TB combined data + retained-snapshot quota**. Its observed available space is sufficient for this quota, not proof of unlimited retention or a full second copy of the existing archive. Plan ingestion and nightly catch-up around HDD contention; replication throughput is unverified. Never delete the last source version until an actual restore succeeds.
- Keep at least daily **independent offline/offsite copies** of irreplaceable assets and matching Nextcloud database/data. Same-chassis replication in option 1 does not survive host loss. Option 2's second host still does not protect against studio-wide disaster. Offsite target, present backup health, capacity and cost are **unverified**, excluded from the quoted subtotal, and required before production cutover.

## Acceptance gate

Repair and confirm data/boot-pool health first. Build on a supported stable TrueNAS release chosen and migration-tested at deployment; installed 24.04.0 is only the observed version, not a recommendation. Verify NIC drivers, link/PCIe widths, thermal behavior, identity/ACL rules, power and cabling. Test the new isolated hot tier against **8GB/s aggregate and 2.5GB/s per-client targets**, then two 24fps EXR streams alongside four/eight cold distinct-scene loads. Check tail latency and CPU use, not just averages. Verify a snapshot restore plus matched Nextcloud database/data restore and the independent backup. **None of these performance/deployment tests ran against production.**

## Sources

[1] https://www.supermicro.com/en/products/motherboard/h12ssl-i
[2] https://www.newegg.com/p/pl?d=supermicro%20h12ssl
[3] https://www.itcreations.com/product/116372
[4] https://nemixram.com/products/ddr4-3200mhz-pc4-25600-rdimm-2rx4.js
[5] https://serversupply.com/SSD/PCI-E/7.68TB/SOLIDIGM/SSDPF2KX076T11Z_395356.htm
[6] https://www.newegg.com/startech-com-model-pex4sff8639-u-2-to-pcie-adapter-x4-pcie/p/N82E16815158437
[7] https://qsfp28optics.com/mcx515a-ccat
[8] https://www.newegg.com/noctua-nh-u9-tr4-sp3-premium-grade-92mm-cpu-cooler-for-amd-tr4-sp3/p/13C-0005-00142
[9] https://www.avadirect.com/Define-7-XL-No-PSU-E-ATX-Black-Full-Tower-Case/Product/13263226
[10] https://www.newegg.com/corsair-rmx-series-atx-3-1-compatible-850-w-cybenetics-gold-power-supply-black-rm850e/p/N82E16817139336
[11] https://amazon.com/Noctua-NF-A14-PWM-Premium-Cooling/dp/B00CP6QLY6
[12] https://www.bhphotovideo.com/c/product/1606519-REG/samsung_mz_77e500b_am_500gb_870_evo_sata.html
[13] https://telecomcreations.com/collections/networked-storage/products/st18000nt001-18tb-ironwolf-pro-enterprise-seagate-nas-single
[14] https://shop.linktechs.net/crs518-16xs-2xq-rm-2
[15] https://qsfp28optics.com/MCX4121A-ACAT
[16] https://mikrotik.com/product/xq_da0001
[17] https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Pool%20Structure/Special%20vdev.html
[18] https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Pool%20Structure/Caching.html
[19] https://www.truenas.com/docs/scale/shares/mixedmodeshares
[20] https://docs.nextcloud.com/server/stable/admin_manual/configuration_files/external_storage_configuration_gui.html
[21] https://solidigmtechnology.jp/content/dam/solidigm/en/site/products/data-center/d7/p5520/documents/Solidigm_D7_P5520_P5620.pdf
[22] https://mikrotik.com/product/crs518_16xs_2xq
[23] https://www.msi.com/Motherboard/b450-tomahawk-max/Specification