- Joined
- Mar 14, 2023
- Messages
- 97,152
- Thread Author
-
- #2
Microsoft has officially authorized a supported mixed storage topology that lets Storage Spaces Direct (S2D) and traditional SAN storage operate side‑by‑side inside the same Windows Server Failover Cluster (WSFC), giving enterprises a practical bridge between existing SAN investments and modern hyperconverged performance. This capability—documented by Microsoft and amplified in industry coverage—applies to Windows Server 2022 and is carried forward into Windows Server 2025, and it enforces strict separation between the two storage domains while enabling administrators to place workloads where they perform best.
Storage strategy in enterprise datacenters has long been a tradeoff. Traditional SANs deliver mature enterprise features—snapshots, array replication, centralized management and predictable operational models—while Storage Spaces Direct (S2D) delivers cost‑efficient, low‑latency, node‑local pooling across NVMe/SSD/HDD with ReFS optimizations that suit high‑IOPS modern workloads. Historically, these two approaches required distinct clusters or a rip‑and‑replace approach for migration to hyperconverged infrastructure. Microsoft’s mixed topology formalizes a third option: run both on one WSFC and use each technology for the workloads it fits best. The net result: fewer forklift upgrades, more gradual migrations, and the ability to keep SAN‑based workflows (and SLAs) while adopting S2D for latency‑sensitive workloads such as large VM farms, SQL/DB workloads, container layers, and AI/ML nodes. This is a pragmatic, brownfield‑friendly approach that lowers the barrier for organizations wanting to modernize storage without discarding existing investments.
Source: Petri IT Knowledgebase Windows Server Failover Clusters Get S2D and SAN Coexistence Support
Storage strategy in enterprise datacenters has long been a tradeoff. Traditional SANs deliver mature enterprise features—snapshots, array replication, centralized management and predictable operational models—while Storage Spaces Direct (S2D) delivers cost‑efficient, low‑latency, node‑local pooling across NVMe/SSD/HDD with ReFS optimizations that suit high‑IOPS modern workloads. Historically, these two approaches required distinct clusters or a rip‑and‑replace approach for migration to hyperconverged infrastructure. Microsoft’s mixed topology formalizes a third option: run both on one WSFC and use each technology for the workloads it fits best. The net result: fewer forklift upgrades, more gradual migrations, and the ability to keep SAN‑based workflows (and SLAs) while adopting S2D for latency‑sensitive workloads such as large VM farms, SQL/DB workloads, container layers, and AI/ML nodes. This is a pragmatic, brownfield‑friendly approach that lowers the barrier for organizations wanting to modernize storage without discarding existing investments.Overview: what Microsoft now supports
Microsoft’s documentation defines a “hyperconverged with SAN storage” model: S2D‑backed volumes (formatted as ReFS and exposed as CSVs) coexist with SAN‑LUN–backed CSVs (formatted as NTFS) on the same cluster. The two storage domains operate side‑by‑side but remain logically separate—SAN LUNs are not part of the S2D pool and must never be added to it. Supported SAN connectivity includes Fibre Channel, iSCSI, and supported iSCSI targets. The guidance applies to Windows Server 2022 and Windows Server 2025 (and corresponding Azure Local builds). Key platform rules enforced by Microsoft:- S2D is DAS‑only: S2D will only aggregate locally attached drives discovered by cluster nodes; it does not and must not claim SAN LUNs.
- Filesystem separation: S2D volumes should be formatted with ReFS (CSVFS_ReFS); SAN volumes that will be CSVs must be formatted NTFS before conversion to CSV. This aligns with how S2D and CSV code paths are optimized.
- Strict operational separation: SAN disks cannot be added to the S2D storage pool; administrators must maintain independent management and observability for each storage subsystem.
Technical fundamentals: S2D vs SAN (concise)
What Storage Spaces Direct (S2D) gives you
- Software‑defined pooling of local disks across cluster nodes.
- Native resiliency (mirroring, nested resiliency, erasure coding) and automatic rebuilds across nodes.
- Strong performance when paired with NVMe and RDMA networks (SMB Direct), and default ReFS optimizations for large virtualization workloads.
What SAN storage gives you
- Centralized, vendor‑managed arrays with mature features: snapshots, replication, deduplication, encryption and advanced telemetry.
- Predictable operational model supported by vendor tooling and enterprise SLAs, especially for archival, long‑term retention, and regulated datasets.
Why mix them?
A mixed cluster lets you:- Place latency sensitive VMs and high‑IOPS workloads on S2D/ReFS volumes.
- Keep large archival workloads, compliance‑sensitive data, or array‑managed features on SAN/NTFS volumes.
- Migrate VMs progressively using storage live migration tools rather than performing a forklift migration.
Important configuration guidelines and hard constraints
Administrators must follow precise rules to keep the cluster supported and reliable. These are the operational must‑dos:- Never add SAN‑presented disks to an S2D storage pool. Mixing physical origins in a pool is unsupported and can cause unrecoverable states.
- Format S2D volumes as ReFS (CSVFS_ReFS) before converting to CSV. This ensures S2D’s performance and resiliency features are available.
- Format SAN volumes as NTFS prior to converting them to CSVs. NTFS CSVs use different code paths and semantics than ReFS CSVs.
- Keep firmware, drivers, and vendor software versions consistent and validated against Microsoft’s compatibility lists and vendor guidance. Multipath I/O (MPIO) needs to be correctly configured for SAN paths.
- Segment and protect network traffic: dedicate physical/network paths or VLAN/QoS for S2D east‑west replication (SMB Direct / RDMA if used) and separate SAN traffic to avoid contention.
Practical migration scenarios and a recommended playbook
The mixed topology shines when used as a migration tool or tiered‑storage strategy. Below is a recommended phased playbook—execute it in a lab and iterate.- Inventory and classify VMs and data by I/O profile, latency sensitivity, and filesystem requirements. Tag workloads that need ReFS features vs those that depend on SAN features.
- Validate hardware and firmware versions; obtain Microsoft‑validated compatibility matrices for host SKUs, HBAs, SAN arrays and MPIO drivers. Confirm support for your OS build (Windows Server 2022 or 2025).
- Build a test cluster that mirrors the production PHY/NIC, RDMA, and SAN zoning to verify failovers, live migrations, and stress behavior. Run Test‑Cluster and S2D validation tests.
- Prepare destination CSVs: create S2D volumes formatted as CSVFS_ReFS and SAN LUNs formatted as NTFS, then add them to the cluster as CSVs.
- Backup and snapshot before migration. Use array snapshots for SAN‑resident VMs and application‑consistent backups for S2D residents to ensure rollback options.
- Pilot migrations: use Move‑VMStorage or Move‑VirtualMachineStorage for Hyper‑V workloads; monitor IOPS, CPU, and CSV latency during test runs. Validate application behavior.
- Scale migrations in controlled batches, automate via PowerShell or orchestration tools, and maintain throttling to avoid saturating fabrics.
Network design and performance considerations
S2D is network‑sensitive. Latency and throughput directly affect rebuild times and VM performance.- Use dedicated low‑latency networks for S2D replication. Microsoft recommends 10+ GbE with RDMA (RoCE or iWARP) when available—enabling SMB Direct greatly reduces CPU overhead.
- Isolate SAN fabric traffic (Fibre Channel or iSCSI) so heavy SAN I/O cannot interfere with S2D east‑west traffic. Plan NIC/VLANs and QoS accordingly.
- Monitor SMB, NIC, and CSV latency counters closely. Synthetic IO testing during pilot migration will expose contention windows.
Observability, monitoring and runbooks
Mixed clusters increase operational surface area. Treat monitoring as a first‑class requirement.- Monitor each storage domain with vendor tools (array telemetry, snapshot/replication health) and Windows S2D health counters (resiliency, pool usage, rebuild progress). Correlate alerts across systems.
- Instrument cluster CSV owner node metrics and SMB performance counters to detect cross‑domain contention. Automate synthetic IO tests during scheduled windows.
- Build provisioning guards: preflight checks that enforce filesystem type (ReFS for S2D, NTFS for SAN CSVs) and prevent SAN LUNs from being added to S2D pools via automation.
Backup, snapshots and ransomware strategies
A mixed cluster offers opportunities for layered protection:- Use SAN‑array snapshots/replication as an immutable/array‑level recovery tier for VMs that were migrated from S2D to SAN CSVs or that are persistently kept on SAN. This can provide very fast RTOs and array‑native replication capabilities.
- For S2D volumes (ReFS), integrate Microsoft and third‑party backup tooling that understands CSVs and ReFS behaviors; keep backups off‑cluster to remove single‑location risk.
- Plan ransomware responses that include isolated snapshot retention and tested recovery playbooks across both storage domains; mixing technologies helps diversify recovery targets.
Vendor ecosystem, Azure Local and partner activity
The mixed topology has triggered partner responses: storage vendors and cloud‑adjacent offerings (Azure Local) are publishing integration patterns and previews showing SAN attach and validated topologies. These vendor announcements indicate the approach is being operationalized, but the authoritative support boundary remains Microsoft’s compatibility and validation documentation—request validated compatibility matrices from vendors before procurement. Enterprises should treat vendor press as signals of momentum but base production acceptance on joint vendor‑Microsoft validation certificates that list firmware/driver/host combos. This prevents finger‑pointing in complex post‑migration incidents.Strengths, risks, and what to watch for (critical analysis)
Strengths- Practical modernization path: Enables progressive adoption of S2D while preserving SAN investments and vendor workflows.
- Workload optimization: Place high‑IOPS workloads on S2D/ReFS and retention/compliance workloads on SAN/NTFS to balance performance and operational needs.
- Cost and operational flexibility: Potential CAPEX/OPEX savings by reusing SAN capacity and rightsizing hyperconverged nodes for compute.
- Misconfiguration hazards: Accidentally adding SAN disks to S2D pools or mixing filesystems creates unsupported states and potential data loss. Strict automation and provisioning checks are mandatory.
- Performance interference: SAN I/O can congest fabrics and affect S2D replication; network isolation and QoS are non‑optional.
- Operational complexity: Two storage models require cross‑team coordination, synchronized firmware lifecycles, and more sophisticated monitoring.
- Upgrade and functional level pitfalls: Rolling OS or cluster functional upgrades have corner cases; always validate upgrade paths and test rollbacks. Community reports note edge cases during certain in‑place upgrades.
- Any marketing statements promising specific reduction ratios, universal performance gains, or “turnkey” migration speeds should be treated cautiously until proven in a representative PoC. Ask vendors for dataset‑specific validation and contractual guarantees where possible.
Recommended checklist for planning and execution
- Obtain Microsoft’s current documentation for “Hyperconverged with SAN storage” and read the formatting and separation rules.
- Collect validated hardware/firmware matrices from storage vendors and host OEMs.
- Build a test cluster and run validation tests (Test‑Cluster, S2D validation).
- Automate provisioning guards to enforce filesystem types and prevent SAN disks entering S2D pools.
- Design isolated network fabrics for S2D replication and SAN traffic; include QoS and RDMA where appropriate.
- Pilot migrations with representative workloads, instrumenting CPU, SMB/CSV latency, and array telemetry.
- Update runbooks, monitoring, and backup/DR procedures to manage both storage domains.
Conclusion
Microsoft’s decision to support S2D and SAN coexistence inside a single Windows Server Failover Cluster removes a major operational and financial blocker for many enterprises. When applied with discipline—strict filesystem separation, careful network design, validated hardware matrices, and a staged migration plan—the mixed topology delivers a pragmatic path to modernize without abandoning mission‑critical SAN investments. The feature is a powerful tool for tiered storage strategies, phased migrations, and hybrid cloud patterns, but it increases operational surface area and requires rigorous validation, automation, and cross‑team discipline to avoid unsupported configurations. For datacenter architects, the immediate next steps are clear: validate with vendors, pilot in an isolated lab, harden provisioning automations, and update runbooks to manage the new mixed reality of storage on Windows Server.Source: Petri IT Knowledgebase Windows Server Failover Clusters Get S2D and SAN Coexistence Support
Similar threads
- Featured
- Article
- Featured
- Article
- Featured
- Article
- Article