C-72T, C-168T, C-192T, C-432T, and K-432T Networking

IN THIS ARTICLE

This article outlines the various ways in which you can connect a C-72T, C-168T, C-192T, C-432T, or K-432T cluster to your network.

REQUIREMENTS

• A network switch that meets the following criteria:
  
  • C-72T and C-168T: 25 Gbps Ethernet
  • K-432T: 25 Gbps Ethernet
  • C-192T and C-432T: 100 Gbps Ethernet
  • Fully non-blocking architecture
  • IPv6 capable
• Compatible network cables
• Enough ports to connect all nodes to the same switch fabric
• One static IP per node per defined VLAN

RECOMMENDATIONS

• One set of redundant switches
  • Jumbo Frame support with a minimum of 9000 MTU 
• One physical connection per node to each redundant switch
• One LACP port-channel on each node with the following configuration:
  • Active mode
  • Slow transmit rate
  • Trunk port with a native VLAN
• DNS servers
• Time server (NTP)
• Firewall protocol/ports allowed for Proactive Monitoring
• N-1 (N=number of nodes) floating IPs per node per client-facing VLAN
  Note: The number of floating IPs depends on your workflow and clients connecting to the cluster, with a minimum of 2 floating IPs per node per client-facing VLAN, but no more than 10 floating IPs per node per client-facing VLAN.
• You can achieve advertised performance only if you connect your nodes at their maximum Ethernet speed. To avoid network bottlenecks, Qumulo validates system performance with this configuration by using clients that are connected at the same link speed and to the same switch as the nodes.

DETAILS

The Qumulo C-series platform uses a networking configuration where both back end and front end traffic are handled by the same NIC.

C-72T and C-168T

C-192T, C-432T, and K-432T

Notes:

• Certain node configurations use different NICs for design (rather than performance) reasons. For more information, see Identifying NIC and Choosing Transceivers and Cables for Your Nodes on the Documentation Portal.
• The eth port labels vary depending on the NIC manufacturer.

  NIC Manufacturer	Port Location	Port Labels
  Mellanox	Top	1 (eth3)
  Mellanox	Bottom	2 (eth4)
  Broadcom	Top	1 (eth0)
  Broadcom	Bottom	2 (eth1)

For reliability, the recommended configuration is fully-cabled where both ports on each node should be connected. Connecting a single port on the NIC is not recommended, because if the single connection fails, the node will be unavailable. 

Caution: Don't use the LAN On Motherboard (LOM) ports. Only use the external NIC ports (typically eth0 and eth1) for the connections, as highlighted above. 

Connect to Redundant Switches 

The details below outline how to connect a 4 node Qumulo C-series cluster to dual switches for redundancy. This is the recommended configuration for Qumulo C-series hardware. If either switch goes down, the cluster will still be accessible from the remaining switch.

• The two NIC ports on the nodes are connected to separate switches
• Uplinks to the client network should equal the bandwidth from the cluster to the switch
• The two ports form an LACP port channel via multi-chassis link aggregation group

Connect to a Single Switch

The details below outline how to connect a 4 node Qumulo C-series cluster to a single switch. Note if this switch goes down, the cluster will not be accessible.

• Each node contains two ports that are connected to the switch
• Uplinks to the client network should equal the bandwidth from the cluster to the switch
• The two ports form an LACP port channel 

Tip: For IPMI configuration details such as port location and configuration, see the IPMI Reference Guide for Qumulo C-Series.

ADDITIONAL RESOURCES

Getting Started Guide: Qumulo C-Series 1U

Getting Started Guide: Qumulo C-Series 2U

IPMI Reference Guide for Qumulo C-Series

QQ CLI: Networks and IP Addresses