HPe Synergy 12000 MPIO QSFP to 10GB SFP Setup

KB ID 0001495

Problem

I was recently involved in deploying an HPe Synergy 12000 Frame. And the network connections from it were ‘a little unusual’ so I thought I’d document that here, to save anyone else the problems I had.

I was connecting to an HP/Aruba 5412 switch so my cables were all HP/Aruba (to be on the safe side).

What you can see (above) is the MPIO Cable (K2Q46A P/N 800867-001) fixed onto the left (and above boxed,) there is a QSFP (P/N 817040-B21.) Note: this can be used either as 4 x 10Gbe or 4 x 8GbFC). On the right you can see the cable ends in 4x Standard LC fibre connectors, so you will also need 4x 10GB SR SFP+ Modules (Aruba P/N J1950D) – shown bottom right.

So what does it do? (Apart from cost a fortune!) Well the QSFP connects at 40Gb and splits the traffic down into 4 x 10Gb 

Cabling and Configuring MPIO QSFP

Connecting up is pretty straight forward, REMEMBER when you connect the 40GB QSFP to the Synergy it will light purple if its connected, and flash purple when it sees activity.

Connecting to the switch is also easy enough, (WARNING: All the ports need to be Trunked (HP) or Ether Channelled (Cisco,)) with LACP enabled. You don’t need to worry about configuring LACP on the Synergy, that’s handled automatically by the ‘Uplink set’.

So the back of the ‘Frame’ has two interconnect links (If you are from a switch background think of these like stacking cables). And two MPIO uplink cables.

HPe/Aruba Switch Config For MPIO

As previously stated, the switch I’m using is an Aruba 5412, with two 8 Port 1Gb/10Gb modules (J9993-A). Here’s the relevant switch config;

[box]

Firstly give the interfaces a sensible name;
!
interface A2
   name "Trunk Link to Synergy VC1 Port Q1"
   exit
interface A3
   name "Trunk Link to Synergy VC1 Port Q1"
   exit
interface A4
   name "Trunk Link to Synergy VC1 Port Q1"
   exit
interface A5
   name "Trunk Link to Synergy VC1 Port Q1"
   exit
!
interface B2
   name "Trunk Link to Synergy VC2 Port Q1"
   exit
interface B3
   name "Trunk Link to Synergy VC2 Port Q1"
   exit
interface B4
   name "Trunk Link to Synergy VC2 Port Q1"
   exit
interface B5
   name "Trunk Link to Synergy VC2 Port Q1"
   exit
!

Show any 'already configured' Trunk links with a 'show trunk' command
In my case two existed, (Trk1 and Trk2). So I used Trk3;

!
trunk A2-A5,B2-B5 Trk3 LACP
!

Now UNTAG vlan 1 (assuming that's your default VLAN) And TAG and VLANS that 
need to be used in the Synergy Deployment. (Note on an HP switch simply add
the Trk3 to the existing settings like so;

!
vlan1
untagged A6-A8,B6-B8,E1-E24,F1-F24,G3-G12,H3-H12,Trk1-Trk3
!
vlan 100
   tagged Trk1-Trk3
   exit
vlan 101
   tagged Trk1-Trk3
   exit
vlan 102
   tagged Trk1-Trk3
   exit
vlan 103
   tagged Trk1-Trk3
   exit
etc.

[/box]

Cisco Switch Config For MPIO

If you have a Cisco Switch then instead of ‘Trunking’ you will be ‘Ether Channelling’ for a more detailed explanation see the following post

HP and Cisco – VLANs and Trunks Confusion!

[box]

!
interface port-channel 1
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface range Te1/1 - 4 , Te2/1 - 4
switchport trunk encapsulation dot1q
switchport mode trunk
channel-protocol lacp
channel-group 1 mode active
!

[/box]

Configuring Synergy 12000 Networking

The process is, you add  Networks, then collect Networks together in Network Sets, Then you create Logical Interconnect  Groups. Part of creating a Logical Interconnect Groups, involves creating an Uplink Set, which consists of both your Networks, and the The Uplink ports.

Note: A Network Set is used by a Server Profile, (or a Server Profile Template).

Create Networks

One View > Networking > Networks > Create Network

Create Network Sets

One View > Networking > Networks Sets  > Create Network Set  > Give it a name > Add Networks > Create.

Create Logical Interconnect Group

One View > Networking > Logical Interconnect Group  > Create Logical Interconnect Group > Give it a name > Select the correct  Interconnect Bay Set (see diagram above) > Select Interconnects > Add Uplink Set.

Give the set a name > Select the Type > Add in the Networks > Add in the Uplinks > Create.

Note: You only need to add in ALL the LOGICAL interfaces i.e. Q1:1, Q1:2,Q1:3,Q1:4 for EACH Interconnect module. .

After a few minutes if you look under One View > Networking > Logical Interconnects > You will see one listed that has the name of your Logical Interconnect group (with a divide symbol on the end!) Make sure ALL the logical uplinks are connected. (If not you will see LACP errors on the switch).

 

Related Articles, References, Credits, or External Links

NA

Stacking (VSF) Aruba Switches

KB ID 0001492

I noticed some shiny Aruba switches on the bench today, they were for a job my colleague is working on. (Note: Each switch in a stack should be the same model, so these will need two stacks!)

I work on the occasional HP/Aruba core switch, but it’s been a while since I did any work on distribution switches like these. The first thing I learned, was there’s no dedicated stacking cable for them. They simply use a 10Gb (Twinax / DAC) cable. Which I suppose is pretty straight forward, but it means you lose an SFP+ port (which is a bit pants).*

*Note: You can stack with 1GB cables, but you can’t mix and match!

So I said “Give me a shoult when you stack them and I’ll take a nosey!”

Solution

In the ‘land of Aruba’ this is called creating a VSF (Virtual Switching Fabric). As you can see from the photo, these are 2930F Switches, and you can stack up to four switches in a VSF. The same stacking method is used on the 5400R (v3) and 5412, where you can link two 5400R or 5412’s).

Also this method is NOT to be confused with ‘Fabric Stacking’ which is available on the 2920,2930M,3800,3810M models, (that is more like Cisco FlexStack, with a dedicated 100Gb stack cable).

So, assuming you have your switch new and fresh, connect in with your console cable, and dedicate a port to use for VSF.

[box]

Aruba-2930F-24G-PoEP-4SFPP# conf t
Aruba-2930F-24G-PoEP-4SFPP(config)# vsf member 1 link 1 ethernet 25
All configuration on this port has been removed and port is placed in VSF mode.

[/box]

Then place the switch into a VSF domain

[box]

Aruba-2930F-24G-PoEP-4SFPP(config)# vsf enable domain 1
This will save the current configuration and reboot the switch.

[/box]

The switch will ask for a reboot, let it do so.

Repeat the procedure on the second switch, (but this will be member 2).

[box]

Aruba-2930F-24G-PoEP-4SFPP# conf t
Aruba-2930F-24G-PoEP-4SFPP(config)# vsf member 1 link 1 ethernet 25
All configuration on this port has been removed and port is placed in VSF mode.
Aruba-2930F-24G-PoEP-4SFPP(config)# vsf enable domain 1
This will save the current configuration and reboot the switch.

[/box]

Once again let the switch reboot. 

Post reboot you will see the ports are ‘re-numbered’ 1/{port-number} on vsf member 1, 2/{port-number} on vsf member 2 etc.

[box]

Aruba-2930F-24G-PoEP-4SFPP# show interfaces
Status and Counters - Port Counters

                                                                 Flow Bcast
  Port         Total Bytes    Total Frames   Errors Rx Drops Tx  Ctrl Limit
  ------------ -------------- -------------- --------- --------- ---- -----
  1/1          0              0              0         0         off  0    
  1/2          0              0              0         0         off  0    
  1/3          0              0              0         0         off  0    
  1/4          0              0              0         0         off  0    
<---------------Output Removed For The Sake Of Brevity-------------->   
  1/10         0              0              0         0         off  0    
  1/11         0              0              0         0         off  0    
  1/12         0              0              0         0         off  0    
  1/13         0              0              0         0         off  0  
<---------------Output Removed For The Sake Of Brevity--------------> 
  1/19         0              0              0         0         off  0    
  1/20         0              0              0         0         off  0    
  1/21         0              0              0         0         off  0       
  1/25         1,496,823,949  23,354,845     0         0         off  0
<---------------Output Removed For The Sake Of Brevity--------------> 
  2/1          0              0              0         0         off  0    
  2/2          0              0              0         0         off  0    
  2/3          0              0              0         0         off  0    
  2/4          0              0              0         0         off  0    
<---------------Output Removed For The Sake Of Brevity--------------> 
  2/22         0              0              0         0         off  0    
  2/23         0              0              0         0         off  0    
  2/24         0              0              0         0         off  0    
  2/25         1,536,016,322  23,966,915     0         0         off  0    
  2/26         0              0              0         0         off  0    
  2/27         0              0              0         0         off  0    
  2/28         0              0              0         0         off  0    
 

[/box]

If you need to Stack 3 or 4 Switches then you need to add a second link, and create a ring;

i.e.

  • Switch 2 (2nd link now to switch 3) vsf member 2 link 2 ethernet 26
  • Switch 3 (1st link to switch 2 ) vsf member 2 link 1 ethernet 25
  • Switch 3 (2nd link to switch 4 ) vsf member 2 link 2 ethernet 26
  • Switch 4 (1st link to switch 3 ) vsf member 4 link 1 ethernet 25
  • Switch 4 (2nd link to switch 1 ) vsf member 4 link 2 ethernet 26

Useful Aruba VSF Commands

show vsf or show vsf detail :  Shows the list of provisioned chassis members.

show vsf link or show vsf link detail : Shows the state of vsf links for all members.

show vsf lldp-mad status : Shows LLDP MAD (Multi-Active Detection).

show vsftrunk-designated-forwarder : Shows designated forwarders for each trunk.

Related Articles, References, Credits, or External Links

NA