I was recently involved in deploying an HPe Synergy 12000 Frame. And the network connections from it were ‘a little unusual’ so I thought I’d document that here, to save anyone else the problems I had.
I was connecting to an HP/Aruba 5412 switch so my cables were all HP/Aruba (to be on the safe side).
What you can see (above) is the MPIO Cable (K2Q46A P/N 800867-001) fixed onto the left (and above boxed,) there is a QSFP (P/N 817040-B21.) Note: this can be used either as 4 x 10Gbe or 4 x 8GbFC). On the right you can see the cable ends in 4x Standard LC fibre connectors, so you will also need 4x 10GB SR SFP+ Modules (Aruba P/N J1950D) – shown bottom right.
So what does it do? (Apart from cost a fortune!) Well the QSFP connects at 40Gb and splits the traffic down into 4 x 10Gb
Cabling and Configuring MPIO QSFP
Connecting up is pretty straight forward, REMEMBER when you connect the 40GB QSFP to the Synergy it will light purple if its connected, and flash purple when it sees activity.
Connecting to the switch is also easy enough, (WARNING: All the ports need to be Trunked (HP) or Ether Channelled (Cisco,)) with LACP enabled. You don’t need to worry about configuring LACP on the Synergy, that’s handled automatically by the ‘Uplink set’.
So the back of the ‘Frame’ has two interconnect links (If you are from a switch background think of these like stacking cables). And two MPIO uplink cables.
HPe/Aruba Switch Config For MPIO
As previously stated, the switch I’m using is an Aruba 5412, with two 8 Port 1Gb/10Gb modules (J9993-A). Here’s the relevant switch config;
[box]
Firstly give the interfaces a sensible name;
!
interface A2
name "Trunk Link to Synergy VC1 Port Q1"
exit
interface A3
name "Trunk Link to Synergy VC1 Port Q1"
exit
interface A4
name "Trunk Link to Synergy VC1 Port Q1"
exit
interface A5
name "Trunk Link to Synergy VC1 Port Q1"
exit
!
interface B2
name "Trunk Link to Synergy VC2 Port Q1"
exit
interface B3
name "Trunk Link to Synergy VC2 Port Q1"
exit
interface B4
name "Trunk Link to Synergy VC2 Port Q1"
exit
interface B5
name "Trunk Link to Synergy VC2 Port Q1"
exit
!
Show any 'already configured' Trunk links with a 'show trunk' commandIn my case two existed, (Trk1 and Trk2). So I used Trk3;
!
trunk A2-A5,B2-B5 Trk3 LACP
!
Now UNTAG vlan 1 (assuming that's your default VLAN) And TAG and VLANS that need to be used in the Synergy Deployment. (Note on an HP switch simply add
the Trk3 to the existing settings like so;
!
vlan1
untagged A6-A8,B6-B8,E1-E24,F1-F24,G3-G12,H3-H12,Trk1-Trk3
!
vlan 100
tagged Trk1-Trk3
exit
vlan 101
tagged Trk1-Trk3
exit
vlan 102
tagged Trk1-Trk3
exit
vlan 103
tagged Trk1-Trk3
exit
etc.
[/box]
Cisco Switch Config For MPIO
If you have a Cisco Switch then instead of ‘Trunking’ you will be ‘Ether Channelling’ for a more detailed explanation see the following post
The process is, you add Networks, then collect Networks together in Network Sets, Then you create Logical Interconnect Groups. Part of creating a Logical Interconnect Groups, involves creating an Uplink Set, which consists of both your Networks, and the The Uplink ports.
Note: A Network Set is used by a Server Profile, (or a Server Profile Template).
Create Networks
One View > Networking > Networks > Create Network
Create Network Sets
One View > Networking > Networks Sets > Create Network Set > Give it a name > Add Networks > Create.
Create Logical Interconnect Group
One View > Networking > Logical Interconnect Group > Create Logical Interconnect Group > Give it a name > Select the correct Interconnect Bay Set (see diagram above) > Select Interconnects > Add Uplink Set.
Give the set a name > Select the Type > Add in the Networks > Add in the Uplinks > Create.
Note: You only need to add in ALL the LOGICAL interfaces i.e. Q1:1, Q1:2,Q1:3,Q1:4 for EACH Interconnect module. .
After a few minutes if you look under One View > Networking > Logical Interconnects > You will see one listed that has the name of your Logical Interconnect group (with a divide symbol on the end!) Make sure ALL the logical uplinks are connected. (If not you will see LACP errors on the switch).
Related Articles, References, Credits, or External Links
I noticed some shiny Aruba switches on the bench today, they were for a job my colleague is working on. (Note: Each switch in a stack should be the same model, so these will need two stacks!)
I work on the occasional HP/Aruba core switch, but it’s been a while since I did any work on distribution switches like these. The first thing I learned, was there’s no dedicated stacking cable for them. They simply use a 10Gb (Twinax / DAC) cable. Which I suppose is pretty straight forward, but it means you lose an SFP+ port (which is a bit pants).*
*Note: You can stack with 1GB cables, but you can’t mix and match!
So I said “Give me a shoult when you stack them and I’ll take a nosey!”
Solution
In the ‘land of Aruba’ this is called creating a VSF (Virtual Switching Fabric). As you can see from the photo, these are 2930F Switches, and you can stack up to four switches in a VSF. The same stacking method is used on the 5400R (v3) and 5412, where you can link two 5400R or 5412’s).
Also this method is NOT to be confused with ‘Fabric Stacking’ which is available on the 2920,2930M,3800,3810M models, (that is more like Cisco FlexStack, with a dedicated 100Gb stack cable).
So, assuming you have your switch new and fresh, connect in with your console cable, and dedicate a port to use for VSF.
[box]
Aruba-2930F-24G-PoEP-4SFPP# conf t
Aruba-2930F-24G-PoEP-4SFPP(config)# vsf member 1 link 1 ethernet 25
All configuration on this port has been removed and port is placed in VSF mode.
[/box]
Then place the switch into a VSF domain
[box]
Aruba-2930F-24G-PoEP-4SFPP(config)# vsf enable domain 1
This will save the current configuration and reboot the switch.
[/box]
The switch will ask for a reboot, let it do so.
Repeat the procedure on the second switch, (but this will be member 2).
[box]
Aruba-2930F-24G-PoEP-4SFPP# conf t
Aruba-2930F-24G-PoEP-4SFPP(config)# vsf member 1 link 1 ethernet 25
All configuration on this port has been removed and port is placed in VSF mode.
Aruba-2930F-24G-PoEP-4SFPP(config)# vsf enable domain 1
This will save the current configuration and reboot the switch.
[/box]
Once again let the switch reboot.
Post reboot you will see the ports are ‘re-numbered’ 1/{port-number} on vsf member 1, 2/{port-number} on vsf member 2 etc.
[box]
Aruba-2930F-24G-PoEP-4SFPP# show interfaces
Status and Counters - Port Counters
Flow Bcast
Port Total Bytes Total Frames Errors Rx Drops Tx Ctrl Limit
------------ -------------- -------------- --------- --------- ---- -----
1/1 0 0 0 0 off 0
1/2 0 0 0 0 off 0
1/3 0 0 0 0 off 0
1/4 0 0 0 0 off 0
<---------------Output Removed For The Sake Of Brevity-------------->
1/10 0 0 0 0 off 0
1/11 0 0 0 0 off 0
1/12 0 0 0 0 off 0
1/13 0 0 0 0 off 0
<---------------Output Removed For The Sake Of Brevity-------------->
1/19 0 0 0 0 off 0
1/20 0 0 0 0 off 0
1/21 0 0 0 0 off 0
1/25 1,496,823,949 23,354,845 0 0 off 0
<---------------Output Removed For The Sake Of Brevity-------------->
2/1 0 0 0 0 off 0
2/2 0 0 0 0 off 0
2/3 0 0 0 0 off 0
2/4 0 0 0 0 off 0
<---------------Output Removed For The Sake Of Brevity-------------->
2/22 0 0 0 0 off 0
2/23 0 0 0 0 off 0
2/24 0 0 0 0 off 0
2/25 1,536,016,322 23,966,915 0 0 off 0
2/26 0 0 0 0 off 0
2/27 0 0 0 0 off 0
2/28 0 0 0 0 off 0
[/box]
If you need to Stack 3 or 4 Switches then you need to add a second link, and create a ring;
i.e.
Switch 2 (2nd link now to switch 3) vsf member 2 link 2 ethernet 26
Switch 3 (1st link to switch 2 ) vsf member 2 link 1 ethernet 25
Switch 3 (2nd link to switch 4 ) vsf member 2 link 2 ethernet 26
Switch 4 (1st link to switch 3 ) vsf member 4 link 1 ethernet 25
Switch 4 (2nd link to switch 1 ) vsf member 4 link 2 ethernet 26
Useful Aruba VSF Commands
show vsf or show vsf detail : Shows the list of provisioned chassis members.
show vsf link or show vsf link detail : Shows the state of vsf links for all members.
show vsf lldp-mad status : Shows LLDP MAD (Multi-Active Detection).
show vsftrunk-designated-forwarder : Shows designated forwarders for each trunk.
Related Articles, References, Credits, or External Links
