Fortigate Load Balancing

KB ID 0001762

Problem

I’ve been getting through my NSE4, and one of todays topics was NAT, just as an offhand comment the ‘narrator‘ (I say narrator because it’s a monotonous robot AI voice,) mentioned Fortigate Load Balancing.

In the past (with my Cisco hat on) when I’ve been asked about load balancing, I’ve said ‘If you want to load balance, buy a load balancer‘. But the Fortigate does try to be ‘all things to all men‘ so I wondered just how good a load balancer can it be?

Turns out, quite a decent one, if you just want simple http round robin, it does that, it you want weighted traffic routing, or host health monitoring, or HTTP cookie persistence, and even SSL offload. It’s as good as anything I’ve ever worked on before. Here’s my Fortigate ‘Test Bench‘, you will see I’ve added three web servers (on the right) called Red, Green, and Blue (the significance of which will become apparent). Note: Yes there’s another web server at the bottom, (I’m too lazy to remove it from the lab!)

I’m going to setup simple round robin load balancing between these three web servers, and I’m going to get the Fortigate to monitor their health by simply making sure they respond to ping packets. (Note: it can monitor http availability or something a little better if you wish).

Solution

This tripped me up for a while! Load balancing is a feature, you need to  turn it on first, System > Feature Visibility > Load Balancing > Enable.

FortiGate Load Balancing: Create a Health Check

Cisco Types: Think of this as a tracked SLA

Policy & Objects > Health Check > Create New > Give it a name > Type = Ping > Interval = 10 > Timeout = 2 > Retry = 3 > OK

Now create a Virtual Server (not a VIP!) Policy  & Objects > Virtual Servers > Create New  > Name = Give it a sensible name > Type = HTTP > Interface = Your Outside/WAN interface  > Virtual Server IP (Externally!) > Virtual Server Port = 80 > Load Balancing method = Round Robin > Persistence = HTTP cookie > Heath Check = Select the one you created above.

Scroll down > Real Servers > Create New.

Add in the first (internal server IP) > Port = 80 > Max connections  = 0 (that’s unlimited) > OK.

Repeat the process to add the remaining servers > OK.

FortiGate Load Balancing: Enable Firewall Policy

Now you need to ‘allow’ traffic in (it is a firewall after all!) Policy & Objects > Firewall Policy (or IPv4 policy on older firewalls) > Create New > Name = Give it a sensible name > Incoming Interface = Outside > Outgoing Interface = Inside > STOP Change Inspection Mode to PROXY Based > Destination = Your Virtual Server (it’s not visible unless you have enabled proxy based!) > Schedule = Always > Service = All > Action = Accept > NAT = Enabled > You may also enable AV inspection > OK.

FortiGate Load Balancing:Testing and Tweaking

So from ‘Outside’ let’s hit our load balanced page.

That’s great but if you hit refresh a few times nothing changes (in production nothing would change anyway, but to prove my back end servers are getting used and load balanced, each of mine serves a different coloured page (hence the red, green and blue server names). The reason I’m only seeing the blue one, is because we enabled ‘HTTP cookie Persistence‘ let’s just nip back onto the firewall and disable that (set it to None > OK).

Now when I refresh by browser I can see it cycling though the back end servers.

FortiGate SSL Offload

To use and process SSL requires some CPU power, some websites (like this one) serve their webpages protected by https and the certificate that enables that lives on the web server, for sites like mine that are getting about 12k hits a day that’s fine but if you are getting hundreds of thousands of hits a minute that’s a MASSIVE drain on CPU resources. That’s what SSL offload is all about, getting another device (in this case the Fortigate) to do all the heavy lifting for you. Then the back end servers can get on with the job of serving web pages. 

Upload the Certificate to the FortiGate

For HTTPS you will need a web certificate that will be trusted by your visitors. I’m lazy and tight so I’ll just create one in Microsoft Certificate Services, but in Production you will need Publicly Signed Certificate. System > Certificates (if you can’t see certificates) > Import > Local Certificate.

Mine’s in PFX format so I need to select PKCS#12 > upload the certificate and supply a password > OK

FortiGate: Enable SSL Offload

On your Virtual Server, change the Type to HTTPS > Virtual ServerPort to 443 > Certificate to the one you just uploaded > OK.

We are now serving pages securely even though the web servers are not configured for https.

Related Articles, References, Credits, or External Links

NA

F5: Setup Basic Web Load Balancing

KB ID 0001698

Problem

In past articles I’ve got my F5 BIG IP appliance up and running, and I’ve built some web servers to test load balancing. Now to actually connect things together and start testing things. Below is my lab setup, I will be deploying simple web load balancing (Static: Round Robin) between three web servers, each serving a simple HTTP web site.

Test F5 to Web Server Connectivity

For obvious reasons the F5 needs to be able to speak to the web servers, so it needs to be on the same network/VLAN and have connectivity. To test that we can log onto the the F5 console directly, and ‘ping’ the web servers.

So connectivity is good, let’s make sure we can actually see the web content on those boxes, the best tool for that is to use curl, which will make a web request, and the wen server ‘should’ return some HTML.

[box]curl http://10.2.0.11[/box]

F5 BIG-IP Load Balancing Terminology

Yeah I said ‘load balancing‘ and not ADC sue me! There are a number of building blocks that F5 uses, and you need to understand the terminology to put things together, firstly lets look at things BEHIND the F5 appliance;

  • Node: An actual machine/appliance, (be that physical or virtual.) That provides some sort of service or a collections of services e.g. a web server, telnet server, FTP site etc.
  • Pool Member: Is a combination of a Node AND a Port/Service, e.g. 192.168.1.100:80 (IP address and TCP port 80 (or HTTP)).
  • Pool: A Logical collection on Pool Members, that provide the same service e.g a collection of pool members offering a website on TCP port 80.

F5 BIG-IP Adding Nodes

While connected to the web management portal > Local Traffic > Nodes > Create (Note: You can also press the green ‘add’ button on the Node pop-out on newer versions).

Specify a name > Description (optional) > IP address (or FQDN) > ‘Repeat‘ > Continue to add Nodes as required, then click ‘Finished‘.

F5 BIG-IP Adding Pools

Now we have our Nodes, We need to create a Pool. Local Traffic > Pools > Create, (again on newer versions theres a green add button on the pop-out).

Add a Name > Description (Optional) > Add an applicable Health Monitor (in our case http) > Select the ‘Node List’ radio button > Select your first Node > Set the Port/Service  > Add  > Continue to Add the remaining Nodes.

Note: Here is where you add the IPs to the Port/Service and create the Pool Members.

Sorry! Busy Screenshot

When all the Nodes are added > ‘Finished‘.

Your web pool ‘should‘ show healthy, Note: that does not mean ALL the nodes are online!

To make sure ‘all’ the Nodes are healthy > Go to the Members Tab.

F5 BIG-IP Virtual Servers

I’m not a fan of using this term ‘Virtual Server‘ I prefer Virtual IP (or VIP,) but we are where we are! Above we’ve looked at things BEHIND the F5, now we need to present those services IN FRONT of the F5 (Note: I don’t say publicly, because we deploy plenty of BIIG-IP solutions inside  networks). So a Virtual Server is the outside IP address or FQDN of that a ‘consumer’ will connect to;

Local Traffic > Virtual Servers > Create.

Supply a name > Description (optional)  > Destination Address (the ‘available outside’) IP address > Set the service/port > Scroll down to the bottom.

Set the ‘Default Pool’ to the pool you created (above) > ‘Finished‘.

For a brief overview or check what you have created  > Click Local Traffic > Network Map Note: This will look different on older versions of the F5.

Then test the service form the outside, here each web server serves a different colour page so I can test it’s working properly.

My Web Page Does Not Change? If you keep seeing the same colour/page then it’s probably because you chose browser is ‘caching’ web content on your test machine, you may need to disable caching on your chosen web browser, for an accurate test.

So that’s Static Round Robin (Equal Ratio) Based Load Balancing. In the next article I’ll look at how you can manipulate the ratios, to better serve your hardware, and requirements.

Related Articles, References, Credits, or External Links

NA

VMware: Change IOPS Limit From 1000 to 1

KB ID 0001532

Problem

I got asked to do this by a client this week, HP has requested that this be set for connections to their Storevirtual VSA that had been having some problems.

Solution

I followed the instructions and was at first confused because I could not see the settings that needed changing? That’s because this only applies if you have MULTIPATHING enabled and set to ‘Round Robin’. So if your storage does NOT look like below, (All paths Active I/O). then this procedure is not applicable.

So assuming you are using round robin multipathing, and, <ahem!> the storage vendor hasn’t just pulled a solution from a list of things that might work, rather than actually diagnosing the problem. Then you can see the current setting with the following command;

[box]

esxcli storage nmp device list

[/box]

Take note of the iSCSI storage names, below you can see they all start with naa.6000, you can also see the IOPS value is set to 1000.

To change the value use the following command (change the value in red to match yours);

[box]

for i in `esxcfg-scsidevs -c |awk '{print $1}' | grep naa.6000`; do esxcli storage nmp psp roundrobin deviceconfig set --type=iops --iops=1 --device=$i; done

[/box]

Then recheck, the new value should be ‘1’.

Related Articles, References, Credits, or External Links

Disable ATS Heartbeat

VMware ‘Disable DelayedAck’ Does Not Work?

 

Remote Desktop Services: Balancing Sessions Hosts and Connection Brokers

KB ID 0001424

Problem

I got an email from a colleague who was setting up an RDS farm, (2012 R2). He was having some problems and asked me; “If the Connection Broker brokers the connections to the Session Hosts, how do I RDP to the Session Broker?”

This threw me completely, I usually jump on the console in VMware or use a third party remote management tool, I don’t tend to to RDP onto servers. I had fallen into the same trap he had. I assumed: You connect to a SESSION BROKER and it BROKERS YOUR SESSION to the least busy session host, (or reconnects your broken sessions).

THIS IS WRONG!

 

How Session Brokers Work

You don’t connect to a session broker, (unless you are an admin who is about to do some work on the Session Broker). You connect to a DNS RECORD, and that record points to a SESSION HOST, (I know that makes no sense, but bear with me). And you create a DNS record with the SAME NAME for every Session Host like so,

 

This works because, (by default) Windows DNS uses ’round robin’ so if it has multiple values for one DNS name is responds with the first one to the first request, the second one to the second etc.

But Pete? Round Robin is Bobbins for Load Balancing? Yes it is, that’s what the Session Broker is for! In reality this is what happens;

This is two scenarios that should clear things up, User1 queries DNS for TSFarm.my-domain.com and gets an IP of 192.168.1.1, They go to that SESSION HOST, the the session host CHECKS WITH THE CONNECTION BROKER, firstly to see if User1 already has a session on another session host, if so they are reconnected to that session, above that’s on SESSION HOST 2.

Then User2 attempts to connect toTSFarm.my-domain.com and gets an IP of 192.168.1.2 (Because of DNS ’round-robin’). They go to that SESSION HOST, then the session host CHECKS WITH THE CONNECTION BROKER, firstly to see if User2 already has a session on another session host, in this case they don’t. But, this host already has User1 connected to it, so it redirects User2 onto SESSION HOST 1.

Of course a user can connect to a SESSION HOST and after checking with the the CONNECTION BROKER they get connected to the host they originally queried if, (for example) the other session hosts are busier, (and the user has no existing sessions.)

But With Server 2012 You Can Do Connection Broker Load Balancing? Yes, you can, but that’s load balancing for the connection brokers, NOT the user sessions!

 

Related Articles, References, Credits, or External Links

Thanks to James White for making me do some work!

Windows Server: Connecting to iSCSI Storage Using MPIO

KB ID 0001392

Problem

In my scenario my Windows Server is a VMware virtual machine. To enable MPIO (Multipath I/O) I’m going to need two network cards, connected to the two iSCSI networks. 

Above I’ve shown both iSCSI networks in  different colours 192.168.51.0/24 and 192.168.50.0/24 in production I would also have these in their own VLANs, (or even separate physical networks).

This article is not about setting up your iSCSI Target/Storage, I’m assuming you have this up and running with the correct IP addresses connected to the correct networks ready to go.

Note: I’m also NOT using iSCSI authentication, and I’m also assuming you have allowed either the two IP addresses of the Windows server, (or more likely its iSCSI iqn address), access to the storage.

Solution

Firstly MPIO is NOT enabled or installed by default, you need to add it. Open Server Manager > Manage > Add Roles and Features > Follow the wizard all the way to ‘features’ > Enable Multipath I/O > Complete the Wizard.

Back in Server Manager > Tools > MPIO > Discover  Multi-Paths > Add support for iSCSI devices > Yes  > Let the server reboot.

After the reboot go back into the MPIO properties, and make sure iSCSI is now listed, (MSFT2005iSCSIBusType_0x9). You can close the MPIO properties now.

Now back in Server Manager > Tools > iSCSI Initiator.

First task is to add the TWO iSCSI Target IP’s (192.168.50.200 and 192.168.51.200) > Discovery > Discover Portal > Put in the first iSCSI Target IP > Advanced > Local Adapter = Microsoft iSCSI Initiator > Initiator IP = The Servers NIC that’s on the same iSCSI network as this target, (i.e. 192.168.50.6 or 192.168.51.6) > OK > OK > Apply > OK.

NOW REPEAT THE PROCEDURE FOR THE SECOND iSCSI TARGET

Assuming your iSCSI and networking setup are correctly, you should start to see the storage appearing on the ‘Targets’ tab. Select the first piece of storage you want to attach > Connect > Tick ‘Enable Multi-path’ > Advanced > Local Adapter = Initiator IP (either 192.168.50.6 or 192.168.51.6)  > Target Portal IP  = (The iSCSI Target IP that corresponds to the IP you have just set, either 192.168.50.200 or 192.168.51.200) > OK > OK > Apply > OK.


The status should change to connected.

NOW REPEAT THE PROCEDURE A ‘SECOND TIME’ FOR THE SAME PEICE OF STORAGE, BUT CONNECT TO IT FROM THE OTHER iSCSI IP ADDRESS, TO THE OTHER iSCSI TARGET IP. THERE YOU CONNECT TO EACH ONE ‘TWICE’ (ONCE OVER EACH iSCSI NETWORK).

If you now look in the properties of the storage, you will see it has two identifiers and two IPv4 Portal groups.

At this point you would need to go into ‘Disk Management’ (Server Manager > Tools > Computer Management > Disk Management). You will see the storage presented but ‘Offline’ you will need to bring the drive online > Create a partition on it, (if it does not already have one),  and you can also assign a new drive letter. Note: Look in the Properties here, and you can prove MPIO is working and change the MPIO policy (if you require).

Related Articles, References, Credits, or External Links

NA