I use EVE-NG a lot, it’s an awesome tool. Also I’m lucky enough to have my own ESX servers, so that’s where it lives. I’ve noticed this problem before, but I’ve either given up, and done something else, or it’s manifested itself in an ‘odd’ way that I can work around.
If you’re new to connecting EVE-NG to a live network you might want to read the following post first;
When setting up a new lab, I connected a Ciso IOL router to my cloud object, and it successfully got an IP from DHCP, but could not communicate with the outside world. So I replaced it with a Cisco (Dynamips) router, same thing! So I replaced with with a Cisco ASAv, same thing!
I moved the EVE-NG machine onto its own standard vSwitch, (no difference).
I hadn’t committed the ‘schoolboy error‘ of forgetting to allow promiscuous mode on the port group.
I could also see from my physical network, that there was layer 2 connectivity happening, as it was in in the ARP cache of my core switch.
I did notice that if I waited for a long time, it would start working, so (on the Core Switch) I flushed the ARP cache, and pinged the EVE-NG device and got a response, then it worked fine from EVE-NG, (for a while, in a manner of speaking!) If I tried to NAT any other traffic, or do anything else, then the problem returned. I could keep flushing the ARP cache on the switch, but that’s a bit annoying?
Solution
Well, (in my case) the problem turned out to be a problem with the fact I have ‘teamed‘ physical NICs on the vSwitch, which you can see above (vmnic0, and vmnic7). AS SOON as I removed one, and only had one physical uplink it worked faultlessly?
Everything works now.
Note: I tried changing the load balancing algorithms to ‘Route based on IP hash‘, ‘Route based on source MAC hash, and even ‘Use Explicit failover order‘, none of these worked.
I did see other people in forums that were saying, ‘I only have one physical uplink‘, I’m suspecting that in their case, it’s promiscuous mode was missing, but feel free to comment below, if any one manages a better work-around / fix / explanation.
Related Articles, References, Credits, or External Links
I see this get asked in online forums A LOT. If you have an existing AnyConnect VPN setup, and then need to add another network how do you do it?
Well that depends on where the new network is, and how it’s entering the firewall, these diagrams can be either way round, but the new network will either be coming into the ASA on the same interface (i.e you’ve spun up a new network for phones, or a new department etc), or the traffic will be coming into the ASA through a different interface, (like a DMZ, or partner network).
Option A: New Network is on another interface;
Option B: New Network is on existing (inside) interface;
Note: The process for adding the new network is the same for both, BUT depending on which interface the new traffic is coming in on, this will change your NAT command.
Locate the NAT Exemption for the AnyConnect traffic, and add a new one on the SAME interface.
If usingSplit Tunnelingadd the new network to the Spit Tunnel ACL
Locate the Nat Exception (or NO NAT on old Cisco Money) that prevents the AnyConnect traffic form getting NATTED.
[box]
ASA-1(config)# show run natnat (inside,outside) source static any any destination static Obj-ANYCONNECT-SUBNET Obj-ANYCONNECT-SUBNET no-proxy-arp route-lookup
nat (inside,outside) source static Obj-Local-LAN Obj-Local-LAN destination static Obj-Remote-LAN Obj-Remote-LAN no-proxy-arp route-lookup
!
[/box]
From the output above we can see that the Nat exemption for the existing traffic is highlighted, we simply need to add another one, for the new interface (which we will call dmz).
Note: I’m using any, yours might be using a specific object or object-group, if so, you will need to create new objects or object groups for the new network(s), for the next step, I’m sticking with any!
[box]
ASA-1# configure terminal
ASA-1(config)# nat (dmz,outside) source static any any destination static Obj-ANYCONNECT-SUBNET Obj-ANYCONNECT-SUBNET no-proxy-arp route-lookup
[/box]
Now let’s see if we are using split tunnelling;
[box]
ASA-1# show run group-policy
group-policy DfltGrpPolicy attributes
webvpn
customization value Portal
group-policy GroupPolicy_AnyConnectVPN internal
group-policy GroupPolicy_AnyConnectVPN attributes
wins-server none
dns-server value 192.168.1.10 192.168.1.11
vpn-tunnel-protocol ssl-client ssl-clientless
split-tunnel-policy tunnelspecified split-tunnel-network-list value SplitTunneling
default-domain value petenetlive.com
webvpn
anyconnect profiles value AnyConnect type user
customization value Portal
group-policy GroupPolicy3 internal
group-policy GroupPolicy3 attributes
vpn-tunnel-protocol ikev1
group-policy GroupPolicy2 internal
group-policy GroupPolicy2 attributes
vpn-filter value ACL_Filter
vpn-tunnel-protocol ikev1
group-policy GroupPolicy1 internal
group-policy GroupPolicy1 attributes
vpn-tunnel-protocol ikev1
webvpn
customization value PulsantPortal
[/box]
So we are using split tunnelling, and our split tunnel ACL is called SplitTunneling.
[box]
ASA-1# show run access-list SplitTunnelingaccess-list SplitTunneling standard permit 192.168.1.0 255.255.255.0
[/box]
Now simply just add the new subnet;
[box]
ASA-1(config)# access-list SplitTunneling standard permit 192.168.100.0 255.255.255.0
[/box]
Option B: New Network is on the Same Interface
Tasks on ASA
Locate the NAT Exemption for the AnyConnect traffic, and add a new one for the new interface.
If usingSplit Tunnelingadd the new network to the Spit Tunnel ACL
Locate the Nat Exception (or NO NAT on old Cisco Money) that prevents the AnyConnect traffic form getting NATTED.
[box]
ASA-1(config)# show run natnat (inside,outside) source static any any destination static Obj-ANYCONNECT-SUBNET Obj-ANYCONNECT-SUBNET no-proxy-arp route-lookup
nat (inside,outside) source static Obj-Local-LAN Obj-Local-LAN destination static Obj-Remote-LAN Obj-Remote-LAN no-proxy-arp route-lookup
!
[/box]
From the output above we can see that the Nat exemption for the existing traffic is highlighted, IMPORTANT: Because I’m using any, (yours might be using a specific object or object-group), I DONT NEED TO ADD ANYTHING as any covers the newly added subnet. if you didn’t use any, then you will need to create new objects or object groups for the new network(s), for the next step, I’m sticking with any! then
With regards to Ping, out of the box a Cisco firewall will allow you to ping the interface you are connected to, so in a normal setup inside clients can ping the inside interface, and the firewalls outside interface can be pinged from outside.
OK – to understand pinging through a Cisco Firewall you need to understand that Ping is part of the ICMP protocol suite, and unlike other protocols is not “connection orientated” what that means is, (on a new firewall that has no rules applied outbound) the firewall happily lets ping traffic out but it won’t let ping traffic back in – this results in a failure on the client.
Cisco ASA and Cisco PIX (version 7 and above) From CLI
Version 7 introduced an ICMP inspection engine so that it could track ICMP requests like other protocols. It’s NOT turned on by default. And the command is “inspect icmp” but you need to enter the default map first (this assumes you have the standard policy-map). Connect to the firewall and use the following commands from config terminal mode. then save the changes with a “write mem” command.
[box]
PetesASA>
PetesASA> enable
Password:********
PetesASA# conf t
PetesASA(config)# policy-map global_policy
PetesASA(config)# (config-pmap)#class inspection_default
PetesASA(config)#inspect icmp
PetesASA(config)# write mem
Building configuration...
Cryptochecksum: 5c8dfc45 ee6496db 8731d2d5 fa945425
8695 bytes copied in 3.670 secs (2898 bytes/sec)
[OK]
PetesASA(config)#
[/box]
Cisco ASA and Cisco PIX (version 7 and above) FromASDM
Connect to the ASDM > Configuration > Firewall > Service Policy Rules > Select “inspection_default” > Edit > Rule Actions > Tick ICMP > OK > Apply > File > Save running configuration to flash.
Older firewalls do not have an inspection map, nor was there a “fixup” for ICMP and ping traffic, so you need to explicitly allow the return icmp traffic back in. Note: this assumes you already have an inbound access-list called “inbound”, and we are adding some more lines to it, change the works inbound to match the name/number of your inbound access list (the “show access-group” will tell you what is applied to the outside interface),
e.g.
[box]
PetesASA>
PetesASA> enable
Password: ********
PetesASA# show access-group
access-group inbound in interface outside <<This ones called inbound.
[/box]
If you already have an access-list applied then simply substitute the name word “inbound” for the name on your ACL.
[box]
PetesASA# conf t
PetesASA(config)# access-list inbound permit icmp any any echo-reply
PetesASA(config)# access-list inbound permit icmp any any time-exceeded
PetesASA(config)# access-list inbound permit icmp any any unreachablePetesASA(config)# access-list inbound permit icmp any any source-quench
[/box]
If you HAD an inbound ACL skip this step, If you didn’t have one you need to apply the ACL with an access-group command.
[box]
PetesASA(config)# access-group inbound in interface outside
[/box]
Lastly save your work with a write mem command.
[box]
PetesASA# write mem
Building configuration...
Cryptochecksum: 4d7f7ccd 5c55a9e1 6ced12c4 46728bc7
[OK]
PetesASA#
Connect to the PDM > Configuration > Access Rules > Rules > Add > Permit > Outside Inside > Tick ICMP > Select “echo-reply”> OK > Apply > File > Save running configuration to flash.
Then repeat for time-exceeded, unreachable and source-quench
Stop Interfaces replying to Ping traffic
As stated above all firewall interfaces will respond to pings if they are on the network you are connected to. To stop this you use the “icmp” command.
[box]
Syntax
icmp {permit or deny} {any or IP and subnet}{type of icmp traffic}{interface name}
Example
icmp deny any echo outside
Connect to the PDM > Configuration > system Properties > Administration > ICMP > Add > > Specify the Type, Interface, Source etc > OK > Apply > File > Save running configuration to flash.
If you are still having problems, see the following article;
On a newly deployed FirePOWER service module I wanted to test connectivity and attempted to ping a public IP address. This is what happened;
> expert admin@Petes-SFR:~$ ping 8.8.8.8 ping: icmp open socket: Operation not permitted
My first though was, “Well you have to set a default gateway on the SFR when you set it up, so the firewall is probably blocking ICMP”. So I checked the default policy and indeed there was no ICMP inspection turned on.
UPDATE: The correct way to do this, (as pointed out below) is the use;
[box]system support ping {IP or Hostname}[/box]
-=-=-=- Back to the original post -=-=-=-
On checking the Firewall, (Note: The decrement-ttl-tcl isn’t really needed, I do that to make tracerts work).
[box]
Petes-FW(config)# policy-map global_policy
Petes-FW(config-pmap)# class inspection_default
Petes-FW(config-pmap-c)# inspect icmp
Petes-FW(config-pmap-c)# class class-default
Petes-FW(config-pmap-c)# set connection decrement-ttl
Petes-FW(config-pmap-c)# exit
Petes-FW(config-pmap)# exit
Petes-FW(config)# write mem
Building configuration...
Cryptochecksum: ae83c992 52a79538 b8370d6f 36125070
3120 bytes copied in 0.290 secs
[OK]
Petes-FW(config)#
[/box]
Unfortunately it still didn’t work!
Solution
Some forum searching yielded the same error on other flavours of Linux, the fix there was to change some permissions, so I gave that a try, and it worked.
[box]
admin@Petes-SFR:~$ sudo chmod u+s /bin/ping
We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:
#1) Respect the privacy of others.
#2) Think before you type.
#3) With great power comes great responsibility.
Password:{Your-SFR-Admin-Password}
admin@Petes-SFR:~$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_req=1 ttl=52 time=12.8 ms
64 bytes from 8.8.8.8: icmp_req=2 ttl=52 time=13.7 ms
64 bytes from 8.8.8.8: icmp_req=3 ttl=52 time=12.6 ms
64 bytes from 8.8.8.8: icmp_req=4 ttl=52 time=13.2 ms
[/box]
Related Articles, References, Credits, or External Links
I have plenty of real ASA firewalls to play with, but it’s getting round to re-certification time, so for ease I thought I’d run up a Virtual ASA on my vSphere 5 box just for testing and breaking.
However no matter what type of Virtual Network adaptor I used (vmxnet3, e1000, etc) I could not connect to the virtual firewall.
Solution
1. Firstly make sure you actually have an IP address in the correct range set on the ASA, and the interface is up.
Note: To connect a virtual ASA to the outside world you need to have a ‘Cloud’, then assign your machines NIC to the cloud. Finally you join the cloud to the ASA with a switch.
2. From my laptop (out on the network), I could not connect to, or ping the ASA 🙁
3. However if you log into your ESX/vCenter Server and change the settings on the virtual switch that the parent VM is attached to. Configuration > Networking > Switch > Properties > vSwitch > Edit > Security Tab > Change Promiscuous Mode to Accept > OK.
4. Now it works!
Related Articles, References, Credits, or External Links
Considering we use ICMP to test connectivity, the fact that it is not a stateful protocol can be a major pain! Last week one of my colleagues rang me up and said, “Can you jump on this firewall, I’ve got no comms, and I cant ping external IP addresses. I can ping the internet from the firewall and I can ping internal IP addresses form the firewall”.
Solution
1. Before we start, lets get the basics out of the way, does the client you are pinging from have a firewall turned on? Can you ping the inside interface of the firewall?
2. Pinging will never work unless you have ICMP inspection turned on on the firewall. See the following article.
3. At this point we troubleshoot as we would for any other traffic through the firewall. To do this we use packet-tracer, the syntax is slightly different for ICMP, than it is for TCP and UDP though. We need to specify an ICMP Type and an ICMP code, to make sure the traffic leaves the firewall we trace ICMP type 8 (echo), with ICMP code 0 (none). And ICMP type 0 (Echo-reply), with ICMP code 0 (none), for traffic inbound.
Test Outbound Ping
Petes-ASA# packet-tracer input inside icmp 192.168.1.1 8 0 4.2.2.2
Testing Inbound Ping(where 123.123.123.123 is the public IP you are mapped to)
Petes-ASA# packet-tracer input outside icmp 4.2.2.2 0 0 123.123.123.123
4. Make sure the client you are on is getting NATTED or PATTED through the firewall. Below we will assume my internal IP address is 192.168.1.1.
Note: If you have names enabled and 192.168.1.1 has a name, you will get no results! issue a no names command from configure terminal mode to check.
[box]
Petes-ASA# show xlate | incl 192.168.1.1
If this machine was being NATTED to another public IP address it would look like..
NAT from inside:192.168.1.1 to outside:123.123.123.124
If this machine was being PATTED to a public IP address it would look like..
ICMP PAT from inside:192.168.1.1/1 to outside:123.123.123.123/1 flags ri idle 0:00:07 timeout 0:00:30
[/box]
If it fails at this stage then check you network translation configuration on the firewall.
5. If all appears normal so far you can capture the traffic as it passes though the firewall, below I’m successfully capturing the ICMP traffic though the firewall.
