I have made a lot of tests in order to discard MTU and connection problems and I got the following results:
From each ESXi CLI I executed the following ping tests:
vmkping -s 1572 -S vxlan -I vmk10 -d 192.168.12.x
"x" is each vxlan vmkernel (VTEP) of the other two ESXi and lastly the VTEP IP of the Edge Node.
From the two hosts which have the Geneve tunnels down against the Edge node (host 5 and 6) I can reach all the VTEPs of the other host transport nodes (ESXi hosts) but not the edge one. It is to say, I cannot ping the VTEP of the edge node which is what I see from the GUI, the Geneve tunnel against it is down from those two hosts. But they are up against the other two transport nodes.
I make another test that was taking out the "size" argument of the ping against the VTEP IP of the Edge Node in order to see if it was a MTU problem and I neither could reach it. It is as if the Edge Node's VTEP didn't exist for those hosts. No ping response.
But from the only ESXi host of the three ones that has the Geneve tunnel up and running against the Edge Node I can ping all the VTEPs (the ones that belong to the host transport nodes and the one that belongs to the edge node). It is also consistent with what I see in the GUI, that that ESXi host (host 14) has the tunnel with the Edge Node up.
Strange thing:
Just to see if there was something wrong in the configuration of the VDS where the Edge VM is connected to, I created in the same Distributed Port Group a vmkernel port for hosts 5 and 6 (as default, without associating it to any IP stack). I configured it with an IP address of the same subnet of the VTEPs. Remember those two hosts have the tunnels with the Edge Node down.
I executed the same ping using that vmkernel interface as the source and I couldn't reach any of the other hosts VTEPs (the ones that could be reached before using as source interface the VTEPs) but I could reach the Edge Node one!!! It is to say, for some reason from a vmkernel port in the same VTEP VLAN and subnet I cannot reach the other hosts VTEPs but I can reach the VTEP that I cannot reach from the other VTEPs that are created in the N-VDS of each host.
I don't know if it has anything to do with the used IP stack..
But from the point of view of the Edge Node VTEP, what is the difference between the ping request it receives from the VTEPs of hosts 5 or 6, and the same ping request it receives from the vmkernel port of host 5?? Just the source IP address. In the ping packet there is no information about the source ESXi interface that originates the request nor the IP stack. It is an ICMP standard packet!!!
One thing I don't know how to test:
I don't kwnow how to generate the same ping requests that I do from the ESXi CLI but do it from the Edge Node. Because from the Edne Node CLI the only interface I see (if iI issue the "get interfaces" command) is the managent one, not the VTEP is has configured. I see the VTEP issuing the command "get logical switches", but is there any ways to generate a ping request from that IP address as the source?
Thank you,
Guido
