In this post, we will talk about BGP routing configuration required for vCloud Foundation 4.0. In my last post, we deployed vCloud Foundation 4.0 without AVN (Application Virtual Networks). This post includes AVN and its configuration. Application Virtual Networks are logical segments (Logical Switches in NSX-V) that gets created in NSX-T env. These networks can be created manually as per your needs.
In VCF deployment parameter sheet, you see an option to enable AVN.
If you select ‘NO’, VCF does the deployment of NSX-T managers, Creates VIP & installs NSX-T vibs on esxis, but does not deploy edge and no routing gets configured.
I have selected ‘Yes’ and filled all required parameters in the sheet. We will talk in detail on all these parameters. I have not configured anything on my TOR (top of the rack) yet. We will see errors in deployment and configure it accordingly instead of configuring it beforehand.
Note: VCF does not validate BGP and TOR information in validation process. If you select ‘Yes’ and if something is missing / incorrectly configured, you will see an error in deployment.
With that, lets get started. I selected ‘Yes’ without configuring anything on TOR.
Deployment of SDDC started and showed an error ‘INPUT_PARAM_ERROR Invalid parameter: {0}’ while validating NSX-T Edge parameter. After digging into logs (vcf-bringup.log), I see this…
‘Unexpected edge node form factor: EXTRA_SMALL’
For some reason, it did not like that parameter. The parameter sheet was already uploaded to VCF and it can not be changed once the deployment starts. The only option is to get into respective JSON file and change the values. I did not bother to change the json file. Changed the value deployment parameter to ‘medium’ and restarted the deployment.
Next Error on Cloud Builder: “BGP neighbor configuration is invalid.”
At this stage, it is looking for 2 TOR IP’s for BGP peering. We configured these two IP’s here…
Let’s configure BGP on our VyoS Router. Get into your 1st VyOS router and run these commands.
set interfaces ethernet eth4 address ‘172.27.11.1/24’
set interfaces ethernet eth4 mtu ‘9000’
set protocols bgp 65001 parameters router-id 172.27.11.1
set protocols bgp 65001 neighbor 172.27.11.2 update-source eth4
set protocols bgp 65001 neighbor 172.27.11.2 remote-as ‘65003’
set protocols bgp 65001 neighbor 172.27.11.3 remote-as ‘65003’
set protocols bgp 65001 neighbor 172.27.11.2 password VMw@re1!
set protocols bgp 65001 neighbor 172.27.11.3 password VMw@re1!
Checkout my previous blog for more information on VyOS configuration here.
https://virtualrove.com/2020/04/30/vyos-virtual-router/
Then get into your 2nd VyOS router and run these commands.
set interfaces ethernet eth0 address 172.27.12.1/24
set interfaces ethernet eth1 mtu ‘9000’
set protocols bgp 65001 parameters router-id 172.27.12.1
set protocols bgp 65001 neighbor 172.27.12.2 update-source eth0
set protocols bgp 65001 neighbor 172.27.12.2 remote-as ‘65003’
set protocols bgp 65001 neighbor 172.27.12.3 remote-as ‘65003’
set protocols bgp 65001 neighbor 172.27.12.2 password VMw@re1!
set protocols bgp 65001 neighbor 172.27.12.3 password VMw@re1!
We are done with the BGP configuration on my both routers. Hit retry on cloud builder and you should pass that error.
Keep it mind that we also need to configure inter routing between these two VyOS.
VCF requires 2 TOR (VyOS in our case) to be in place for BGP peer. Let’s discuss about configuring inter-routing between 2 VyOS routers.
This is how interfaces on VyOS looks like…
VyOS1
Eth4 has the router id (172.27.11.1) which will be used as a 1st BGP peer by NSX-T env.
VyOS 2
Eth0 has the router id (172.27.12.1) which will be used as a 2nd BGP peer by NSX-T env.
To enable inter-routing between these two VyOS, I created a port group on my physical esxi called ‘Routable’ and attached both VyOS by adding one more nic card to it. Have a look at eth6 on VyoS 1 and eth1 on VyOS 2. Then added static route enable routing.
I was able to ping all networks of VyOS 1 from VyOS 2 after adding this static route.
We are done with routing at this stage. Lets go back to Cloud Builder and hit retry.
Next Error on Cloud Builder: ‘Failed to validate BGP route distribution.’
At this stage, routing has been configured in your NSX-T environment, both edges have been deployed and BGP peering has been done. If you check bgp peer information on edge as well as VyOS router, it will show ‘established’ and even routes from NSX-T environment appears on your VyOS router. Which means, route redistribution from NSX to VyOS works fine and this error means that there are no routes advertised from VyOS (TOR) to NSX environment. Let’s get into VyOS and run some commands.
set protocols bgp 65001 address-family ipv4-unicast network 172.16.31.0/24
set protocols bgp 65001 address-family ipv4-unicast network 172.16.32.0/24
I have advertised 1631 & 1632 vlan from VyoS to BGP 65001 network. Your network admin will decide on which networks to be published/advertised to NSX-T env.
And we are done..!!!
vCloud Foundation 4.0 with AVN has been installed and configured successfully.
To test the routing, I created a segment in NSX-T environment called ‘Accounts’ & CIDR as ’50.50.50.50/24’. I see this network populated on my TOR (VyOS router)
That’s it for this post. In my next post, we will deploy NSX-T edges and configure routing manually to get more understanding of the BGP routing in NSX-T environment.
Are you looking out for a lab to practice VMware products..? If yes, then click here to know more about our Lab-as-a-Service (LaaS).
Do subscribe to my posts if you like it.
Welcome to VirtualRove.com 