VIX | About VIX | Technology | BGP Communities
last change: July 22, 2019

Site-Specific BGP Communities

Theoretical background

As we try to use the VIX infrastructure in the best possible way, one motivation is to keep the traffic on the cross-site trunks as low as possible. The most efficient way to achieve that is to exchange all possible traffic site-local. Keeping the traffic local not only lowers the load on the cross-site trunks, but also delivers lower delay/jitter between peering routers and faster routing convergence in case of outages. The graphic below shows the ideal flow of payload when all VIX participants use this convention.

BGP


To ensure optimal results, all VIX participants should announce their prefixes with the appropriate site-specific community string. Even single-connected VIX participants should announce their prefixes with site-specific communities to give the multiconnected ISPs a chance to distinguish on their routers where the prefix comes from. Also those who cannot use others' community strings (for whatever reason) should announce them to the others to give them the chance to use them.

Dual-connected VIX participants must ensure that (also) their iBGP sessions are configured with "next-hop-self".

But keep in mind:

  • ISPs who also peer at other IXPs with same ASes as they peer at VIX might have to adapt this simple system to work properly.
  • ISPs who have the contractual need to accept unmodified metrics (MEDs) from their VIX peers might need to choose different ways of traffic engineering or are even unable to do so.
  • Do not use "local-preference" (to much weight shifting - e.g. overrules as-path length).

Technical implementation

To identify which announcements come from which site we have designed location-specific BGP communities. These communities are:

Site BGP Community
University of Vienna 1120:1
Interxion 1120:2
e-shelter 1120:3

Below you can see an example of a dual-connected ISP's BGP configuration (Cisco style - others appreciated, please mail to noc(at)vix.at). The configuration example can also be used by a single-connected participant.

Site: University of Vienna

router bgp <Your AS>
 neighbor VIX peer-group
 neighbor VIX version 4
 neighbor VIX next-hop-self
 neighbor VIX send-community
 neighbor VIX remove-private-AS
 neighbor VIX route-map from-VIX in
 neighbor VIX route-map to-VIX out

 neighbor 193.203.0.1 remote-as 1853
 neighbor 193.203.0.1 peer-group VIX
 neighbor 193.203.0.1 description #peering: ACOnet
 neighbor 193.203.0.25 remote-as 1120
neighbor 193.203.0.25 peer-group VIX
neighbor 193.203.0.25 description #peering: VIX Service AS

route-map from-VIX permit 10
 match community 11

route-map from-VIX permit 20
set metric +1

route-map to-VIX permit 10
 match <whatever you want to announce>
 set community 1120:1

ip community-list 11 permit 1120:1

Site: Interxion

router bgp <Your AS>
 neighbor VIX peer-group
 neighbor VIX version 4
 neighbor VIX next-hop-self
 neighbor VIX send-community
 neighbor VIX remove-private-AS
 neighbor VIX route-map from-VIX in
 neighbor VIX route-map to-VIX out

 neighbor 193.203.0.1 remote-as 1853
 neighbor 193.203.0.1 peer-group VIX
 neighbor 193.203.0.1 description #peering: ACOnet
 neighbor 193.203.0.25 remote-as 1120
neighbor 193.203.0.25 peer-group VIX
neighbor 193.203.0.25 description #peering: VIX Service AS

route-map from-VIX permit 10
 match community 12

route-map from-VIX permit 20
set metric +1

route-map to-VIX permit 10
 match <whatever you want to announce>
 set community 1120:2

ip community-list 12 permit 1120:2

Site: e-shelter

router bgp <Your AS>
 neighbor VIX peer-group
 neighbor VIX version 4
 neighbor VIX next-hop-self
 neighbor VIX send-community
 neighbor VIX remove-private-AS
 neighbor VIX route-map from-VIX in
 neighbor VIX route-map to-VIX out

 neighbor 193.203.0.1 remote-as 1853
 neighbor 193.203.0.1 peer-group VIX
 neighbor 193.203.0.1 description #peering: ACOnet
 neighbor 193.203.0.25 remote-as 1120
neighbor 193.203.0.25 peer-group VIX
neighbor 193.203.0.25 description #peering: VIX Service AS

route-map from-VIX permit 10
 match community 13

route-map from-VIX permit 20
set metric +1

route-map to-VIX permit 10
 match <whatever you want to announce>
 set community 1120:3

ip community-list 13 permit 1120:3