VIX | About VIX | Technology | BGP Community
last change: August 21, 2018

Site Specific Community Configuration

Theoretical background

As we try to use the VIX infrastructure in the best possible way, one motivation is to keep the traffic on the cross-site trunks as low as possible. The most efficient way to reduce load on the cross-site trunks is to exchange all possible traffic site-local. Keeping the traffic local not only lowers the load on the cross-site trunk, but also delivers lower delay/jitter between peering routers and faster routing convergence in case of outages. The graphic below shows the ideal flow of payload when all members use this convention.

BGP

To ensure optimal results, all VIX-members should announce their prefixes with the appropriate site specific community string. Even singleconnected VIX members should announce their prefixes with site specific communities to give the multiconnected ISPs a chance to distinguish on their routers where the prefix comes from. Also those who cannot use others' community strings (for whatever reason) should announce them to the others to give them the chance to use them.

Dual connected VIX members must ensure that (also) their iBGP sessions are configured with "next-hop-self".

But keep in mind:

  • ISPs who also peer at other IXPs with same ASes as they peer at VIX might have to adapt this simple system to work properly
  • ISPs who have the contractual need to accept unmodified metrics (MEDs) from their VIX-peers might need to choose different ways of traffic engineering or are even unable to do so
  • DO NOT USE LOCAL-PREFERENCE (to much weight shifting, for example overrules as-path length)

Technical implementation

To identify which announcements come from which site we designed location specific communities.

These were

Site Community
University of Vienna 1120:1
Interxion 1120:2
e-shelter 1120:3

Here you can see a configuration example of a dual connected ISPs BGP config (Cisco style, others appreciated, please mail to noc(at)vix.at). The configuration example can also be used by a single connected customer.

Site @ Univie

router bgp <Your AS>
 neighbor VIX peer-group
 neighbor VIX version 4
 neighbor VIX next-hop-self
 neighbor VIX send-community
 neighbor VIX remove-private-AS
 neighbor VIX route-map from-VIX in
 neighbor VIX route-map to-VIX out

 neighbor 193.203.0.1 remote-as 1853
 neighbor 193.203.0.1 peer-group VIX
 neighbor 193.203.0.1 description #peering: ACOnet
 neighbor 193.203.0.25 remote-as 1120
neighbor 193.203.0.25 peer-group VIX
neighbor 193.203.0.25 description #peering: VIX Service AS

route-map from-VIX permit 10
 match community 11

route-map from-VIX permit 20
set metric +1

route-map to-VIX permit 10
 match <whatever you want to announce>
 set community 1120:1

ip community-list 11 permit 1120:1

Site @ Interxion

router bgp <Your AS>
 neighbor VIX peer-group
 neighbor VIX version 4
 neighbor VIX next-hop-self
 neighbor VIX send-community
 neighbor VIX remove-private-AS
 neighbor VIX route-map from-VIX in
 neighbor VIX route-map to-VIX out

 neighbor 193.203.0.1 remote-as 1853
 neighbor 193.203.0.1 peer-group VIX
 neighbor 193.203.0.1 description #peering: ACOnet
 neighbor 193.203.0.25 remote-as 1120
neighbor 193.203.0.25 peer-group VIX
neighbor 193.203.0.25 description #peering: VIX Service AS

route-map from-VIX permit 10
 match community 12

route-map from-VIX permit 20
set metric +1

route-map to-VIX permit 10
 match <whatever you want to announce>
 set community 1120:2

ip community-list 12 permit 1120:2

Site @ e-shelter

router bgp <Your AS>
 neighbor VIX peer-group
 neighbor VIX version 4
 neighbor VIX next-hop-self
 neighbor VIX send-community
 neighbor VIX remove-private-AS
 neighbor VIX route-map from-VIX in
 neighbor VIX route-map to-VIX out

 neighbor 193.203.0.1 remote-as 1853
 neighbor 193.203.0.1 peer-group VIX
 neighbor 193.203.0.1 description #peering: ACOnet
 neighbor 193.203.0.25 remote-as 1120
neighbor 193.203.0.25 peer-group VIX
neighbor 193.203.0.25 description #peering: VIX Service AS

route-map from-VIX permit 10
 match community 13

route-map from-VIX permit 20
set metric +1

route-map to-VIX permit 10
 match <whatever you want to announce>
 set community 1120:3

ip community-list 13 permit 1120:3