Implementation for site specific community configuration
Theoretical background
As we try to use the VIX infrastructure in the best possible way, one motivation is to keep the traffic of the cross-site trunks as low as possible. The most efficient way to reduce load on the cross-site Trunks is to exchange all possible traffic site-local. Keeping the traffic local not only lowers the load on the cross-site trunk, but also delivers lower delay/jitter between peering routers and faster routing convergence in case of outages. The picture below shows the ideal flow of payload when all members use this convention.
To ensure the optimal results all VIX-members should announce their prefixes with the appropriate site specific community string. Even singleconnected VIX members should announce their prefixes with site specific communities to give the multiconnected ISPs a chance to distinguish on their routers where the prefix comes from. Also those who cannot use others community strings (for what ever reason) should announce them to the others to give them the chance to use it.
Dual connected VIX members have to ensure that (also) their iBGP sessions are configured with "next-hop-self".
But keep in mind:
- ISPs who also peer at other IXPs with same ASes as they peer at VIX might have to adopt this simple system to work properly
- ISPs who have the contractual need of accepting unmodified metrics (MEDs) from their VIX-peers might have to choose different ways of traffic engineering or are even unable to do so
- DO NOT USE LOCAL-PREFERENCE (to much weight shifting, for example overrules as-path length)
Technical implementation
To identify which announcements come from which site we designed location specific communities.
These were
Site | Community |
Vienna University | 1120:1 |
Interxion | 1120:2 |
Here you can see a configuration example of a dual connected ISPs BGP config (Cisco style, others appreciated, please mail to noc(at)vix.at). The configuration example can also be used by a single connected customer.
Site @ Univie
router bgp <Your AS>
neighbor VIX peer-group
neighbor VIX version 4
neighbor VIX next-hop-self
neighbor VIX send-community
neighbor VIX remove-private-AS
neighbor VIX route-map from-VIX in
neighbor VIX route-map to-VIX out
neighbor 193.203.0.1 remote-as 1853
neighbor 193.203.0.1 peer-group VIX
neighbor 193.203.0.1 description #peering: ACOnet
neighbor 193.203.0.25 remote-as 1120
neighbor 193.203.0.25 peer-group VIX
neighbor 193.203.0.25 description #peering: VIX Service AS
route-map from-VIX permit 10
match community 11
route-map from-VIX permit 20
set metric +1
route-map to-VIX permit 10
match <whatever you want to announce>
set community 1120:1
ip community-list 11 permit 1120:1Site @ Interxion
router bgp <Your AS>
neighbor VIX peer-group
neighbor VIX version 4
neighbor VIX next-hop-self
neighbor VIX send-community
neighbor VIX remove-private-AS
neighbor VIX route-map from-VIX in
neighbor VIX route-map to-VIX out
neighbor 193.203.0.1 remote-as 1853
neighbor 193.203.0.1 peer-group VIX
neighbor 193.203.0.1 description #peering: ACOnet
neighbor 193.203.0.25 remote-as 1120
neighbor 193.203.0.25 peer-group VIX
neighbor 193.203.0.25 description #peering: VIX Service AS
route-map from-VIX permit 10
match community 12
route-map from-VIX permit 20
set metric +1
route-map to-VIX permit 10
match <whatever you want to announce>
set community 1120:2
ip community-list 12 permit 1120:2