IPv6 Only Data Centers in Yandex

Nikita V. Shirokov, network engineer, AS13238

Before we start

«How we launched IPv6 in Yandex» by Vladimir Ivanov

Two networks:

〉 Backbone (user's traffic handling)

〉 Technological network (so called «Fastbone»; bulk traffic)

!

https://tech.yandex.ru/events/yac/2012/talks/380/

3

Update on external IPv6 connectivity

mx.yandex.ru

4

mc.yandex.ru

Why we decided to build IPv6 only DC

〉We ran out of public v4 addresses (and almost ran out of 1918)

〉We think that public v6 is better that CGN for InterDC connectivity

5

Points of interest

〉WAN

〉 Firewalls and SLBs

〉 Intra DC Network (rack to rack etc)

〉 Services

6

WAN

IPv6 Ready

〉MPLS 6VPE for Intra Project traffic

〉 Native IPv6 for the rest

7

Firewalls

8

Current: optimized IPv6 kernel code and fw!

Future: Netmap based solution!

SLBs

9

WAN

L2 DC Fabric

SLBs(cont)

10

Issues:No checks inside tunnels

No 6over4 in LVS

Someone needs do

decapsulateL3 DC Fabric

WAN

Issues:

– No checks inside tunnels

– No 6over4 in LVS

– Someone needs do

– decapsulate

11

WAN

L3 DC Fabric

Our initial design for IPv6 only DC

12

BackboneFastbone

Services

Most of the services IPv6 ready since 2011+ (or so)

But:

〉 It’s easy to forget about monitoring

〉 or cluster’s management

〉 or replication’s transport

〉 or bootstrapping

13

Deploy. Phase 1

14

BackboneFastbone# show ipv6 bgp summary vrf …

BGP summary information for VRF …, address family IPv6 Unicast

BGP router identifier …, local AS number 65400

BGP table version is 288725, IPv6 Unicast config peers 210, capable peers 210

370 network entries and 370 paths using 45880 bytes of memory

BGP attribute entries [9/1224], BGP AS path entries [2/16]

BGP community entries [2/64], BGP clusterlist entries [0/0]

Not only IPv6 DC

RND: (or «I need something from github!111»)

〉 6to4 Nat. No need for high performance

Non Realtime servers needs to dl something from internet(such as webrobots/spiders etc)

〉 4over6 tunneling or Dedicated racks with /26 IPv4 subnets

〉 Future: high performance 6to4 Netmap bassed NAT/FW

15

Not only IPv6 DC

!

Realtime servers needs to dl/query something from internet (such as public dns resolvers etc)

〉 4over6 tunneling

〉 Future: high performance 6to4 Netmap bassed NAT/FW

16

Questions?

18

Nikita V. Shirokov

Network engineer, AS13238

Contacts

tehnerd@yandex-team.ru

Additional Slides:

19

WAN

L2 DC Fabric

SLB check:

http get from 10.x.x.x to 10.x.x.y

external IP to VIP

external IP to 10.x.x.y

PBR: from 10.x.x.y thru 10.x.x.x

Additional Slides:

20

WAN

L3 DC Fabric

external IP to VIP

external IP to VIP encapsulated into SLB to Server

we cant http get on VIP @ SLB (will go to local IP @ loopback). therefore cant check if this address exist @ Server