NetFPGA Tsinghua Tutorial May 15 -16 2010 1 S T A N F O R D U N I V E R S I T Y

NetFPGA TutorialTsinghua University – Day 2

Presented by: James Hongyi Zeng

(Stanford University)

Joshua Lu(Xilinx China)

Beijing, ChinaMay 15 - 16, 2010

http://NetFPGA.org

NetFPGA Tsinghua Tutorial May 15 -16 2010 2 S T A N F O R D U N I V E R S I T Y

Outline

• Tree Structure

• Develop a cryptography module– Quick overview of XOR “cryptography”– Implement crypto module– Write software simulations– Synthesize– Write hardware tests

NetFPGA Tsinghua Tutorial May 15 -16 2010 3 S T A N F O R D U N I V E R S I T Y

Tree Structure

NF2

bin

lib

projects

bitfiles

(scripts for running simulations and setting up the environment)

(contains the bitfiles for all projects that have been synthesized)

(stable common modules and common parts needed for simulation/synthesis/design)

(user projects, including reference designs)

NetFPGA Tsinghua Tutorial May 15 -16 2010 4 S T A N F O R D U N I V E R S I T Y

Tree Structure (2)

lib

C

java

Makefiles

Perl5

python

scriptsverilog

(common software and code for reference designs)

(contains software for the graphical user interface)

(makefiles for simulation and synthesis)

(common libraries to interact with reference designs and aid in simulation)

(common libraries to aid in regression tests)

(scripts for common functions)

(modules and files that can be reused for design)

NetFPGA Tsinghua Tutorial May 15 -16 2010 5 S T A N F O R D U N I V E R S I T Y

Tree Structure (3)

projects

doc

include

regress

src

sw

synthverif

(project specific documentation)

(contains file to include verilog modules from lib, and creates project specific register defines files)

(regression tests used to test generated bitfiles)

(contains non-library verilog code used for synthesis and simulation)

(all software parts of the project)

(contains user .xco files to generate cores and Makefile to implement the design)

(simulation tests)

NetFPGA Tsinghua Tutorial May 15 -16 2010 6 S T A N F O R D U N I V E R S I T Y

Cryptography

• Simple cryptography – XOR

A B A ^ B

0 0 0

0 1 1

1 0 1

1 1 0

NetFPGA Tsinghua Tutorial May 15 -16 2010 7 S T A N F O R D U N I V E R S I T Y

Cryptography (cont.)

• Example:

• Explanation:– A ^ A = 0– So, M ^ K ^ K = M ^ 0 = M

Message: 00111011

Key: 10110001

Message ^ Key: 10001010

Message ^ Key ^ Key: 00111011

NetFPGA Tsinghua Tutorial May 15 -16 2010 8 S T A N F O R D U N I V E R S I T Y

Implementing a Crypto Module (1)

• What do we want to encrypt?– IP payload only

• Plaintext IP header allows routing• Content is hidden

– Encrypt bytes 35 onward• Bytes 1-14 – Ethernet header• Bytes 15-34 – IPv4 header (assume no options)

– Assume all packets are IPv4 for simplicity

NetFPGA Tsinghua Tutorial May 15 -16 2010 9 S T A N F O R D U N I V E R S I T Y

Implementing a Crypto Module (2)

• State machine (draw on next page):– Module headers on each packet– Datapath 64-bits wide

• 34 / 8 is not an integer! • Inside the crypto module

NetFPGA Tsinghua Tutorial May 15 -16 2010 10 S T A N F O R D U N I V E R S I T Y

Crypto Module State Diagram

Hint: We suggest 4 states (or 3 if you’re feeling adventurous)

SkipModuleHeaders

NetFPGA Tsinghua Tutorial May 15 -16 2010 11 S T A N F O R D U N I V E R S I T Y

State Diagram to Verilog (1)

Module location

1. Crypto module to encrypt and decrypt packets

MACRxQ

CPURxQ

MACRxQ

CPURxQ

MACRxQ

CPURxQ

MACRxQ

CPURxQ

Input Arbiter

Output Port Lookup

MACTxQ

CPUTxQ

MACTxQ

CPUTxQ

MACTxQ

CPUTxQ

MACTxQ

CPUTxQ

Output Queues

Crypto

NetFPGA Tsinghua Tutorial May 15 -16 2010 12 S T A N F O R D U N I V E R S I T Y

Inter-module Communication

Module i+1

Module i

data

ctrlwrrdy

NetFPGA Tsinghua Tutorial May 15 -16 2010 13 S T A N F O R D U N I V E R S I T Y

State Diagram to Verilog (2)

• Projects:– Each design represented by a project

Format: NF2/projects/<proj_name>

– NF2/projects/crypto_nic

– Consists of:

– Missing:

• Verilog source• Simulation tests• Hardware tests

• Libraries• Optional software

• State diagram implementation• Simulation tests• Regression tests

NetFPGA Tsinghua Tutorial May 15 -16 2010 14 S T A N F O R D U N I V E R S I T Y

State Diagram to Verilog (3)

• Projects (cont):– Pull in modules from NF2/lib/verilog

• Generic modules that are re-used in multiple projects

• Specify shared modules in project’s include/lib_modules.txt

– Local src modules override shared modules

– crypto_nic:• Local user_data_path.v, crypto.v• Everything else: shared modules

NetFPGA Tsinghua Tutorial May 15 -16 2010 15 S T A N F O R D U N I V E R S I T Y

State Diagram to Verilog (4)

• Your task:

1. Copy NF2/lib/verilog/module_template/src/module_template.v to NF2/projects/crypto_nic/src/crypto.v

2. Implement your state diagram in src/crypto.v– Small fallthrough FIFO– Generic register interface– Registers to be used defined in include/crypto_defines.v

NetFPGA Tsinghua Tutorial May 15 -16 2010 16 S T A N F O R D U N I V E R S I T Y

Generic Registers Modulegeneric_regs # ( .UDP_REG_SRC_WIDTH (UDP_REG_SRC_WIDTH), .TAG (`CRYPTO_BLOCK_TAG), .REG_ADDR_WIDTH (`CRYPTO_REG_ADDR_WIDTH),

.NUM_COUNTERS (0), .NUM_SOFTWARE_REGS (1), .NUM_HARDWARE_REGS (0)) crypto_regs ( .reg_req_in (reg_req_in), … .reg_src_out (reg_src_out), … .software_regs (key), .hardware_regs (), …

NetFPGA Tsinghua Tutorial May 15 -16 2010 17 S T A N F O R D U N I V E R S I T Y

First Break

(While writing Verilog modules)

NetFPGA Tsinghua Tutorial May 15 -16 2010 18 S T A N F O R D U N I V E R S I T Y

Testing: Simulation (1)

• Simulation allows testing without requiring lengthy synthesis process

• NetFPGA provides Perl simulation infrastructure to:– Send/receive packets

• Physical ports and CPU– Read/write registers– Verify results

• Simulations run in ModelSim/VCS

NetFPGA Tsinghua Tutorial May 15 -16 2010 19 S T A N F O R D U N I V E R S I T Y

Testing: Simulation (2)

• Simulations located in project/verif• Multiple simulations per project

– Test different features• Example:

– crypto_nic/verif/test_nic_short• Send one packet from CPU, expect packet out

physical port• Send one packet in physical port, expect packet to

CPU

NetFPGA Tsinghua Tutorial May 15 -16 2010 20 S T A N F O R D U N I V E R S I T Y

Testing: Simulation (3)

• Useful functions:– nf_PCI_read32(delay, batch, addr, expect)– nf_PCI_write32(delay, batch, addr, value)– nf_packet_in(port, length, delay, batch, pkt)– nf_expected_packet(port, length, pkt)– nf_dma_data_in(length, delay, port, pkt)– nf_expected_dma_data(port, length, pkt)– make_IP_pkt(length, da, sa, ttl, dst_ip, src_ip)– encrypt_pkt(key, pkt)– decrypt_pkt(key, pkt)

NetFPGA Tsinghua Tutorial May 15 -16 2010 21 S T A N F O R D U N I V E R S I T Y

Testing: Simulation (4)

• Your task:

1. Template files NF2/projects/crypto_nic/verif/test_crypto_encrypt/make_pkts.pl NF2/projects/crypto_nic/verif/test_crypto_decrypt/make_pkts.pl

2. Implement your Perl verif tests– Use the example verif test (test_nic_short)

NetFPGA Tsinghua Tutorial May 15 -16 2010 22 S T A N F O R D U N I V E R S I T Y

Running Simulations

• Use command nf21_run_test.pl– Optional parameters

• --major <major_name>• --minor <minor_name>• --gui (starts the default viewing environment)

test_crypto_encrypt

• Set env. variables to reference your project• NF2_DESIGN_DIR=/root/NF2/projects/<project>• PERL5LIB=/root/NF2/projects/<project>/lib/Perl5: /root/NF2/lib/Perl5:

major minor

NetFPGA Tsinghua Tutorial May 15 -16 2010 23 S T A N F O R D U N I V E R S I T Y

Running Simulations

• When running modelsim interactively:– Click "no" when simulator prompts to finish

– Changes to code can be recompiled without quitting ModelSim:

• bash# cd /tmp/$(whoami)/verif/<projname>; make model_sim• VSIM 5> restart -f; run -a

– Do ensure $NF2_DESIGN_DIR is correct

NetFPGA Tsinghua Tutorial May 15 -16 2010 24 S T A N F O R D U N I V E R S I T Y

Synthesis

• To synthesize your project– Run make in the synth directory

(NF2/projects/crypto_nic/synth)

NetFPGA Tsinghua Tutorial May 15 -16 2010 25 S T A N F O R D U N I V E R S I T Y

Second Break

(while hardware compiles)

NetFPGA Tsinghua Tutorial May 15 -16 2010 26 S T A N F O R D U N I V E R S I T Y

Regression Tests

• Test hardware module

• Perl Infrastructure provided to – Read/Write registers– Read/Write tables– Send Packets– Check Counters

NetFPGA Tsinghua Tutorial May 15 -16 2010 27 S T A N F O R D U N I V E R S I T Y

Example Regression Tests

• Reference Router – Send Packets from CPU– Longest Prefix Matching– Longest Prefix Matching Misses– Packets dropped when queues overflow– Receiving Packets with IP TTL <= 1– Receiving Packets with IP options or non IPv4– Packet Forwarding– Dropping packets with bad IP Checksum

NetFPGA Tsinghua Tutorial May 15 -16 2010 28 S T A N F O R D U N I V E R S I T Y

Perl Libraries• Specify the Interfaces

– eth1, eth2, nf2c0 … nf2c3

• Start packet capture on Interfaces

• Create Packets– MAC header– IP header– PDU

• Read/Write Registers

• Read/Write Reference Router tables– Longest Prefix Match– ARP– Destination IP Filter

NetFPGA Tsinghua Tutorial May 15 -16 2010 29 S T A N F O R D U N I V E R S I T Y

Regression Test Examples

• Reference Router– Packet Forwarding

• regress/test_packet_forwarding– Longest Prefix Match

• regress/test_lpm– Send and Receive

• regress/test_send_rec

NetFPGA Tsinghua Tutorial May 15 -16 2010 30 S T A N F O R D U N I V E R S I T Y

Creating a Regression Test

• Useful functions:– nftest_regwrite(interface, addr, value)– nftest_regread(interface, addr)– nftest_send(interface, frame)– nftest_expect(interface, frame)– encrypt_pkt(key, pkt)– decrypt_pkt(key, pkt)

– $pkt = NF2::IP_pkt->new(len => $length,            DA => $DA, SA => $SA,            ttl => $TTL, dst_ip => $dst_ip,            src_ip => $src_ip);

NetFPGA Tsinghua Tutorial May 15 -16 2010 31 S T A N F O R D U N I V E R S I T Y

Creating a Regression Test (2)

• Your task:

1. Template files NF2/projects/crypto_nic/regress/test_crypto_encrypt/run

2. Implement your Perl verif tests

NetFPGA Tsinghua Tutorial May 15 -16 2010 32 S T A N F O R D U N I V E R S I T Y

Running Regression Test

• Run the commandnf2_regress_test.pl --project crypto_nic

NetFPGA Tsinghua Tutorial May 15 -16 2010 33 S T A N F O R D U N I V E R S I T Y

Third Break

(while testing hardware)

NetFPGA Tsinghua Tutorial May 15 -16 2010 34 S T A N F O R D U N I V E R S I T Y

The New 2.0 Release

• Modular Registers– Simplifies integration of multiple modules

• Many users control NetFPGAs from software– Register set joined together at build time

• Project specifies registers in XML list

• Packet Buffering in DRAM– Supports Deep buffering

• Single 64MByte queue in DDR2 memory

• Programmable Packet Encapsulation– Packet-in-packet encapsulation

• Enables tunnels between OpenFlowSwitch nodes

NetFPGA Tsinghua Tutorial May 15 -16 2010 35 S T A N F O R D U N I V E R S I T Y

NetFPGA.org

NetFPGA Tsinghua Tutorial May 15 -16 2010 36 S T A N F O R D U N I V E R S I T Y

Project Ideas for the NetFPGA• IPv6 Router (in high demand)• TCP Traffic Generator• Valiant Load Balancing • Graphical User Interface (like CLACK)• MAC-in-MAC Encapsulation• Encryption / Decryption modules• RCP Transport Protocol • Packet Filtering ( Firewall, IDS, IDP )• TCP Offload Engine• DRAM Packet Queues• 8-Port Switch using SATA Bridge• Build our own MAC (from source, rather than core) • Use XML for Register Definitions

http://www.netfpga.org/foswiki/NetFPGA/OneGig/ModuleWishlist

NetFPGA Tsinghua Tutorial May 15 -16 2010 37 S T A N F O R D U N I V E R S I T Y

NetFPGA DesignsProject (Title & Summary) Base Status Organization Docs.

IPv4 Reference Router 2.0 Functional Stanford University Guide Quad-Port Gigabit NIC 2.0 Functional Stanford University Guide Ethernet Switch 2.0 Functional Stanford University Guide Hardware-Accelerated Linux Router 2.0 Functional Stanford University Guide Packet Generator 2.0 Functional Stanford University Wiki OpenFlow Switch 2.0 Functional Stanford University Wiki DRAM-Router 2.0 Functional Stanford University Wiki NetFlow Probe 1.2 Functional Brno University Wiki AirFPGA 2.0 Functional Stanford University Wiki Fast Reroute & Multipath Router 2.0 Functional Stanford University Wiki NetThreads 1.2.5 Functional University of Toronto Wiki

URL Extraction 2.0 Functional Univ. of New South Wales Wiki

zFilter Sprouter (Pub/Sub) 1.2 Functional Ericsson Wiki Windows Driver 2.0 Functional Microsoft Research Wiki IP Lookup w/Blooming Tree 1.2.5 In Progress University of Pisa Wiki DFA 2.0 In Progress UMass Lowell Wiki G/PaX  ?.? In Progress Xilinx Wiki Precise Traffic Generator 1.2.5 In Progress University of Toronto Wiki Open Network Lab 2.0 In Progress Washington University Wiki KOREN Testbed  ?.? In Progress Chungnam-Korea Wiki RED 2.0 In Progress Stanford University Wiki Virtual Data Plane 1.2 In Progress Georgia Tech Wiki Precise Time Protocol (PTP) 2.0 In Progress Stanford University Wiki Deficit Round Robin (DRR) 1.2 Repackage Stanford University Wiki

.. And more on http://netfpga.org/foswiki/NetFPGA/OneGig/ProjectTable

NetFPGA Tsinghua Tutorial May 15 -16 2010 38 S T A N F O R D U N I V E R S I T Y

Group Discussion• Your plans for using the NetFPGA

– Teaching– Research– Other

• Resources needed for your class– Source code– Courseware– Examples

• Your plans to contribute– Expertise – Capabilities– Collaboration Opportunities

NetFPGA Tsinghua Tutorial May 15 -16 2010 39 S T A N F O R D U N I V E R S I T Y

NetFPGA Developers Workshops• You already know that the NetFPGA implements a Gigabit NIC, a hardware-accelerated Internet

router, a traffic generator, an OpenFlow switch, a NetFlow probe and more. What else can it do? We invite you to contribute a NetFPGA project and present your work at a NetFPGA workshop

• NetFPGA Events: – http://netfpga.org/php/events.php

• 2010 Locations: – USA (Stanford, California)– Europe (UK)– Asia (Korea)

• Format– Presentations– Demonstrations

• Example– http://NetFPGA.org/DevWorkshop

“What can you built with your NetFPGA?”

NetFPGA Tsinghua Tutorial May 15 -16 2010 40 S T A N F O R D U N I V E R S I T Y

• The contest is split into two challenges. Teams can participate in either or both challenges. Design challenges will be posted on the start date of the contest. The design teams have 120 days to produce a working implementation employing any HW and SW design methodology and targeting the NetFPGA development platform.

• Dates– Contest starts on Feb 9th 2010– Contest ends on June 1st 2010.

• Challenge 1: The Best Network Tester/Packet Capture system – There are a small number of very expensive packet generator and capture systems on

the market, used for testing networks and network equipment. The goal of this design is to provide a usable, powerful and open-source alternative for use by universities and organizations unable to afford such expensive equipment.

• Challenge 2: The Best Overall Network System Design – The goal is to design and implement a novel design on the NetFPGA system. Use your

imagination to devise a new use case for the NetFPGA. We are looking for solutions utilizing the NetFPGA cards that perform significant networking functionality.

• Rules and more Information– http://www.netfpga.org/foswiki/NetFPGA/OneGig/DesignContest2010

NetFPGA Design Contest

NetFPGA Tsinghua Tutorial May 15 -16 2010 41 S T A N F O R D U N I V E R S I T Y

Thoughts for Developers

• Build Modular components– Describe shared registers (as per 2.0 release)– Consider how modules would be used in larger systems

• Define functionality clearly – Through regression tests– With repeatable results

• Disseminate projects– Post open-source code– Document projects on Web, Wiki

• Expand the community of developers– Answer questions in the Discussion Forum – Collaborate with your peers to build new applications

NetFPGA Tsinghua Tutorial May 15 -16 2010 42 S T A N F O R D U N I V E R S I T Y

Visit http://NetFPGA.org

NetFPGA Tsinghua Tutorial May 15 -16 2010 43 S T A N F O R D U N I V E R S I T Y

Join the NetFPGA.org Community

• Log into the Wiki

• Access the Beta code

• Join the netfpga-beta mailing list

• Join the discussion forum

NetFPGA Tsinghua Tutorial May 15 -16 2010 44 S T A N F O R D U N I V E R S I T Y

Contribute to the Project

• Search for related work

• List your project on the Wiki

• Link your project homepage

NetFPGA Tsinghua Tutorial May 15 -16 2010 45 S T A N F O R D U N I V E R S I T Y

Survey

• How did you like this this tutorial?– What did you find useful?– What should be improved?– What should be removed?– What should be added?

• Can we post the video from this event?– If not, please let us know.

• Complete On-line survey– http://netfpga.org/tutorial_survey.html

NetFPGA Tsinghua Tutorial May 15 -16 2010 46 S T A N F O R D U N I V E R S I T Y

Acknowledgments

NetFPGA Team at Stanford University (Past and Present):

Nick McKeown, Glen Gibb, Jad Naous, David Erickson, G. Adam Covington, John W. Lockwood, Jianying Luo, Brandon Heller,

Paul Hartke, Neda Beheshti, Sara Bolouki, James Zeng, Jonathan Ellithorpe, Sachidanandan Sambandan

NetFPGA Tsinghua Tutorial May 15 -16 2010 47 S T A N F O R D U N I V E R S I T Y

Special thanks to:

Other NetFPGA Tutorials Presented At:

SIGMETRICS

Patrick Lysaght, Veena Kumar, Paul Hartke, Anna AcevedoXilinx University Program (XUP)

See: http://NetFPGA.org/tutorials/

NetFPGA Tsinghua Tutorial May 15 -16 2010 48 S T A N F O R D U N I V E R S I T Y

Acknowledgments• Support for the NetFPGA project has been provided

by the following companies and institutions

Disclaimer: Any opinions, findings, conclusions, or recommendations expressed in these materials do not necessarily reflect the views of the National Science Foundation or of any other sponsors supporting this project.