Porting NetBSD to the LatticeMico32 open source CPU by Yann Sionneau
82
Por$ng NetBSD on the open source La5ceMico32 CPU Yann Sionneau MLabs @ EuroBSDCon 2014
-
Upload
eurobsdcon -
Category
Technology
-
view
74 -
download
0
Transcript of Porting NetBSD to the LatticeMico32 open source CPU by Yann Sionneau
Por$ng NetBSD on
the open source La5ceMico32 CPU
Yann Sionneau M-‐Labs
@ EuroBSDCon 2014
About me
• Yann Sionneau • Embedded soGware developer • Working for M-‐Labs – hOp://m-‐labs.hk
• @yannsionneau on twiOer • Email: ys@m-‐labs.hk
I’m going to talk about…
My por$ng of NetBSD and EdgeBSD on the Milkymist One board
Agenda
• I) The hardware part: the MMU – How La5ceMico32 MMU works
• II) The soGware part – My journey por$ng NetBSD to a new CPU
Milkymist One?!
Milkymist One?!
Milkymist One?!
The Milkymist One uses an FPGA
What’s an FPGA??
• A chip
FPGA internals
Milkymist System-‐on-‐Chip
La5ceMico32 CPU
• 32 bits Harvard Architecture RISC • Big Endian • 6 stages • Fully bypassed • Op$onal configurable I/D caches – Direct mapped or – 2-‐way set associa$ve
• Wishbone on-‐chip bus
Works on …
• Everything actually : device agnos$c
DE0-‐Nano
Milkymist One
Papilio Pro
Mixxeo
KC705
La5ceMico32 , Good points
• Small • Portable (works with several FPGA vendors) • Fast (~100 MHz on Slowtanpartan 6, 125 MHz on Kintex-‐7, up to 200 MHz without DDR3)
• Actually works • GCC/Binu$ls/GDB/Qemu/uCLinux/OpenWRT support
• OPEN SOURCE
La5ceMico32, Bad points
• No Memory Management Unit… yet!
La5ceMico32, Bad points
• No Memory Management Unit… yet! Done þ
Used in…
• Closed source commercial ASICs • Open source projects
• Can achieve 800 MHz in TSMC 90nm standard cell process
La5ceMico32 pipeline
La5ceMico32 pipeline?
La5ceMico32 pipeline?
Main Memory
CPU Internal
Before
PHYSICAL
ADDRESS
PHYSICAL
ADDRESS
PA
PA
Main Memory
CPU Internal
Raising excep$on
AGer
VIRTUAL ADDRESSES PHYSICAL ADDRESSES
What’s the MMU’s job?
• Translate « virtual addresses » into « physical addresses »
• Memory protecGon against unwanted execu$on of code or data write (e.g. soGware bug or security issue) – Memory right access management
Main Memory
CPU pipeline VA PA
VA : Virtual Address PA : Physical Address
Main Memory
CPU pipeline VA PA
VA : Virtual Address PA : Physical Address
How does the MMU know the VA-‐>PA transla$on ?
Main Memory
CPU pipeline VA PA
VA : Virtual Address PA : Physical Address
Page Table
Main Memory
CPU pipeline VA PA
VA : Virtual Address PA : Physical Address
Page Table Why « PAGE »?
Why « Page »?
• 0x00000004 -‐> 0x10000000 • 0x00000005 -‐> 0x10000001 • 0x00000006 -‐> 0x10000002 Etc…
Why « Page »?
• 0x00000004 -‐> 0x10000000 • 0x00000005 -‐> 0x10000001 • 0x00000006 -‐> 0x10000002 Etc…
This is WRONG!!!
Why « Page »?
• 0x00000*** -‐> 0x10000*** • 0x00001*** -‐> 0x10001*** • 0x00002*** -‐> 0x10002*** Etc…
Main Memory
CPU pipeline VA PA
VA : Virtual Address PA : Physical Address
Page Table
Main Memory
CPU pipeline VA PA
VA : Virtual Address PA : Physical Address
Page Table
TLB TLB : Transla$on Lookaside Buffer
Main Memory
CPU pipeline VA PA
VA : Virtual Address PA : Physical Address
Page Table
TLB OperaGng System
Updates the
Gets informa$on from the
Updates the
Features?
• Page size – Only 4 kB
32 bits physical address : xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx
How many bits of an address indicate the offset within a given page?
Features?
• Page size – Only 4 kB
32 bits physical address : xxxxxxxx xxxxxxxx xxxx xxxx xxxxxxxx
Page number [31:12]
20 bits
Offset [11:0]
12 bits
Features?
• 2 TLB (Transla$on Lookaside Buffer) – ITLB – DTLB
• Each TLB contains 1024 entries – How many bits needed to index the TLB?
Features?
• 2 TLB (Transla$on Lookaside Buffer) – ITLB – DTLB
• Each TLB contains 1024 entries – How many bits needed to index the TLB?
10 bits!
Features?
• No hardware page-‐tree walker – i.e. TLB is soGware assisted
Virtual address
Load or store?
Instruc$on or Data?
Physical address
Access granted/denied
Virtual address
Load or store?
Instruc$on or Data?
Physical address
Access granted/denied
I don’t know!
Let’s have a look inside
Line index Tag [10] Physical page number [20] Read-‐only [1]
Valid [1]
0 0xABC 0xABC00 0 0
1 0x280 0xB0001 1 1
2 0x300 0x00001 0 1
The TLB VA = 0xA0001004
Line index Tag [10] Physical page number [20] Read-‐only [1]
Valid [1]
0 0xABC 0xABC00 0 0
1 0x280 0xB0001 1 1
2 0x300 0x00001 0 1
The TLB VA = 0xA0001 004
Page number
Offset in the page
Line index Tag [10] Physical page number [20] Read-‐only [1]
Valid [1]
0 0xABC 0xABC00 0 0
1 0x280 0xB0001 1 1
2 0x300 0x00001 0 1
The TLB VA = 0xA0001 004 Page offset = 4
Line index Tag [10] Physical page number [20] Read-‐only [1]
Valid [1]
0 0xABC 0xABC00 0 0
1 0x280 0xB0001 1 1
2 0x300 0x00001 0 1
The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001
Line index Tag [10] Physical page number [20] Read-‐only [1]
Valid [1]
0 0xABC 0xABC00 0 0
1 0x280 0xB0001 1 1
2 0x300 0x00001 0 1
The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001
VPN = 0xA0001 à 1010 0000 0000 0000 0001
Line index Tag [10] Physical page number [20] Read-‐only [1]
Valid [1]
0 0xABC 0xABC00 0 0
1 0x280 0xB0001 1 1
2 0x300 0x00001 0 1
The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001 TLB index = 1
VPN = 0xA0001 à 1010 0000 00 00 0000 0001
TLB index, used to select a TLB line
Line index Tag [10] Physical page number [20] Read-‐only [1]
Valid [1]
0 0xABC 0xABC00 0 0
1 0x280 0xB0001 1 1
2 0x300 0x00001 0 1
The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001 TLB index = 1
VPN = 0xA0001 à 1010 0000 00 00 0000 0001
TLB index, used to select a TLB line
Line index Tag [10] Physical page number [20] Read-‐only [1]
Valid [1]
0 0xABC 0xABC00 0 0
1 0x280 0xB0001 1 1
2 0x300 0x00001 0 1
The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001 TLB index = 1
VPN = 0xA0001 à 1010 0000 00 00 0000 0001
Tag = 0x280 à 1010 0000 00 =
Line index Tag [10] Physical page number [20] Read-‐only [1]
Valid [1]
0 0xABC 0xABC00 0 0
1 0x280 0xB0001 1 1
2 0x300 0x00001 0 1
The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001 TLB index = 1
VPN = 0xA0001 à 1010 0000 00 00 0000 0001
Tag = 0x280 à 1010 0000 00 =
⇒ Physical page number = 0xB0001
Line index Tag [10] Physical page number [20] Read-‐only [1]
Valid [1]
0 0xABC 0xABC00 0 0
1 0x280 0xB0001 1 1
2 0x300 0x00001 0 1
The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001 TLB index = 1
VPN = 0xA0001 à 1010 0000 00 00 0000 0001
Tag = 0x280 à 1010 0000 00 =
⇒ Physical page number = 0xB0001 ⇒ Physical Address = 0xB0001004
Por$ng NetBSD
• 1°) NetBSD cross compila$on toolchain – build.sh – Makefiles here and there – Arch-‐specific directories
Allows to do: $ ./build.sh -‐U -‐m milkymist tools
Por$ng NetBSD
• 2°) Support for built-‐ins in libkern – NetBSD kernel is
• Not linked against libgcc • Linked against libkern
– Need to implement basic arithme$c func$ons emiOed by gcc in object code
– Implementa$on in sys/lib/libkern/arch/lm32
Por$ng NetBSD
• 3°) Building my first kernel – Create sys/arch/lm32 and sys/arch/milkymist – Populate
• sys/arch/<cpu|soc>/include • sys/arch/<cpu|soc>/conf
– Stub, stub, stub… Allows to do: $ ./build.sh -‐m milkymist -‐U kernel=GENERIC
Por$ng NetBSD
• 4°) Write basic console driver for early prints
struct consdev milkymist_com_cons = { […] milkymist_com_cngetc, /* cn_getc: kernel getchar interface */ milkymist_com_cnputc, /* cn_putc: kernel putchar interface */ […] };
Por$ng NetBSD
• 5°) Implement excep$on handlers • 6°) Call milkymist_startup() C code – Ini$alize console driver
• -‐> consinit() -‐> milkymist_uart_cnaOach() • cn_tab = &milkymist_com_cons;
– Ini$aliaze virtual memory subsystem • Call MD pmap_bootstrap()
– Let the kernel boot • Call NetBSD MI main()
Por$ng NetBSD
• 7°) Implement pmap.9 pmap -‐-‐ machine-‐dependent por/on of the virtual memory system – pmap_bootstrap() – pmap_init, pmap_create, pmap_destroy … – SW managed TLB? -‐> sys/uvm/pmap/
– used in (PowerPC Booke and LM32)
Por$ng NetBSD
• 8°) Implement copyin/copyout • 9°) Implement atomic opera$ons – No atomic instruc$on à RAS (Restartable Atomic Sequence) CAS (Compare And Swap)
– Other atomic and locking ops built around this CAS
RAS CAS int _atomic_cas_32(vola$le uint32_t *val, uint32_t old, uint32_t new); _atomic_cas_32: _atomic_cas_ras_start:
lw r4, (r1+0) /* load *val into r4 */ bne r4, r2, 1f /* compare r4 (*val) and old (r2) */ sw (r1+0), r3
_atomic_cas_ras_end: 1:
mv r1, r4 /* return (*val) */ ret
Por$ng NetBSD
• 10°) Add support for interrupts – Write a func$on to register interrupt handlers
• 11°) Have a running system clock – Write cpu_initclocks() – Write clock irq handler
• Call hardclock()
Other func$ons to write
• Switch context from one thread to another – cpu_switchto(9)
• Copy data and abort on page fault – kcopy(9)
• Save current context – sebault()
• Low level code to finish up fork() opera$on – cpu_lwp_fork(9)
Other func$ons to write
• Block interrupts to protect cri$cal sec$ons – spl(9)
• Init CPU and print copyright message – cpu_startup(9)
• Determine the root file system device – cpu_rootconf(9)
• Etc…
Por$ng NetBSD
• To boot user space – Create dummy ramdisk with /sbin/init – Build kernel with MFS – Insert ramdisk with mdse$mage – Boot it!
Por$ng NetBSD
DEMO
NetBSD/milkymist virt. Memory Layout
Kernel space User space
0 0xffffffff
0xc0000000
0xc8000000 RAM window
User stack
Kernel stack
DDR SDRAM : 128 MB
RAM window
Physical RAM
128 MB
Start: PA 0x4000 0000
End: PA 0x4800 0000
Virtual Memory Layout
VA: 0xc000 0000
VA: 0xc800 0000
Direct mapping
Page table
Page directory @ 0xc300 0000 0x**** ****
0x**** ****
0x**** ****
0xc400 1000
0x**** ****
Page table @ 0xc400 1000 0x**** ****
0x**** ****
0x1234 1000
0x**** ****
0x**** ****
Virtual address: 0x00C0 2ABC
Page table
Page directory @ 0xc300 0000 0x**** ****
0x**** ****
0x**** ****
0xc400 1000
0x**** ****
Page table @ 0xc400 1000 0x**** ****
0x**** ****
0x1234 1xxx
0x**** ****
0x**** ****
Virtual address: 0x00C0 2ABC
Binary à 0000000011 0000000010 101010111100 10 bits 10 bits 12 bits
1024 entries 1024 entries
Page table
Page directory @ 0xc300 0000 0x**** ****
0x**** ****
0x**** ****
0xc400 1000
0x**** ****
Page table @ 0xc400 1000 0x**** ****
0x**** ****
0x1234 1xxx
0x**** ****
0x**** ****
Virtual address: 0x00C0 2ABC
Binary à 0000000011 0000000010 101010111100
1024 entries 1024 entries
Page table
Page directory @ 0xc300 0000 0x**** ****
0x**** ****
0x**** ****
0xc400 1000
0x**** ****
Page table @ 0xc400 1000 0x**** ****
0x**** ****
0x1234 1xxx
0x**** ****
0x**** ****
Virtual address: 0x00C0 2ABC
Binary à 0000000011 0000000010 101010111100
1024 entries 1024 entries
Page table
Page directory @ 0xc300 0000 0x**** ****
0x**** ****
0x**** ****
0xc400 1000
0x**** ****
Page table @ 0xc400 1000 0x**** ****
0x**** ****
0x4123 1xxx
0x**** ****
0x**** ****
Virtual address: 0x00C0 2ABC
Binary à 0000000011 0000000010 101010111100
1024 entries 1024 entries
Page table
Page directory @ 0xc300 0000 0x**** ****
0x**** ****
0x**** ****
0xc400 1000
0x**** ****
Page table @ 0xc400 1000 0x**** ****
0x**** ****
0x4123 1xxx
0x**** ****
0x**** ****
Virtual address: 0x00C0 2ABC
Binary à 0000000011 0000000010 101010111100
Physical address: 0x4123 1xxx
1024 entries 1024 entries
Page table
Page directory @ 0xc300 0000 0x**** ****
0x**** ****
0x**** ****
0xc400 1000
0x**** ****
Page table @ 0xc400 1000 0x**** ****
0x**** ****
0x4123 1xxx
0x**** ****
0x**** ****
Virtual address: 0x00C0 2 ABC
Binary à 0000000011 0000000010 101010111100
Physical address: 0x4123 1ABC
1024 entries 1024 entries
Accessing the page table
• Kernel running with MMU ON o Direct access through virt. addresses 0xc*** ****
• TLB miss handler running with MMU OFF o Computes phys. addresses from RAM window addresses using following formula
o PA = VA – 0xC000 0000 + 0x4000 0000
Follow progress
• hOp://sionneau.net/edgebsd/ • hOp://edgebsd.org/edgebsd/wiki/14/port-‐lm32
• hOp://github.com/fallen/NetBSD • hOp://git.edgebsd.org/gitweb/?p=edgebsd-‐src.git;a=shortlog;h=refs/heads/fallen-‐port-‐milkymist (aka hOp://goo.gl/9Mbg7U)
Thank you!
Sébas$en Bourdeauducq, Michael Walle, Robert Swindells, Stefan Kris$ansson, Lars-‐Peter Clausen, Pierre Pronchery, Radoslaw Kujawa, Youri Mouton, MaO Thomas, tech-‐kern@, M-‐Labs mailing list, and many more.
QuesLons?
