Lecture 10 1

Lecture 10: Kernel Modules and Device Drivers

ECE 412: Microcomputer Laboratory

Lecture 10 2

Objectives

• Review Linux environment• Device classification• Review Kernel modules• PCMCIA example• Skeleton example of implementing a device driver for

a BlockRAM based device

Lecture 10 3

Review Questions

What are some of the services/features that an IPIF-generated interface to the PLB/OPB bus can provide?• “Byte Steering” for devices with narrow data widths• Address range checking to detect transactions your device

should handle• User-defined registers• Interface to the interrupt hardware• Fixed-length burst transfers• DMA engine• Read/write FIFOs

Lecture 10 4

Linux Execution Environment

Memory

b

STANDARD CLIBRARY

MATHLIBRARYAPPLICATION (mpg123)

MemoryManagement

FilesystemsNetworking

ArchitectureDependent

Code

MemoryManager

File SystemDevices

CharacterDevices

NetworkSubsystem

OPERATING SYSTEM

ProcessManagement

DeviceControl

Network InterfacesCPU

Disk

• Program

• Libraries

• Kernel subsystems

Lecture 10 5

Device Classification

• Most device drivers can be classified into one of three categories.

• Character devices.– Console and parallel ports are examples.– Implement a stream abstraction with operations such as

open, close, read and write system calls.– File system nodes such as /dev/tty1 and /dev/lp1 are used to

access character devices.– Differ from regular files in that you usually cannot step

backward in a stream.

Lecture 10 6

Device Classification (cont’)

• Block devices– A block device is something that can host a filesystem, e.g. disk,

and can be accessed only as multiples of a block.

– Linux allows users to treat block devices as character devices (/dev/hda1) with transfers of any number of bytes.

– Block and character devices differ primarily in the way data is managed internally by the kernel at the kernel/driver interface.

– The difference between block and char is transparent to the user.

• Network interfaces– In charge of sending and receiving data packets.

– Network interfaces are not stream-oriented and therefore, are not easily mapped to a node in the filesystem, such as /dev/tty1.

– Communication between the kernel and network driver is not through read/write, but rather through packet transfer functions.

Lecture 10 7

Linux Execution Environment (review)

• Execution paths

Memory

b

STANDARD CLIBRARY

MATHLIBRARYAPPLICATION (mpg123)

MemoryManagement

FilesystemsNetworking

ArchitectureDependent

Code

MemoryManager

File SystemDevices

CharacterDevices

NetworkSubsystem

OPERATING SYSTEM

ProcessManagement

DeviceControl

Network InterfacesCPU

Disk

malloc

_sbrk

fprintf

vfprintf

writeread

_isnan

sin

pow

Decoder

I/O

HTTP

Network

Initialization

socket

tan

log

wait

rand

qsortscanf

valloc

Lecture 10 8

Process and System Calls• Process: program in execution. Unique “pid”. Hierarchy.• User address space vs. kernel address space• Application requests OS services through TRAP mechanism

– x86: syscall number in eax register, exception (int $0x80)

– result = read (file descriptor, user buffer, amount in bytes)

– Read returns real amount of bytes transferred or error code (<0)

• Kernel has access to kernel address space (code, data, and device ports and memory), and to user address space, but only to the process that is currently running

• “Current” process descriptor. “currentpid” points to current pid• Two stacks per process: user stack and kernel stack• Special instructions to copy parameters / results between user

and kernel space

Lecture 10 9

Kernel Modules• Kernel modules are inserted and unloaded dynamically

– Kernel code extensibility at run time– insmod / rmmod / lsmod commands. Look at /proc/modules– Kernel and servers can detect and install them automatically, for example,

cardmgr (pc card services manager)

• Example of the content of /proc/modules

– nfs 170109 0 - Live 0x129b0000

– The first column contains the name of the module. – The second column refers to the memory size of the module, in bytes. – The third column lists how many instances of the module are currently

loaded. A value of zero represents an unloaded module. – The fourth column states if the module depends upon another module to be

present in order to function, and lists those other modules. – The fifth column lists what load state the module is in: Live, Loading, or

Unloading are the only possible values. – The sixth column lists the current kernel memory offset for the loaded

module. This information can be useful for debugging purposes, or for profiling tools such as oprofile.

Lecture 10 10

Module Execution• Modules execute in kernel space

– Access to kernel resources (memory, I/O ports) and global variables ( look at /proc/ksyms)

– Export their own visible variables, register_symtab ();– Can implement new kernel services (new system calls,

policies) or low level drivers (new devices, mechanisms)– Use internal kernel basic interface and can interact with

other modules– Need to implement init_module and cleanup_module entry

points, and specific subsystem functions (open, read, write, close, ioctl …)

Lecture 10 11

Hello World• hello_world_module.c:

#define MODULE#include <linux/module.h>static int __init init_module(void){ printk("<1>Hello, world\n"); /* <1> is message priority. */ return 0;}static int __exit cleanup_module(void){ printk("<1>Goodbye cruel world\n");}

• printk (basic kernel service) outputs messages to console and/or to /var/log/messages

• To compile and run this code:– root# gcc -c hello_world_module.c– root# insmod hello_world_module.o– root# rmmod hello_world_module

Lecture 10 12

Linking a module to the kernel (from Rubini’s book)

Lecture 10 13

Register Capability• You can register a new device driver with the kernel:

– int register_chrdev(unsigned int major, const char *name, struct file_operations *fops);

– A negative return value indicates an error, 0 or positive indicates success.

– major: the major number being requested (a number < 128 or 256).– name: the name of the device (which appears in /proc/devices).– fops: a pointer to a global jump table used to invoke driver

functions.

• Then give to the programs a name by which they can request the driver through a device node in /dev– To create a char device node with major 254 and minor 0, use:

• mknod /dev/memory_common c 254 0

– Minor numbers should be in the range of 0 to 255.

(Generally, the major number identifies the device driver and the minor number identifies a particular device (possibly out of many) that the driver controls.)

Lecture 10 14

PCMCIA Read/Write Common/Attribute Memory

applicationapplication data = mem_read (address, type)mem_write (address, data, type)

data = mem_read (address, type)mem_write (address, data, type)

/dev/… PCMCIAregistered memory fops

/dev/… PCMCIAregistered memory fops

memory_read(), memory_write()memory_read(), memory_write()

- map kernel memory to I/O window- copy from PCMCIA to user ( &buf)- copy from user to PCMCIA (&data)

- map kernel memory to I/O window- copy from PCMCIA to user ( &buf)- copy from user to PCMCIA (&data)

USER SPACE

KERNEL SPACE

Libc: file I/O

PCMCIA

attribute common

- open(“/dev/memory_[common|attribute]”)- lseek(fd, address)- read(fd, buf,1); return buf;- write(fd, data, 1)

- open(“/dev/memory_[common|attribute]”)- lseek(fd, address)- read(fd, buf,1); return buf;- write(fd, data, 1)

int buf

Card insertion

card_memory_config:- read CIS- config I/O window- config IRQ- register R/W fops

card_memory_config:- read CIS- config I/O window- config IRQ- register R/W fops

Kernel memory

Lecture 10 15

PCMCIA “Button Read” Interrupt handling

applicationapplication data = mem_read (address, type)mem_write (address, data, type)

data = mem_read (address, type)mem_write (address, data, type)

/dev/… PCMCIAregistered memory fops

/dev/… PCMCIAregistered memory fops

memory_button_read()memory_button_read()

- interruptible_sleep_on (PC->queue)- memory_read()

- map kernel memory to I/O window- copy PC to user ( &buf)

- interruptible_sleep_on (PC->queue)- memory_read()

- map kernel memory to I/O window- copy PC to user ( &buf)

USER SPACE

KERNEL SPACE

Libc: file I/O

PCMCIA

attribute common

- open(“/dev/memory_common”)- lseek(fd, address)- read(fd, buf,1); return buf;- write(fd, data, 1)

- open(“/dev/memory_common”)- lseek(fd, address)- read(fd, buf,1); return buf;- write(fd, data, 1)

int buf

Card insertioncard_memory_config:… - config IRQ handler

card_memory_config:… - config IRQ handler

Kernel memory

Button int.int_handler:- wake_up( PC->queue)

int_handler:- wake_up( PC->queue)

Lecture 10 16

Skeleton Example: OCM-Based BlockRAM

• PowerPC has an OCM (on-chip memory) bus that lets you attach fast memory to the cache

• Xilinx provides a core (dso_if_ocm) that handles the interface to the OCM and outputs BRAM control signals– Found under Project->Add/Edit cores– Creates an interface that detects accesses to a specified

physical address range and outputs control signals for a BlockRAM

Lecture 10 17

Software-Side Issues• Xilinx core handles the BlockRAM interface from the

hardware side, but need to make BlockRAM visible/accessible to software

• Two issues:– Programs operate on virtual addresses, even when running

as root– Ideally, want to be able to make BlockRAM visible to user-

mode programs• User-mode programs can’t set virtual->physical address

mappings

Lecture 10 18

Direct Approach -- Use mmap()• Only works for code running as root

fd = open(“/dev/mem”, O_RDWR);

bram = mmap(0x40000000, 2048, PROT_READ |

PROT_WRITE, MAP_SHARED, fd, 0x40000000);

assert(bram == 0x40000000);

• Creates pointer to the /dev entry that describes the physical memory

• Maps 2048 bytes from /dev/mem onto the program’s address space, starting at offset 0x40000000 from the start of the pointer

• Requests that those bytes be mapped onto addresses starting at 0x40000000

• Checks (via assert) that mmap() returned the requested address, as mmap() isn’t required to follow that request

Lecture 10 19

Better Approach -- Device Driver• Create device driver module and install into Linux• Device driver module will map BRAM onto address

space of currently-running program

Lecture 10 20

Device Driver• Device drivers provide mechanisms, not policy.

– Mechanism: “Defines what capabilities are provided?”– Policy: “Defines how those capabilities can be used?”

• This strategy allows flexibility.• The driver controls the hardware and provides an abstract interface to

its capabilities.• The driver ideally imposes no restrictions (or policy) on how the

hardware should be used by applications.

• For example, X manages the graphics hardware and provides an interface to user programs.

• Window managers implement a particular policy and know nothing about the hardware.

• Kernel apps build policies on top of the driver, e.g. floppy disk, such as who has access, the type of access (direct or as a filesystem), etc. -- it makes the disk look like an array of blocks.

Courtesy of UMBC

Lecture 10 21

Device Driver Outline1. Obtain memory map semaphore for currently

running program (to prevent overlapping changes)

2. Insert new virtual memory area (VMA) for BRAM

3. Call get_unmapped_area with physical address range of BRAM

4. Allocate and initialize VMA for the BRAM

5. Call remap_page_range() to build page tables

6. Use insert_vma_struct() and make_pages_present() to enable access to new pages

• See “Running Linux on a Xilinx XUP Board” for more information (on the web, written by John Kelm).

Lecture 10 22

Next Time

• Quiz 1