Post on 21-Jan-2016
description
Content
Preparing a Program for Execution – Program Transformations
– Logical-to-Physical Address Binding
Memory Partitioning Schemes– Fixed Partitions
– Variable Partitions
Allocation Strategies for Variable Partitions
Dealing with Insufficient Memory
Operating Systems PrinciplesMemory Management
Lecture 7: Physical Memory
Preparing a Program
for Execution
Preparing a Program for ExecutionModules reflecting different functions are designed separately, possibly by different programmers.
Modules reflecting different functions are designed separately, possibly by different programmers.
Preparing a Program for Execution
Compilation
...1000functionfun
............
...108inti
...100floatx
. . .AddressTypeName
...1000functionfun
............
...108inti
...100floatx
. . .AddressTypeName
...1000functionfun
............
...108inti
...100floatx
. . .AddressTypeName
...1000functionfun
............
...108inti
...100floatx
. . .AddressTypeName. . . . .
float x;
int i;. . . . .
void fun()
{. . . . .
}
. . . . .
float x;
int i;. . . . .
void fun()
{. . . . .
}
External Symbol Table
Compilation will produce an object module and a corresponding external symbol table.
Preparing a Program for Execution
Compilation
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
Preparing a Program for Execution
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
The linker combines several object modules together to build a load module (EXE) by resolving
external references through symbol tables.
The linker combines several object modules together to build a load module (EXE) by resolving
external references through symbol tables.
Preparing a Program for Execution
Logical-to-physical address mapping is done by the loader to transfer the load
module from the secondary storage to the main memory.
Logical-to-physical address mapping is done by the loader to transfer the load
module from the secondary storage to the main memory.
More on Linking
ObjectModule 1
ObjectModule 2
ObjectModule n
LogicalAddress Space
ObjectModule 1
ObjectModule 1
ObjectModule 2
ObjectModule 2
ObjectModule n
ObjectModule n
Linking Linking
00000000
FFFFFFF
More on Linking
ObjectModule 1
ObjectModule 2
ObjectModule n
LogicalAddress Space
ObjectModule 1
ObjectModule 1
ObjectModule 2
ObjectModule 2
ObjectModule n
ObjectModule n
Linking Linking
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
. . .AddressTypeName . . .AddressTypeName
Load
Module
Logical address space is used by the linker to resolve external references.
Binding can be static or dynamic.
00000000
FFFFFFF
Logical-to-Physical Address Binding
LoadModule 1
LoadModule 1
PhysicalAddress Space
LoadModule 2
LoadModule 2 Load
Module 3
LoadModule 3
LoadModule 3’
LoadModule 2’
LoadModule 1’
00000000
Created by linker(on logical address space)
00000000
aaaaaaaa
bbbbbbbb
cccccccc
Assignment of actual physical addresses to program instruction and data.
Program relocation needed(done by the loader)
Address Binding
Static binding– Programming time– Compilation time– Linking time– Loading time
Dynamic binding– Execution time
Assignment of actual physical addresses to program instruction and data.
Static Binding
Programming-time Binding
– Seldom used
– Used in low-level environments
– E.g., OS, real-time and embedded systems,
control special hardware component.
• Programming time• Compilation time• Linking time• Loading time
Static Binding
SAMPLE PROGRAM FOR MC6802 USING CRS8The following source file has been named MC6802.ASM
CPU 6802 ; 6802 processorHOF MOT ; Motorola Records ORG 0100H ; Start of Data
Source: DFB 'Hello and Welcome'Length: EQU $ - Source ;Length of SourceDestin: DFS Length ; Buffer which has same
; length as SourceORG 0120H ; Start of Code
Entry: LDX #Source ; Point Index Reg to ; Source string
LDAB #Length ; Number of characters to moveLoop: LDAA 0,X
STAA Length,XINXDECBBNE Loop
Fin: JMP FinEND Entry
SAMPLE PROGRAM FOR MC6802 USING CRS8The following source file has been named MC6802.ASM
CPU 6802 ; 6802 processorHOF MOT ; Motorola Records ORG 0100H ; Start of Data
Source: DFB 'Hello and Welcome'Length: EQU $ - Source ;Length of SourceDestin: DFS Length ; Buffer which has same
; length as SourceORG 0120H ; Start of Code
Entry: LDX #Source ; Point Index Reg to ; Source string
LDAB #Length ; Number of characters to moveLoop: LDAA 0,X
STAA Length,XINXDECBBNE Loop
Fin: JMP FinEND Entry
Programming-time
bindingProgramming-time
binding
• Programming time• Compilation time• Linking time• Loading time
Static Binding
• Programming time• Compilation time• Linking time• Loading time
Compile-time Binding– The compiler is given the starting address for th
e program to be load for execution.– The resulting program can execute only when l
oaded into the specific preassigned memory space.
– Not relocatable– Rarely used
Static Binding
• Programming time• Compilation time• Linking time• Loading time
Compile-time Binding– Named variables have their addresses hardcoded
Global variables given offset from start of global data area
Local variables given offset from top of stack
Object variables given offset from start of object data
– Normal function and method calls are hardcoded Normal functions have specific starting address in object file
Object methods have specific starting address in object file
Static Binding
• Programming time• Compilation time• Linking time• Loading time
Link-time Binding– The load module is still not relocatable– The program will be loaded to memory with starti
ng location specified or assumed by the linker (Linkage Editor).
– Linking loader: combining linking and loading– Widely used in smaller, single user systems
(e.g., CP/M)
LoadModuleLoad
Module
Static Binding
• Programming time• Compilation time• Linking time• Loading time
Objectprogram(s)
Objectprogram(s)
Linkage editorLinkage editorLibraryLibrary
Simple LoaderSimple Loader
MemoryMemory
Objectprogram(s)
Objectprogram(s)
Linking loaderLinking loader
MemoryMemory
LibraryLibrary
LoadModuleLoad
Module
Static Binding
• Programming time• Compilation time• Linking time• Loading time
Objectprogram(s)
Objectprogram(s)
Linkage editorLinkage editorLibraryLibrary
Simple LoaderSimple Loader
MemoryMemory
Objectprogram(s)
Objectprogram(s)
Linking loaderLinking loader
MemoryMemory
LibraryLibrary
Static Binding
• Programming time• Compilation time• Linking time• Loading time
Load-Time Binding– The load module is relocatabl
e– What operands are needed to
be relocated (relocatable)? Indicated by compilers and as
semblers Needed for linking and loadin
g– How to perform relocation?
LoadModuleLoad
Module
Objectprogram(s)
Objectprogram(s)
Linkage editorLinkage editorLibraryLibrary
LoaderLoader
MemoryMemory
relocatable
What?
How?
What?
Static Binding
• Programming time• Compilation time• Linking time• Loading time
What operands are needed
to be relocated?
– Register?
– Immediate operand
(constant)?
– Offset to a base register?
– Absolute memory address
– Relative memory address?
LoadModuleLoad
Module
Objectprogram(s)
Objectprogram(s)
Linkage editorLinkage editorLibraryLibrary
LoaderLoader
MemoryMemory
relocatable
What?
How?
What?
Static Binding
• Programming time• Compilation time• Linking time• Loading time
What operands are needed
to be relocated?
– Register?
– Immediate operand
(constant)?
– Offset to a base register?
– Absolute memory address
– Relative memory address?
LoadModuleLoad
Module
Objectprogram(s)
Objectprogram(s)
Linkage editorLinkage editorLibraryLibrary
LoaderLoader
MemoryMemory
relocatable
What?
How?
What?Keep unchanged
Ralocatable
Static Binding
• Programming time• Compilation time• Linking time• Loading time
How to relocate the relocatable operands?
LoadModuleLoad
Module
Objectprogram(s)
Objectprogram(s)
Linkage editorLinkage editorLibraryLibrary
LoaderLoader
MemoryMemory
relocatable
What?
How?
What?
LogicalAddress Space
PhysicalAddress Space
LoadModule
LoadModule’
x
y
Static Binding
How to relocate the relocatable operands?
LoadModuleLoad
Module
Objectprogram(s)
Objectprogram(s)
Linkage editorLinkage editorLibraryLibrary
LoaderLoader
MemoryMemory
relocatable
What?
How?
What?
LogicalAddress Space
PhysicalAddress Space
LoadModule
LoadModule’
x
y
Add all relocatable operands in the load module by th
e amount of y x.
Add all relocatable operands in the load module by th
e amount of y x.
• Programming time• Compilation time• Linking time• Loading time
Example
. . .int i;. . .i = ???;. . .g();. . .
. . .int i;. . .i = ???;. . .g();. . .
LogicalAddress Space
020 i
store 20
call g
f(){ . . .}
g(){ . . .}
f(){ . . .}
g(){ . . .}
LogicalAddress Space
SourceModule 1
SourceModule 2
0
200
functionf
functiong
compiler compiler
Example
. . .int i;. . .i = ???;. . .g();. . .
. . .int i;. . .i = ???;. . .g();. . .
LogicalAddress Space
020 i
store 20
call g
f(){ . . .}
g(){ . . .}
f(){ . . .}
g(){ . . .}
LogicalAddress Space
SourceModule 1
SourceModule 2
0
200
functionf
functiong
compiler compiler
Relative memory address(relocatable)
Relative memory address(relocatable)
External reference
External reference
Example
. . .int i;. . .i = ???;. . .g();. . .
. . .int i;. . .i = ???;. . .g();. . .
LogicalAddress Space
020 i
store 20
call g
f(){ . . .}
g(){ . . .}
f(){ . . .}
g(){ . . .}
LogicalAddress Space
SourceModule 1
SourceModule 2
0
200
functionf
functiong
compiler compiler
Object Module
1(obj1)
Object Module
2(obj2)
Example
020 i
store 20
call g
0
200
functionf
functionf
functiong
functiong
Obj1 Obj20
OthersOthers
Obj3
Link obj3+obj2+obj1, myexe
LogicalAddress Space
0Others
50
250
functionf
functiong
350370 i
store 370
call 250
linker
Example
020 i
store 20
call g
0
200
functionf
functionf
functiong
functiong
Obj1 Obj20
OthersOthers
Obj3
Link obj3+obj2+obj1, myexe
LogicalAddress Space
0Others
50
250
functionf
functiong
350370 i
store 370
call 250
relocatedrelocated
External reference resolved
External reference resolved
Example
020 i
store 20
call g
0
200
functionf
functionf
functiong
functiong
Obj1 Obj20
OthersOthers
Obj3
LogicalAddress Space
0Others
50
250
functionf
functiong
350370 i
store 370
call 250
Load Module
(myexe)
Link obj3+obj2+obj1, myexe
Example0
Others
50
250
functionf
functiong
350370 i
store 370
call 250
myexePhysicalAddressSpace
1000Others
1050
1250
functionf
functiong
13501370 i
store 1370
call 1250
loader
1000
relocated
Dynamic Binding
Dynamic Binding = Binding at Execution Time That is, binding immediately before each memory
reference. Hardware support needed for logical-to-physical a
ddress mapping
Physical address differs from logical address usually by a constant offset.
Operating Systems PrinciplesMemory Management
Lecture 7: Physical Memory
Memory Partitioning
Schemes
Memory Partitioning Schemes
Fixed Partitions– The number of partitions and the size
of each partition are determined at the time the OS is initialized.
Variable Partitions– Memory not partitioned a priori– Partitioning on demand
Fixed Partitions
Single-program systems– 2 partitions (OS/user)
Multi-programmed systems– partitions of different sizes
OSOS
UserUser
OSOS
......
How to Assign Processes to Partitions?
FIFO for each partition– Typically, best-fit is used
for queue assignment.– Problem: Some partitions
may be unused if no processes of appropriate sizes are available.
– To resolve the problem, more complex queue management scheme and process scheduling scheme would be required.
How to Assign Processes to Partitions?
Single FIFO– More complex, but more
flexible.– E.g., rather than leaving
a partition empty, the scheduler may assigned a process to a partition not in best-fit sense.
– E.g., the actual memory requirement of a process may grow and shrink dynamically.
OSOS
Limitations of Fixed Partitions
Program size limited to largest partition Internal fragmentation
– unused space within partitions
Variable Partitions
Memory not partitioned a priori
Each request is allocated portion of free space.
Allocating and releasing memory dynamically cause external fragmentation.
Issues How to allocate
memories of sizes
1.
2.
All free blocks are applicable.
None free block is applicable.
But, the total amount of free memory is large enough.
Allocate which one?
Over time, memory will consist a sequence of variable size blocks. Some are free, and some are not.
Hole Coalescing
A
B
D
E
C
F
A
B
D
E
C
F
A
B
D
E
C
F
B’
A
D
E
F
B’
B’’
G G G G
A
E
F
B’’
G
B’’’
Adjacent holes must be coalesced to prevent increasing fragmentation.
Hole Coalescing
Adjacent holes must be coalesced to prevent increasing fragmentation.
Four cases on hole coalescing:
no adjacent hole
A hole at right A hole at left Holes at two sides
Link List Implementation (I)
Free blocks are kept sorted using a doubly linked list. What would be done to release a block of memory?
– Check both its neighbors for possible coalescing. Problems:
– How to check the right neighbor? – How to check the left neighbor?
G A B C D E Ffree
free
free
occu
pied
occu
pied
occu
pied
occu
pied
size
size
size
size
size
size
size
easy
checked by starting fromthe header (inefficient)
E
Link List Implementation (II)
Free blocks are linked using a doubly linked list – need not be sorted
What would be done to release a block of memory?– Check both its neighbors for possible coalescing.
Problems: – How to check the right neighbor? – How to check the left neighbor?
A Bfree
occu
pied
size
size
easy
free
size
occu
pied
size C
occu
pied
size
occu
pied
size D
occu
pied
size
occu
pied
size
free
size
easy
Bitmap Implementation
Memory divided into fix-size blocks Each block represented by a 0/1 bit in a bi
nary string: the “bitmap” Can be implemented as char or int array Operations use bit masks
– Release: B[i] = B[i] & '11011111'– Allocate: B[i] = B[i] | '11000000'– Search: Repeatedly, Check left-most bit and
Shift mask right: TEST = B[i] & '10000000'
Bitmap Implementation
4K00000000
4K4K4K4K4K4K4K4K4K4K4K4K4K4K4K
00001000
00002000
00003000
00004000
00005000
00006000
00007000
00008000
00009000
0000a000
0000b000
0000c000
0000d000
0000e000
0000f000
4K00010000
4K4K4K4K4K4K4K4K4K4K4K4K4K4K4K
00011000
00012000
00013000
00014000
00015000
00016000
00017000
00018000
00019000
0001a000
0001b000
0001c000
0001d000
0001e000
0001f000
4K00070000
4K4K4K4K4K4K4K4K4K4K4K4K4K4K4K
00071000
00072000
00073000
00074000
00075000
00076000
00077000
00078000
00079000
0007a000
0007b000
0007c000
0007d000
0007e000
0007f000
Bitmap Implementation
4K00000000
4K4K4K4K4K4K4K4K4K4K4K4K4K4K4K
00001000
00002000
00003000
00004000
00005000
00006000
00007000
00008000
00009000
0000a000
0000b000
0000c000
0000d000
0000e000
0000f000
4K00010000
4K4K4K4K4K4K4K4K4K4K4K4K4K4K4K
00011000
00012000
00013000
00014000
00015000
00016000
00017000
00018000
00019000
0001a000
0001b000
0001c000
0001d000
0001e000
0001f000
4K00070000
4K4K4K4K4K4K4K4K4K4K4K4K4K4K4K
00071000
00072000
00073000
00074000
00075000
00076000
00077000
00078000
00079000
0007a000
0007b000
0007c000
0007d000
0007e000
0007f000
4K4K
4K4K4K4K4K4K
4K4K
4K4K4K4K
4K4K4K4K
How to allocate a block of continuous memory of a given size?
Buddy Systems
Compromise between fixed and variable partitions
Fixed number of possible hole sizes; typically, 2i.– Each hole of size 2i can be divided (equally) into 2
buddies of size 2i1.
– Two buddies can be combined into the original hole of
size 2i.
Track holes by size on separate lists– List entries for holes of sizes 20, 21, …, 2m.
2i
2i1
2i1
Buddy Systems
Compromise between fixed and variable partitions
Fixed number of possible hole sizes; typically, 2i.– Each hole of size 2i can be divided (equally) into 2
buddies of size 2i1.
– Two buddies can be combined into the original hole of
size 2i.
Track holes by size on separate lists– List entries for holes of sizes 20, 21, …, 2m.
.
.
.
20
21
22
2m
Buddy Systems
.
.
.
20
21
22
2m
.
.
.
20
21
22
2m
When n bytes requested, find smallest i so that n 2i
– If hole of this size available, allocate it;– otherwise, consider larger holes.
Recursively split each hole into two buddies until smallest adequate hole is created Allocate it and place other holes on appropriate lists
On release, recursively coalesce buddies– Buddy searching for coalescing can be inefficient
Example
void p=malloc(1*blocksize);
free(12*blocksize);
Hole Sizes: 1, 2, 4, 8, 16
3 blocks allocated & 3 holes left
Operating Systems PrinciplesMemory Management
Lecture 7: Physical Memory
Allocation Strategies for
Variable Partitions
Allocation Strategies
Problem:
Given a request for n bytes, find hole ≥ n
Goal: – Maximize memory utilization
(Minimize “external fragmentation”)
– Minimize search time
Common Allocation Strategies
First-fit: – Always start from the beginning of free-list– Simplest– Generally the best choice
Next-fit: – Rotating-first-fit (Resume search)– Improves distribution of holes
Best-fit: – Closest fit – Avoid breaking up large holes.
Worst-fit:– Largest fit– Avoid leaving tiny hole fragments
Measure of Memory Fragmentation
What measurement could be used?
m: #blocks occupied n: #holes
1
4
n
m
2
4
n
m
3
4
n
m
5
4
n
m
Measure of Memory Fragmentation
What measurement could be used?
m: #blocks occupied n: #holes
1
4
n
m
2
4
n
m
3
4
n
m
5
4
n
m
?n
m at equilibrium. ?
n
m at equilibrium.
Measure of Memory Fragmentation
What measurement could be used?
m: #blocks occupied n: #holes
Four types of occupied memory blocks.
Increase one hole on release.
#holes unchanged on
release
Decrease one hole on release.
Measure of Memory Fragmentation
What measurement could be used?
m: #blocks occupied n: #holes
Increase one hole on release.
#holes unchanged on
release
Decrease one hole on release.
Let a, b, c, d be the average number of occupied blocks of each type in memory during equilibrium.
Let a, b, c, d be the average number of occupied blocks of each type in memory during equilibrium.
m a b c d
2
2
d b cn
Fact: b = c
d b d c
Measure of Memory Fragmentation
Increase one hole on release.
#holes unchanged on
release
Decrease one hole on release.
m a b c d n d b d c
(increase one hole) (release)a
P Pm
(decrease one hole) (release) (request) (1 )d
P P P pm
Measure of Memory Fragmentation
Increase one hole on release.
#holes unchanged on
release
Decrease one hole on release.
m a b c d
(decrease one hole) (release) (request) (1 )d
P P P pm
p: The probability that the request memory size doesn’t match any hole size.Fact: p 1.
p: The probability that the request memory size doesn’t match any hole size.Fact: p 1.
(increase one hole) (release)a
P Pm
n d b d c
Measure of Memory Fragmentation
m a b c d
(decrease one hole) (release) (request) (1 )d
P P P pm
(increase one hole) (release)a
P Pm
In equilibrium, (increase one hole) (decrease one hole)P P(request) (release)P P
(1 )a d
pm m
(1 )a d p m
(1 )m d p m b c d
n d b d c
(1 ) 2p m n
Measure of Memory Fragmentation
In equilibrium, (increase one hole) (decrease one hole)P P(request) (release)P P
(1 )a d
pm m
(1 )a d p m
(1 )m d p m b c d (1 ) 2p m n
0.5n pm
0.5n
pm
0.5m
0.5 50% rule (Knuth 1968)
In Equilibrium, of the total number of occupied blocks and holes, 1/3 are holes.
Measure of Memory Utilization
In equilibrium, (increase one hole) (decrease one hole)P P(request) (release)P P
(1 )a d
pm m
(1 )a d p m
(1 )m d p m b c d (1 ) 2p m n
0.5n pm
0.5n
pm
0.5m
0.5 50% rule (Knuth 1968)
In Equilibrium, of the total number of occupied blocks and holes, 1/3 are holes.
How much memory is wasted?
(average) hole_size is not the same as
(average) block_size
How much memory is wasted?
(average) hole_size is not the same as
(average) block_size
Measure of Memory Utilization
0.5n pm 0.5m
The sum of hole sizes
Memory size ( )f
M
Assumeb: average block sizeh: average hole size
hk
b
nh
nh mb
0.5
0.5
mkb
mkb mb
2
k
k
Measure of Memory Utilization
The sum of hole sizes
Memory size ( )f
M
Assumeb: average block sizeh: average hole size
hk
b
2
k
k
1 if 1
3f k 50% rule (Knuth 1968)
Measure of Memory Utilization
The sum of hole sizes
Memory size ( )f
M
Assumeb: average block sizeh: average hole size
hk
b
2
k
k
We want to minimize the value of f.We want to minimize the value of f.
It is equivalent to minimize k.It is equivalent to minimize k.
How to know the value of k?How to know the value of k?
Measure of Memory Utilization
The sum of hole sizes
Memory size ( )f
M
Assumeb: average block sizeh: average hole size
hk
b
2
k
k
How to know the value of k?
Simulations by Knuth (1968) provide an answer. k depends on the ratio of b/M, e.g.,
b M/10, k = 0.22 f 0.1
b M/3, k = 2 f 0.5
Simulations by Knuth (1968) provide an answer. k depends on the ratio of b/M, e.g.,
b M/10, k = 0.22 f 0.1
b M/3, k = 2 f 0.5
Measure of Memory Utilization
The sum of hole sizes
Memory size ( )f
M
Assumeb: average block sizeh: average hole size
hk
b
2
k
k
Conclusion: M must be large
relative to b, otherwise much of main memory will remain unused.
Operating Systems PrinciplesMemory Management
Lecture 7: Physical Memory
Dealing with
Insufficient Memory
Managing Insufficient Memory
What to be done?– when the total size of free memory is large
enough while the memory requirement of a process exceeds the size of the largest partition
– when new process arrives with insufficient free memory?
– When the memory requirement is larger than the physical memory size.
Dealing with Insufficient Memory
Memory compaction– How much and what to move?
Swapping– Temporarily move process to disk– Requires dynamic relocation
Overlays– Allow programs larger than physical memory– Programs loaded as needed according to
calling structure.
Memory Compaction
Request for a memory block of size 10.
InitialComplete
CompactionPartial
CompactionMinimal dataMovement
Swapping
Swap in/out
Swap in/out
More efficient than memory compaction.– Memory compaction may fail to create a large enough
hole.– Effect only small number of processes each time, thus
requiring fewer memory accesses.– Swapping can be overlapped with process execution.– Swapping can be used with both fixed and variable size
partition, whereas memory compaction is applicable only for variable size partition.
What are needed to be swapped out?
• Code?
• Data?
Overlaps
Allow programs large than physical memory Programs loaded as needed, according to
calling structure.
A
B C
D E
A
k1
B
Overlaps
A
B C
D E
A
k1
BC
k2D
Allow programs large than physical memory Programs loaded as needed, according to
calling structure.
Overlaps
A
B C
D E
A
k1
BC
k2DE
Allow programs large than physical memory Programs loaded as needed, according to
calling structure.