Post on 08-Jan-2016
description
11
Randomization, Derandomization, Randomization, Derandomization, and Parallelization and Parallelization
--- take the MIS problem as a demonstration--- take the MIS problem as a demonstration
Speaker: Hong Chih-DuoSpeaker: Hong Chih-Duo
Advisor: Chao Kuen-MaoAdvisor: Chao Kuen-Mao
National Taiwan UniversityNational Taiwan UniversityDepartment of Information EngineeringDepartment of Information Engineering
2
The MIS problemThe MIS problem
►Finding a maximum independent set is NPC►The decision version of MIS is to decide the inde
pendence number α(G) of G -- still NPC►However, there are many good algorithms to app
roximate it, in various settings.►A naive greedy algorithm guarantees a (Δ/(Δ+
1))|OPT| solution. (cf. your textbook!)►Could tossing coins help in this respect?
3
A simple randomized algorithmA simple randomized algorithm
►For each vertex v, add v to I’ with probability p. (p is to be determined later)
►For each edge vu that both v, u I’, remove one ∊of them from I’ uniformly at random.
►The resulting I” is an independent set of G.
4
Step 1Step 1
Step 2Step 2
5
Good luck Bad luck Step 3Step 3
6
Average performance analysisAverage performance analysis
i
( ) ( )
2
2
1, if i 'Let random variable X
0, o.w.
Let
be a pessimistic estimate for the size of ".
Then [ ] .
Choose to maximize [ ] : [ ] .2 4
i i ji V G ij E G
i i j
I
Z X X X
I
E Z nE X mE X X np mp
n np E Z E Z
m m
7
Average performance analysisAverage performance analysis
2
:
For any graph G with n vertices and m edges,
G has an independent set of size n 4 .
:
Could we find this set efficiently?
Theorem
m
Question
8
Some refreshers from probability TheorySome refreshers from probability Theory
1 1
|
|
| , st. | .
| ,..., ,...,
|
x
n n
P X x A P X x A P A
E X A xP X x A
E Y X f X f x E Y X x
E Y X X f X X
X Y E X Y E X
9
Some refreshers from probability TheorySome refreshers from probability Theory
1 1 1 1
1
1 1
1 1
Corollary:
| ,..., | ,..., | ,...,
Since is fully determined by rvs ,..., ,
we may define ,..., | ,..., .
It then follows from the corollay that
,..., ,
r r r
n
n n
r r
E E Z X X X X E Z X X
Z X X
f X X E Z X X Z
E E f x x X
1,..., .rE f x x
► Theorem
10
1 1
1
1 11,0
1 11,0
1 1
1
Let denote the event ,..., .
|
| , | (by corollary)
| , |
| , (independency)
| , 1 | , 0 1
max | , 1 , | ,
r r r
r
r r r
r r r ri
r r ri
r r r r
r r r
A X x X x
E Z A
E E Z A X A
E Z A X i P X i A
E Z A X i P X i
E Z A X p E Z A X p
E Z A X E Z A
1 0rX
Effect of a single toss on E[Z]Effect of a single toss on E[Z]
11
1
1
1 10 1
21 1 1 0
1
(recall that )
But | , could be computed
in time!
If for each 0, we select 0,1 such that
| max | , | ,
then
| | 4 ,
r r
v v uv V vu E
r
r r r ri
E Z A X i
O m n Z X X X
n r x
E Z A E Z A X i E Z A
E f X x E Z A E Z A E Z n m
E f X
21 2 2 2 1
21 1 1
, | | 4 ,...
,..., | | 4 .n n n n
x X x E Z A E Z A n m
E f X x X x E Z A E Z A n m
12
A derandomized resultA derandomized result
►We derived a deterministic procedure to find an assignment {x1, x2,..., xn} that guarantees an indepen
dent set of size f(x1,...,xn) ≥ n2/4m.
►Note that we may have to prune I’ = {i : xi =1}
in order to get an independent set. (why?)►The argument is in fact from a general scheme call
ed the conditional probability method, which is very powerful in derandomizing probabilistic proofs.
13
Notes on conditional probability methodsNotes on conditional probability methods
► In general, it is hard to compute conditional expectations.
►There are many instance where there is no efficient way to compute the required conditional expectation.
►Moreover, the conditional probability method is inherently sequential: the variables xi’s are determined in a fixed order. As a result, the time complexity is Ω(n) even if we have an unbounded number of processors.
►Example
►PRAM
14
The length of computation path is The length of computation path is ΩΩ(n)(n)
X1=0
X2=1
Xn-1=1Xn=0
X3=0
Good points
height = n
15
““Compress” the computation pathCompress” the computation path
► It appears that the bottleneck lies in the number of random variables {Xi}. Do we really need n variables?
►Recall the definition of Z:
wherein X1,...,Xn are i.i.d..
►Note that if X1,...,Xn are pairwise independent, then also E[Z] = n2/4m.
►We may generate pairwise independent X1,...,Xn with fewer i.i.d. random variables!
i i ji V ij E
Z X X X
16
““Compress” the computation pathCompress” the computation path
1
i
1
Let ,..., be i.i.d. rvs, where lg ,
and each W is uniform distributed on [1..2m].
Redefine
1, if (mod 2 )
0, o.w.
where H is a subset of [1..2m] with size n
k
k it tt
i
W W k n
W m HIX
1
, and
,..., is the binary representation of .i ikI I i
1 1We may write ( ,..., ) as ( ,...., ).n kZ f X X Z h W W
17
1 1
1
1 1[1..2 ]
1 1[1..2 ]
1[1..2 ]
[1..2 ]
Let denote the event ,..., .
|
| , | (by corollary)
| , |
| , (independency)
1| ,
2
max | ,
r r r
r
r r r
r r r ri m
r r ri m
r ri m
ri m
A W W
E Z A
E E Z A W A
E Z A W i P W i A
E Z A W i P W i
E Z A W im
E Z A
1rW i
““Compress” the computation pathCompress” the computation path
18
1
1
1 1[1..2 ]
But | , could be computed in
( ) lg time with a single processor,
lg time with ( ) processors.
If for each 0, we select [1..2 ] such that
| max | ,
r r
r
r r ri m
E Z A W i
O m n n
O n O m n
n r m
E Z A E Z A W i
21 1 1 0
21 1 2 2 2 1
21 1 1
| ,
then
| | 4 ,
, | | 4 ,...
,..., | | 4 .
r
k k k k
E Z A
E h W E Z A E Z A E Z n m
E h W W E Z A E Z A n m
E h W W E Z A E Z A n m
19
The length of computation path is now O(lgThe length of computation path is now O(lgn)n)
W1=ω1
W2=ω2
Wk-1= ωk-1Wk=ωk
W3=ω3
Good points
height = k
= ceiling(lg n)
20
A parallelized resultA parallelized result
►We derived a deterministic parallel algorithm to find an independent set of size h(ω1,..., ωk) ≥ n2/4m, for a graph with n vertices and m edges.
►This algorithms can be implemented on an EREW-PRAM in O(lg2n) time with O(m2) processors.
►There is a high-level theorem indicating this fact:
If an RNC-algorithm works properly when the proIf an RNC-algorithm works properly when the probabilities are babilities are suitablysuitably approximated, then it can be approximated, then it can be converted to an equivalent NC-algorithm.converted to an equivalent NC-algorithm.
21
ReferenceReference
Fast Parallel Algorithms for Graph Matching Problems, M.Karpinski, W.Ryttter, p.104~115.
The Probabilistic Method, 2nd Edition, Noga Alon, J.Spencer, p.249 ~ 257.
Randomized Algorithms, R.Motwani, P.Raghavan, p.335~346.
22
Example: looking for the biggest determinantExample: looking for the biggest determinant
►An ∊ Mn[{+1,-1}]
►How big can | det(An) | be ??
►This is a famous (and unsolved) problem of Hadamard.
►Fact: |det(An)| ≦ nn/2. a corollary of Hadamard’s determinant theorem
23
Example: looking for the biggest determinantExample: looking for the biggest determinant
►Let’s toss coins!
►(M. Kac) A random matrix An of {+1, -1} has
E[ |det(AE[ |det(Ann)|)|22 ] = n! ] = n!
►So there exists an n×n matrix An that satisfies |det(An)| ≧ (n!)1/2.
►However, no one knows how to construct one efficiently.
Back
24
:
| | |
:
| |
| , |
, |
( , | )
| .
x
y x
y x
x y
x
Theorem
E E X Y Z E X Z
proof
LHS E xP X x Y Z
xP X x Y y Z P Y y Z
xP X x Y y Z
x P X x Y y Z
xP X x Z RHS
► Corollary
25
Some notes on parallel computation modelsSome notes on parallel computation models
►PRAM: a very general model that is interested mostly in exposing the parallel nature of problems.
►PRAM consists of a number of RAMs that work synchronous and communicate through a common random access memory.
►Typically, technical details such as synchroniza-tion and communication problems are ignored.
►The most “realistic” PRAM variant is EREW, allowing only exclusive-read and exclusive-write on the shared memory.
26
Some notes on parallel complexity classesSome notes on parallel complexity classes
►The main aim of parallel computing is the decrease of computation time. The main class of interests is NC = { problems computable in polylogarithmic time and polynomially many processors. }
►Perhaps the most important question in the theory of parallel computations is: is P = NC ?
► It is strongly believed that the answer is negative. However, this question could be of similar difficulty to the P = NP problem.
Back