I HC QUC GIA H NITRNG I HC CNG NGH

BO CO MNCH HIN I V HTTT

ti: Dominant Resource Fairness: Fair Allocation of Multiple Resource Type

Ging vin: TS. Nguyn Ngc Ha Nhm: 27 H v tn hc vin: Nguyn B Qun

iv

2

Tm tt Tiu lunTi nguyn ca h thng (CPU, b nh, thit b ngoi vi,...) vn rt gii hn, nhng trong cc h thng a nhim, nhiu ngi s dng c th ng thi yu cu nhiu ti nguyn. tha mn yu cu s dng ch vi ti nguyn hu hn v nng cao hiu qu s dng ti nguyn, cn phi c c ch v chin lc thch hp qun l vic phn phi ti nguyn.Ngoi yu cu dng chung ti nguyn tit kim chi ph, ngi s dng cn cn phi chia s thng tin (ti nguyn phn mm) ln nhau, khi cn m bo vic truy xut n cc ti nguyn ny l hp l, khng xy ra tranh chp, mt ng nht,...Tiu lun tp trung tm hiu phng php phn b ngun ti nguyn mt cch cng bng mi ngi s dng u c th s dng ti nguyn mt cch hp l.

1.IntroductionTrong mt h thng c cha cc loi ti nguyn khc nhau mi ngi s dng c th c nhu cu khc nhau cho mi ti nguyn. c th phn phi hp l ti nguyn ny ngi ta ra DRF( Phn phi cng bng ti nguyn). Mt cch tng qut y chnh l cch phn phi cng bng cho tt c cc ti nguyn trong my tnh. Khng ging nh chnh sch khc DRF chia s ti nguyn bng cch m bo rng khng c ngi dng no c u tin v ngi s dng khng th yu cu thm ti nguyn cho mnh nu nh s ti nguyn s dng cng nh khng th u tin phn b cho ngi ny v gim s phn b cho ngi s dng khc.Phn b ti nguyn l cha kha xy dng cc khi quan trng ca bt k s chia s no trong h thng my tnh. l mt trong nhng chnh sch phn phi ph bin nht n ny l phn phi cng bng Max- Min. chnh l phn phi Max- Min c bi ngi dng trong h thng. Gi s mi ngi dng c nhu cu chnh, sch ny cung cp cho mi ngi s dng mt phn bng nhau ca ti nguyn. Max-min s cng bng c khi qut ha bao gm cc khi nim v trng lng, trong mi ngi dng nhn c mt phn ca ti nguyn t l thun vi trng lng ca n.Quan trng ca cng bng max-min xut pht t tng qut ca n v kh nng ca mnh cung cp hiu sut cch ly. Cc m hnh cng bng max-min trng c th h tr mt lot cc chnh sch phn b ngun lc khc, trong u tin, t phng, v thi hn phn b da. Ngoi ra, trng lng s cng bng max-min m bo c lp, trong mt ngi dng c m bo nhn c phn ca mnh khng ph thuc vo nhu cu ca ngi s dng khc.Vi nhng tnh nng, n s n nh l khng c g ngc nhin khi mt s lng ln cc thut ton c xut thc hin (trng s) max-min cng bng vi mc khc nhau ca chnh xc, chng hn nh vng trn mt lt, chia s ti nguyn theo t l, V xp hng cng bng trng lng. Cc thut ton ny c p dng cho mt lot cc ti nguyn, bao gm bng thng lin kt B x l trung tm, b nh v lu tr.Mc d s lng ln cc cng vic v phn b cng bng, s tp trung cho n nay ch yu vo mt loi ti nguyn duy nht. Thm ch trong nhiu ti nguyn mi trng,ni ngi dng c nhu cu ngun lc khng ng nht, phn b thng c thc hin bng cch s dng mt ngun ti nguyn duy tru tng. V d, lp lch cng bng cho Hadoop v Dryad hai khung tnh ton cluster c s dng rng ri, phn b ngun lc cp phn vng c nh kch thc ca cc nt, gi l khe cm iu ny bt chp thc t rng cng vic khc nhau trong cc cm c th c nhu cu rt khc nhau cho CPU, b nh, v I / O cc ngun lc Qu trnh thc hin v nh gi DRF trong Mesos mt ngi qun l ti nguyn trong nhiu khung tnh ton cluster, nh Hadoop v MPI, c th chy. so snh DRF vi cng bng chng trnh chia s khe da trn s dng trong Hadoop v Dryad v cho thy rng khe da trn chia s cng bng c th dn n hiu sut km hn, khi lng cng vic nht nh trng pht bt cng, trong khi cung cp s bo m cch ly yuTrong bi bo ny tp trung vo vic phn b ngun lc trong trung tm d liu, v vy DRF l p dng chung cho cc mi trng a ngun ti nguyn khc, ni ngi dng c nhu cu khng ng nht, chng hn nh trong cc my a nhn.

2. Motivation

Trong khi vic trc y v quyn cng bng max-min tp trung vo cc ngun ti nguyn duy nht, s ra i ca cc b vi x l in ton m my v a li lm tng nhu cu i vi chnh sch phn b cho cc mi trng nhiu ngun lc v nhu cu ngi s dng khng ng nht. Bi nhiu ngun lc chng ti c ngha l ngun lc ca cc loi khc nhau, thay v nhiu trng hp ca cc ngun ti nguyn thay th cho nhau cng. thc y nhu cu i vi phn b a ti nguyn, chng ta v cc cu hnh s dng ti nguyn ca cc nhim v trong nm 2000-nt Hadoop nhm ti Facebook hn mt thng (thng 10 2010) trong Hnh 1. Cc v tr ca mt vng trn trong hnh 1 cho bit cc b nh v ti nguyn CPU c tiu th bi cc nhim v. Kch thc ca mt vng trn l logarit vi s nhim v trong khu vc ca hnh trn Mc d phn ln cc nhim v l CPU-nng, c tn ti nhng cng vic m b nh nng l tt, c bit l i vi cc hot ng gim

Hin lp lch cng bng cho cc cm, nh Quincy v Hadoop Fair Scheduler ,b qua tnh khng ng nht ca nhu cu ngi s dng, v phn b ngun lc mc chi tit ca khe, ni mt khe cm l mt phn c nh ca mt nt. iu ny dn n vic phn b khng hiu qu nh mt khe cm l thng xuyn hn khng phi l mt trn u ngho cho cc nhu cu cng vic.Hnh 2 nh lng mc cng bng v s bit lp ca cc Hadoop MapReduce cng bng lch [2, 34]. Con s ny cho thy CDF ca t l gia nhu cu cng vic CPU v CPU khe chia s, v cc t l gia nhu cu b nh nhim v v chia s b nh khe. Chng ti tnh ton b nh khe v c phiu CPU bng cch chia i tng s lng b nh v CPU do s lng cc khe. Mt t l 1 tng ng vi mt kt hp hon ho gia nhu cu cng vic v ngun lc khe, mt t l di 1 tng ng vi nhim v underutilizing ngun khe ca h, v mt t l trn 1 tng ng vi nhim v trn, khi s dng ngun khe ca h, trong c th dn n trn n. Hnh 2 cho thy rng hu ht cc nhim v hoc underutilize hoc overutilize mt s ti nguyn khe ca h. Thay i s lng cc khe trn mi my s khng gii quyt vn nh th ny c th dn n mt trong hai trong mt s dng tng th thp hn hoc nhiu nhim v experiencingpoor hiu sut do vic s dng qu (Phn ee s 7).

3 Allocation PropertiesBy gi chng ta chuyn s ch ca chng ti thit k mt chnh sch phn b cng bng max-min cho nhiu ti nguyn v yu cu khng ng nht. minh ha vn , xem xt mt h thng bao gm 9 CPU v 18 GB RAM, v hai ngi s dng: Mt ngi s dng chy cc tc v i hi CPU h1, 4 GBI nhau, v ngi s dng B chy tc v yu cu CPU h3, 1 GBI mi. To nn ci m mt chnh sch phn b cng bng i vi trng hp ny?Mt kh nng c th phn b cho mi hip ca ngi s dng mi ngun lc. Mt kh nng khc l cn bng tng hp (v d, CPU vi b nh) phn b cho mi ngi dng. Trong khi n l tng i d dng n vi mt lot cc kh nng phn b "cng bng", n l khng r rng nh th no nh gi v so snh cc phn b. gii quyt thch thc ny, chng ta bt u vi mt tp hp cc tnh cht mong mun rng chng ti tin rng bt k chnh sch phn b ngun lc cho nhiu ti nguyn v nhu cu khng ng nht nn p ng. Sau chng ti mi nhng c tnh hng dn s pht trin ca mt chnh sch phn b cng bng. Chng ti tm thy bn thuc tnh sau y l quan trng:1. Chia s khuyn khch: Mi ngi s dng nn c tt hn off chia s cc cluster, hn c quyn s dng phn vng ring ca mnh ca cluster. Hy xem xt mt cluster vi cc nt v n ngi dng ging nhau. Sau , ngi dng khng nn c th phn b nhim v nhiu hn trong mt phn vng cm gm 1 / n ca tt c cc ngun lc.2. Chin lc-proofness: Ngi dng khng nn c th hng li bng cch ni di v nhu cu ti nguyn ca h. iu ny cung cp kh nng tng thch khuyn khch, nh mt ngi s dng khng th ci thin vic phn b ca mnh bng cch ni di3. Envy-freeness: Mt ngi s dng khng nn thch vic phn b ca ngi dng khc. Khch sn ny l hin thn ca nim v s cng bng [13, 30].4. hiu qu Pareto: N khng phi l c th lm tng s phn b ca mt ngi s dng m khng lm gim s phn b t nht mt ngi dng khc. Khch sn ny l quan trng v n dn n vic ti a ha i tng s dng h thng p ng cc thuc tnh khc

Chng ti cng bnh lun v chin lc proofness v chia s khch l ti sn, m chng ti cho l c tm quan trng c bit trong mi trng trung tm d liu. Bng chng t cc nh khai thc in ton m my m chng ti ni chuyn vi ch ra rng chin lc proofness l quan trng, v n l ph bin i vi ngi dng c gng thao tc lp lch. V d, mt trong nhng trung tm d liu Hadoop MapReduce ca Yahoo! c cc s khc nhau ca khe cho bn v gim nhim v. Mt ngi s dng pht hin ra rng cc khe bn c tranh lun, v do a ra tt c cc cng vic ca mnh nh gim di giai on, trong s t lm nhng cng vic m MapReduce hin trong giai on bn ca n. Mt cng ty cung cp my tm kim ln dnh ring cho cng vic ch nu ngi s dng c th m bo s dng cao. Cng ty ny sm pht hin ra rng ngi dng s rc m ca h vi cc vng trong hu hn nhn to thi phng mc s dng.Hn na, bt k chnh sch p ng cc bt ng sn khuyn khch chia s cng cung cp hiu nng c lp, v n bo m vic phn b ti thiu cho mi ngi dng (v d, mt ngi s dng c th khng lm ti t hn vic s hu 1 / n ca cluster) m khng tnh n nhu cu ca ngi s dng khc.N c th d dng thy rng trong trng hp ca mt ti nguyn duy nht, p ng s cng bng max-min tt c cc properties.However trn, t c nhng ti sn trong trng hp nhiu ngun lc v nhu cu ngi s dng khng ng nht l khng nh. V d, cc c ch u i cng bng phn chia trong l thuyt kinh t vi m, cn bng cnh tranh t Thu nhp bnh ng [22, 30, 33], khng strategyproofNgoi cc c tnh trn, chng ti xem xt c tnh khc c tnh cng bng ti nguyn: i vi mt ngun duy nht, cc gii php cn gim cng bng max-min. s cng bng nt c chai: Nu c mt ngun ti nguyn l phn trm-khn ngoan i hu ht bi mi ngi dng, sau cc gii php gim nn cng bng max-min cho ti nguyn . Dn s n iu: Khi mt ngi s dng ri khi h thng v tuyn b t b cc ngun lc ca mnh, khng ai trong s cc phn b ca ngi s dng cn li s gim. n iu Resource: Nu c nhiu ngun ti nguyn c b sung vo h thng, khng c s phn b ca ngi dng hin ti s gim.4. Dominant Resource Fairness (DRF).Chng ti xut Dominant Resource Cng bng (DRF), mt chnh sch phn b mi cho nhiu ti nguyn p ng tt c bn ca cc thuc tnh cn thit trong cc phn trc. i vi mi ngi s dng, DRF tnh th phn ca mi ti nguyn c phn b cho ngi . Ti a trong tt c cc c phiu ca mt ngi dng c gi l c phn chi phi ca ngi dng, v cc ngun ti nguyn tng ng vi phn chi phi c gi l ti nguyn thng tr. Ngi s dng khc nhau c th c cc ngun ti nguyn khc nhau chi phi. V d, cc ti nguyn thng tr ca mt ngi dng chy mt cng vic tnh ton b rng buc l CPU, trong khi cc ngun ti nguyn thng tr ca mt ngi dng chy mt I / O cng vic b rng buc l bandwidth.1 DRF n gin ch p cng bng maxmin qua c phn chi phi ca ngi s dng. l, DRF tm cch ti a ha th phn chim u th nh nht trong h thng, sau th hai nh nht, v nh vy.

Chng ti bt u bng cch minh ha DRF vi mt v d (4.1), sau trnh by mt thut ton cho (4.2) DRF v mt nh ngha v trng (4.3) DRF. Trong phn 5, chng ti trnh by hai chnh sch phn b khc: cng bng ti sn, mt chnh sch n gin nhm mc ch cn bng tng hp ngun lc c phn b cho mi ngi dng, v cn bng cnh tranh t thu nhp ngang nhau (CEEI), mt ph bin chnh sch phn b cng bng a thch trong kinh t vi m minTrong phn ny, chng ta xem xt mt m hnh tnh ton vi n ngi dng v ti nguyn m. Mi ngi s dng chy cc nhim v c nhn, v mi cng vic c c trng bi mt vector nhu cu, xc nh s lng ti nguyn theo yu cu ca cng vic, v d nh, CPU h1, 4 GBI. Ni chung, cng vic (thm ch nhng ngi thuc cng mt ngi dng) c th c nhu cu khc nhau.4.1 An Example.Hy xem xt mt h thng vi 9 CPU, 18 GB RAM, v hai ngi s dng, ngi dng A chy nhim v vi h1 CPU vector nhu cu, 4 GBI, v ngi s dng B chy nhim v vi CPU h3 vector cu, 1 GBI miTrong trng hp trn, mi cng vic t ngi s dng A tiu th 1/9 tng s CPU v 2/9 ca tng s b nh, v vy ngi dng ca mt ti nguyn thng tr l b nh. Mi nhim v t ngi dng B tiu th 1/3 tng s CPU v 1/18 ca tng b nh, v vy ngi s dng ti nguyn ominant B l CPU. DRF s bng c phn chi phi ca ngi s dng, cho vic phn b trong hnh 3: ba nhim v cho ngi s dng A, vi tng s CPU h3, 12 GBI, v hai nhim v cho ngi s dng B, vi tng s CPU h6, 2 GBI. Vi vic phn b ny, mi ngi s dng kt thc vi phn vt tri cng, tc l, ngi dng A c 2/3 b nh RAM, trong khi ngi dng B c 2/3 ca CPU.S phn b ny c th c tnh ton hc nh sau. Cho x v y l s cc nhim v c phn b bi DRF cho ngi dng A v B, tng ng. Sau , ngi dng A nhn CPU hx, 4x GBI, trong khi ngi dng B c CPU h3y, y GBI. Tng s lng ti nguyn c phn b cho c ngi dng l (x + 3Y) CPU v (4x + y) GB. Ngoi ra, cc c phn chi phi ca ngi dng A v B l 4x / 18 = 2x / 9 v 3Y / 9 = y / 3, tng ng (c phiu tng ng ca h v b nh v CPU). Sau phn b DRF c a ra bi cc gii php cho cc vn ti u ha sau y:

Gii quyt vn ny yields2 x = 3 v y = 2. Do , ngi dng A c CPU h3, 12 GBI v B c CPU h6, 2 GBI.Lu rng DRF khng cn lun cn bng c phn chi phi ca ngi s dng. Khi tng nhu cu ca ngi dng c p ng, ngi dng s khng cn nhiu nhim v hn, v vy cc ngun lc d tha s c chia u cho nhng ngi s dng khc, ging nh trong s cng bng max-min. Ngoi ra, nu mt ngun ti nguyn b cn kit, ngi dng khng cn ti nguyn c th vn tip tc nhn c phiu cao hn cc ngun ti nguyn khc. Chng ti trnh by mt thut ton phn b DRF trong phn tip theo.

4.2 DRF thut ton lp lch.

Thut ton 1 lm pseudo-code cho cc thut ton DRF scheduling.The di tng ngun lc phn b cho tng ngi s dng cng nh phn chi phi ca ngi dng, si. Ti eachstep, DRF kn ngi dng vi phn chi phi thp nht trong s nhng ngi c nhim v sn sng chy. Nu nhu cu cng vic ca ngi dng c th hi lng, tc l, c ngun lc sn c trong h thng, mt trong nhng nhim v ca c c tung ra. Chng ti xem xt trng hp tng qut trong ngi dng c th c nhim v vi vect nhu cu khc nhau, v chng ti s dng bin Di biu th vector nhu cu ca ngi s dng nhim v tip theo ti mun khi ng. n gin, gi m khng nm bt c s kin ca mt kt thc nhim v. Trong trng hp ny, ngi s dng pht hnh cc ngun lc ca cng vic v DRF li la chn cho ngi s dng vi cc phn chi phi nh nht chy cng vic ca mnh.Hy xem xt v d hai ngi dng trong phn 4.1. Bng 1 m t cc qu trnh giao DRF v d ny. DRF u tin chn B chy mt nhim v. Kt qu l, cc c phiu ca B tr h3 / 9, 1 / 18i, v c phn chi phi tr nn max (3/9, 1/18) = 1/3. Tip theo, DRF chn A, nh phn chi phi mnh l 0. Qu trnh ny tip tc cho n khi n khng cn c th chy cc nhim v mi. Trong trng hp ny, iu ny xy ra ngay sau khi CPU b bo ha.Vo cui ca vic phn b trn, ngi dng A c CPU h3, 12 GBI, trong khi ngi dng B c CPU h6, 2 GBI, tc l, mi ngi s dng c 2/3 ti nguyn thng tr ca mnh.Lu rng trong v d ny, phn b dng li ngay khi bt k ti nguyn c bo ha. Tuy nhin, trong trng hp chung, n c th l c th tip tc giao nhim v thm ch sau khi mt s ti nguyn c bo ha, nh mt s nhim v c th khng c bt k nhu cu v ti nguyn bo ha.Cc thut ton trn c th c thc hin bng cch s dng mt ng nh phn m cc ca hng c phn chi phi ca mi ngi dng. Mi quyt nh lp k hoch sau phi mt O (log n) thi gian cho n ngi dng.4.3 weighted DRF.Trong thc t, c rt nhiu trng hp trong phn b ngun lc u trn tt c ngi s dng khng phi l policy.Instead mong mun, chng ta c th phn b nhiu ti nguyn hn cho ngi s dng chy cc cng vic quan trng hn, hoc cho ngi dng ng gp thm ngun lc cc cluster. t c mc tiu ny, chng ti xut trng DRF, mt s tng qut ca c DRF v trng s cng bng max-min.Vi trng DRF, mi i dng c lin quan n mt vector trng lng Wi = (wi, 1,..., Wi, mi), ni wi, j i din cho trng lng ca ngi dng i cho ti nguyn j. Cc nh ngha ca mt c phn chi phi cho ngi dng i thay i si = maxj {ui, j / wi, j}, ni ui, j l ngi s dng ti chia s ca ti nguyn j. Mt trng hp c bit quan tm l khi tt c cc trng s ca ngi s dng ti u bnh ng, tc l, wi, j = wi (1 j m). Trong trng hp ny, t l gia cc c phn chi phi ca ngi dng i v j s ch n gin l wi / wj. Nu trng lng ca tt c ngi dng c thit lp 1, trng DRF gim trivially DRF.5. Alternative Fair Allocation Policies.Xc nh phn b cng bng trong mt h thng a ngun ti nguyn khng phi l mt cu hi d dng, nh khi nim "cng bng" l t m tho lun. Trong nhng n lc ca chng ti, chng ti xem xt nhiu chnh sch phn b trc khi quyt nh DRF nh l ch tha mn tt c bn ca cc thuc tnh cn thit ti mc 3: chia s khch l, chin lc proofness, hiu qu Pareto, v ghen t-freeness. Trong phn ny, chng ta xem xt hai trong s nhng la chn thay th, chng ti iu tra: Cng bng ti sn, mt chnh sch n gin v trc quan m nhm mc ch cn bng tng hp ngun lc c phn b cho mi ngi dng, v cn bng cnh tranh t Thu nhp bnh ng (CEEI), cc chnh sch v la chn kh phn b ngun lc trong lnh vc kinh t vi m.5.1 Cng bng ti sn. tng ng sau ti sn Cng bng l nhng phn bng nhau ca cc ngun ti nguyn khc nhau c gi tr nh nhau, tc l 1% ca tt c cc CPU c gi tr l ging nh 1% b nh v 1% / O bng thng I. Cng bng ti sn sau c gng cn bng cc gi tr ti nguyn tng hp phn b cho mi ngi dng. c bit, Cng bng ti sn tnh cho mi ngi dng i phn theaggregate xi = Pj si, j, ni si, j l chia s ti nguyn j cho ngi dng i. Sau n c p dng max-min trn c phiu tng hp ca ngi s dng, v d, n lin tc tung ra cc nhim v cho ngi s dng vi cc phn tng hp ti thiu.Hy xem xt cc v d trong phn 4.1. V c hai ln nh nhiu GB ca RAM nh CPU (tc l, 9 CPU v 18 GB RAM), mt CPU c gi tr gp i so vi mt GB b nh RAM. Gi s rng mt trong GB tr gi $ 1 v mt CPU l gi tr $ 2, sau ngi dng A dnh 6 $ cho mi cng vic, trong khi ngi dng B dnh $ 7. Cho x v y l cc s nhim v giao ti sn Cng bng cho ngi s dng A v B, tng ng. Sau , vic phn b ti sn hp l c a ra bi cc gii php cho cc vn ti u ha sau y:

Gii quyt cc vn trn sn lng x = 2,52 v y = 2,16. V vy, ngi dng A c CPU h2.5, 10.1 GBI, trong khi ngi dng B c CPU h6.5, 2.2 GBI, tng ng.Trong khi chnh sch phn b ny c v hp dn trong s n gin ca n, n c mt nhc im quan trng: n vi phm s hu chia s khch l. Nh chng ta thy trong Phn 6.1.1, cng bng ti sn c th dn n mt ngi s dng nhn c t hn 1 / n ca tt c cc ngun lc, trong n l tng s ngi dng.5.2 Cn bng cnh tranh t Thu nhp bnh ng.Theo l thuyt kinh t vi m, phng php a thch phn chia cc ngun lc l kh cn bng cnh tranh t Thu nhp bnh ng (CEEI) [22, 30, 33]. Vi CEEI, mi ngi s dng nhn ban u 1 / n ca mi ti nguyn, v sau , mi ngi s dng ti nguyn ngh ca mnh vi nhng ngi dng khc trong mt market.3 cnh tranh hon ho Kt qu ca CEEI l c ghen t min ph v hiu qu Pareto.Chnh xc hn, vic phn b CEEI c a ra bi cc thng lng Nash solution4 [22, 23]. Cc gii php thng lng Nash chn phn b kh thi nhm ti a ha Qi ui (ai), ni ui (ai) l tin ch cho ngi s dng ti nhn c t vic giao c ai. n gin ha vic so snh, chng ti gi nh rng cc tin ch m ngi dng nhn c t vic giao c ch n gin l phn chi phi, si ca mnh.Xem xt li cc v d hai ngi dng trong phn 4.1. Nh li rng c phn chi phi ca ngi A 4x / 18 = 2x / 9, trong khi cc c phn chi phi ca ngi dng B l 3Y / 9 = y / 3, trong x l s lng nhim v c giao A v y l s nhim v c giao B. Ti a ho cc sn phm ca cc c phn chi phi l tng ng vi ti a ha cc sn phm x y. Nh vy, CEEI nhm gii quyt cc vn ti u ha sau y.

Gii quyt cc vn trn sn lng x = 45/11 v y = 18/11. V vy, ngi dng A c CPU h4.1, 16,4 GBI, trong khi ngi dng B c CPU h4.9, 1.6 GBI.

Tht khng may, trong khi CEEI l ghen t min ph v Pareto cch hiu ficient, n ch ra rng n khng phi l chin lc bng chng, nh chng ta s thy trong phn 6.1.2. Do , ngi dng c th tng phn b ca h bng cch ni di v nhu cu ti nguyn ca h.5.3 So snh vi DRF. cung cp cho ngi c mt s hiu bit trc quan ca ti sn Cng bng v CEEI, chng ta so snh phn b ca h v d trong phn 4.1 m ca DRF trong hnh 4.Chng ti thy rng DRF equalizes c phn chi phi ca ngi s dng, v d, ngi s dng ca mt phn b nh v s dng CPU phn B ca. Ngc li, ti sn Cng bng equalizes tng phn nh ti nguyn c phn b cho mi ngi dng, v d, cc khu vc ca hnh ch nht cho mi ngi dng trong hnh. Cui cng, v CEEI gi nh mt th trng cnh tranh hon ho, n tm thy mt gii phng mt bng gii php th trng p ng, ni mi ngun lc c phn b. Tht khng may, bt ng sn chnh xc iu ny lm cho n c th la CEEI: ngi dng c th yu cu c y cn thm mt s ti nguyn s dng ng mc, ngay c khi c y khng, dn CEEI cung cp cho nhiu nhim v tng th cho ngi dng ny t c gii phng mt bng th trng.6 Analysis.Trong phn ny, chng ti tho lun m cc thuc tnh trnh by trong phn 3 c hi lng bi Asset Cng bng, CEEI, v DRF. Chng ti cng nh gi tnh chnh xc ca DRF khi kch thc cng vic khng ph hp vi cc ngun lc sn c chnh xc.6.1 Fairness Properties.

Bng 2 tm tt cc thuc tnh cng bng c hi lng bi Asset Cng bng, CEEI, v DRF. Phn ph lc c cha cc bng chng ca cc thuc tnh chnh ca DRF, trong khi bo co k thut ca chng ti [14] cha mt danh sch y hn v kt qu cho DRF v CEEI. Trong phn cn li ca phn ny, chng ti tho lun v mt s cc mc mt tch th v trong bng, v d, tnh cht vi phm ca tng mn hc. c bit, chng ti hin th thng qua cc v d ti sao ti sn Cng bng v CEEI thiu cc ti sn m h lm, v chng ti chng minh rng khng c chnh sch c th cung cp ti nguyn m khng n iu iolating hoc chia s khuyn khch hoc hiu qu Pareto gii thch l do ti sao DRF thiu n iu ti nguyn.

6.1.1 Properties Violated by Asset Fairness.Trong khi c cc chnh sch n gin, ti sn Cng bng l vi phm mt s thuc tnh quan trng: s khuyn khch chia s, cng bng nt c chai, v n iu ti nguyn. Tip theo, chng ti s dng v d cho thy cc vi phm ca cc c tnh ny.nh l 1 ti sn vi phm Cng bng ti sn chia s khch l.Proof xem xt v d sau y, minh ha trong Hnh 5: hai ngi s dng trong mt h thng vi h30, tng ngun 30I c nhu cu vect D1 = h1, 3i, v D2 = h1, 1i. Cng bng ti sn s phn b s dng u tin 6 nhim v v ngi dng th hai 12 nhim v. Ngi dng u tin s nhn c h6, ngun 18i, trong khi th hai s s dng H12, 12i.While vi mi ngi dng mt phn tng hp bng nhau ca 24 60, ngi dng th hai c t hn mt na (15) ca c hai ngun lc. iu ny vi phm ti sn chia s khch l, l ngi dng th hai s c tt hn phn vng tnh cluster v mt na ca ring ca cc nt.nh l 2 ti sn Cng bng l vi phm s hu cng bng nt c chai.Proof xem xt mt kch bn vi mt vector ngun tng s H21, 21i v hai ngi dng vi nhu cu vect D1 = h3, 2i v D2 = h4, 1i, lm cho ti nguyn 1 s cng bng nt c chai resource.Asset s cung cp cho mi ngi dng 3 nhim v, cn bng tng ca chng s dng n 15. Tuy nhin, iu ny ch cung cp cho ngi s dng u tin 3 / 7of ngun 1 (cc ti nguyn nt c chai tranh), vi phm cng bng nt c chai.

Proof Xt hai ngi A v B vi nhu cu h4, 2i v h1, 1i v 77 n v ca hai ngun ti nguyn. Cng bng ti sn phn b A tng cng h44, 22i v B H33, 33i vchtng ca cc c phiu 66/77. Nu ti nguyn hai l tng gp i, chia s cc ngun ti nguyn th hai c hai ngi s dng gim i mt na, trong khi ti nguyn u tin c bo ha. Cng bng ti sn doanh nghip gim phn b A ti h42, 21i v tng ca B h35, 35I, cn bng t s c phn ca mnh 42/77 +21/154 = 35/77 +35/154 = 105/154 Nh vy ti nguyn n iu c vi phm.6.1.2 Thuc tnh Vi phm bi CEEI.Trong khi CEEI l ghen t min ph v hiu qu Pareto, n ch ra rng n khng phi l bng chng chin lc. Bng trc gic, iu ny l bi v CEEI gi nh mt th trng cnh tranh hon ho m chieves gii phng mt bng th trng, tc l, ph hp vi cung cu v phn b ca tt c cc ngun lc sn c. iu ny c th dn n CEEI cho c phiu cao hn nhiu cho ngi dng s dng nhiu hn mt t ti nguyn-tranh s dng y cc ngun ti nguyn . Do , ngi dng c th khng nh rng c y cn nhiu hn ca mt s ti nguyn s dng ng mc tng th phn tng th ca mnh v ti nguyn. Chng ti minh ha di ynh l 4 CEEI khng phi l chin lc chng.

Proof xem xt v d sau y, th hin trong hnh 6. Gi s mt vector ngun tng s H100, 100i, v hai ngi dng c nhu cu H16, 1i v h1, 2i. Trong trng hp ny, CEEI giao 100/31 v 1500-1531 nhim v cho tng ngi s dng tng ng (khong 3,2 v 48,8 nhim v). Nu ngi s dng thay i 1 vector nhu cu ca mnh H16, 8I, i hi nhiu hn cc ngun ti nguyn 2 hn c thc s cn, CEEI cung cp cho cc ngi dng v 25/6 100/3 nhim v tng ng (khong 4,2 v 33,3 nhim v). V vy, ngi s dng ci thin 1 s nhim v 3,2-4,2 c ni di v vector nhu cu ca mnh. User 2 b v iu ny, nh phn cng nhim v ca mnh gim i.Ngoi ra, vi l do cng trc quan (gii phng mt bng th trng), chng ta c cc kt qu sau:nh l 5 CEEI vi phm n iu dn.Proof Xem xt tng H100 vector ngun, 100i v ba ngi dng vi cc vect cu sau D1 = h4, 1i, D2 = h1, 16I, v D3 = H16, 1i (xem hnh 7) .CEEI s mang li vic phn b A1 = h11. 3, 5.4, 3.1i, ni cc con s trong ngoc n i din cho s ca nhim v c phn b cho mi ngi dng. Nu ngi s dng 3 ri khi h thng v tuyn b t b ti nguyn ca mnh, cung cp cho cc CEEI mi phn b A2 = h23.8, 4.8i, m lm cho ngi s dng 2 ti t hn trong A1.6.1.3 Resource n iu so vi u i Chia s v hiu qu ParetoNh th hin trong Bng 2, DRF t c tt c cc thuc tnh tr n iu ti nguyn. Thay v l mt hn ch ca DRF, y l mt h qu ca thc t l khuyn khch chia s, hiu qu Pareto, v n iu ti nguyn khng th t c cng mt lc. K t khi chng ti xem xt hai u tin ca nhng ti sn ny l quan trng hn (xem phn 3) v t khi b sung ngun lc mi cho mt h thng l mt s kin tng i him, chng ti chn chia s khch l p ng v hiu qu Pareto, v b n iu ti nguyn. c bit, chng ti c cc kt qu sau.nh l 6 Khng c chnh sch giao p ng cc khuyn khch chia s v tnh hiu qu Pareto cng c th p ng n iu ti nguyn.Proof Chng ti s dng mt v d n gin chng minh property.Consider ny hai ngi A v B i xng vi nhu cu h2, 1i, v h1, 2i, tng ng, v gi nh bng lng ca c hai ngun lc. Chia s khuyn khch yu cu ngi dng A c t nht mt na ti nguyn 1 v s dng B c mt na ngun 2. By hiu qu Pareto, chng ta bit rng t nht mt trong hai ngi phi c phn b thm ngun lc. Khng mt tnh tng qut, gi s rng ngi dng A c a ra hn mt na ngun 1 (mt i s i xng gi nu s dng B c a ra hn mt na ti nguyn 2). Nu tng lng ti nguyn 2 by gi tng theo h s 4, ngi B khng cn nhn c phn uaranteed ca mt na s ti nguyn 2. By gi, vic phn b ch kh thi p ng cc khuyn khch chia s l cho c ngi s dng mt na s ti nguyn 1, m s yu cu gim ser 1 chia s ca ti nguyn 1, do vi phm ti nguyn n iu.nh l ny gii thch ti sao c DRF v n iu CEEI vi phm ti nguyn.6.2 Discrete Resource Allocation.Cho n nay, chng ta mc nhin tha nhn mt h bi ti nguyn ln c ngun ti nguyn c th c phn b mt lng nh ty . Tt nhin, iu ny thng khng phi l trng hp trong practice.For d, cm gm nhiu my nh, ni cc ngun lc c phn b cho cc nhim v vi s lng ri rc. Trong li nhc nh ca phn ny, chng ti cp n hai kch bn ny l lin tc, v kch bn ri rc, tng ng. By gi chng ta chuyn s ch ca chng ti nh th no cng bng c nh hng trong kch bn ri rc.Gi s mt cm gm my K. Hy ti a tc v biu th vector ti a nhu cu trn tt c cc vect nhu cu, tc l, max-task = {hmaxi di, 1}, {maxi di, 2}, , maxi {di, m} i. Gi s thm rng bt k nhim v c th c d kin trn mi my, tc l, tng s lng ti nguyn trn mi my tnh l t nht ti a tc v. Chng ti ch xem xt cc trng hp khi mi ngi dng c nhu cu nghim ngt tch cc. Vi nhng gi nh, chng ta c kt qu sau.nh l 7 Trong kch bn ri rc, c th phn b cc ngun lc nh vy m s khc bit gia cc phn b ca hai ngi c bao bc bi mt max-nhim v so vi kch bn phn b lin tc.Proof Gi s chng ta bt u phn b ngun lc vo mt my ti mt thi gian, v rng chng ti lun dnh mt nhim v cho ngi s dng vi cc phn chi phi thp nht. Min l c t nht mt ti a tc v c sn trn my tnh u tin, chng ti tip tc giao nhim v cho ngi s dng tip theo vi phn vt tri nht. Mt khi cc ngun lc sn c trn my tnh u tin tr nn t hn mt kch thc ti a tc v, chng ti di chuyn n cc my tip theo v lp li qu trnh ny. Khi hon thnh vic phn b, s khc bit gia hai phn b ca ngi s dng ti nguyn thng tr ca h so vi cc kch bn lin tc nhiu nht l ti a tc v. Nu y khng phi l trng hp, sau mt s ngi dng A s c nhiu hn max-nhim v khc bit wrt cho mt ngi dng B. Tuy nhin, iu ny khng th l trng hp, v thi gian qua A c giao mt nhim v, B nn c giao mt nhim v thay th.

7. Kt qu thc nghim.Phn ny nh gi DRF qua vi v macrobenchmarks. Cc cu c thc hin thng qua cc th nghim chy mt thc thi ca DRF trong qun l ti nguyn cm Mesos [16]. Sau ny c thc hin bng cch s dng m phng tracedriven.Chng ti bt u bng cch hin th DRF ng iu chnh cc c phiu ca cc cng vic c nhu cu ngun lc khc nhau ti mc 7.1. Trong phn 7.2, chng ta so snh vi khe DRF cp chia s cng bng (nh thc hin bi Hadoop Fair Scheduler [34] v Quincy [18]), v CPU ch chia s cng bng. Cui cng, ti mc 7.3, chng ta s dng Facebook so snh du vt DFT v Fair Scheduler ca Hadoop v s dng v thi gian hon thnh cng vic.7.1 Dynamic Resource Sharing.Trong th nghim u tin ca chng ti, chng ti hin th nh th no DRF ng chia s ti nguyn gia cc cng vic c yu cu khc nhau. Chng ti gp hai cng vic trn 48-nt Mesos cluster trn Amazon EC2, s dng "cc ln" hp vi 4 li CPU v 15 GB b nh RAM. Chng ti cu hnh mesos phn b ln n 4 CPU v 14 GB b nh RAM trn mi nt, li 1 GB cho h iu hnh. Chng ti gi hai cng vic m a ra cc nhim v vi nhu cu ngun lc khc nhau vo nhng thi im khc nhau trong mt khong thi gian 6 pht.Con s 8 (a) v 8 (b) cho thy CPU v b nh phn b cho tng cng vic nh l mt hm ca thi gian, trong khi hnh 8 (c) cho thy c phn chi phi ca h. Trong 2 pht u tin, vic s dng 1 CPU h1, 10 GB Rami mi nhim v v cng vic ca 2 s dng CPU h1, 1 GB Rami mi nhim v. Ngun lc chi phi cng vic ca 1 l RAM, trong khi ti nguyn thng tr cng vic ca 2 l CPU. Lu rng DRF equalizes c phiu ca cc cng vic ca cc ngun ti nguyn thng tr ca h. Ngoi ra, v cng vic c ngun lc chi phi khc nhau, c phn chi phi ca h vt qu 50%, tc l, vic s dng 1 khong 70% ca RAM trong khi vic s dng 2 khong 75% ca CPU. V vy, cc cng vic c hng li t chy trong mt cm chia s ch khng phi dng mt na cc nt tng. iu ny nm bt c bn cht ca ti sn chia s u i.Sau 2 pht, cc kch c nhim v ca c hai cng vic thay i, CPU h2, 4 GBI cho cng vic v 1 CPU h1, 3 GBI cho cng vic 2. By gi, ti nguyn thng tr c hai cng vic "l CPU, v vy DRF equalizes c phn CPU ca h. Lu rng DRF tc phn b t ng bi c Mesos phc v ngun lc cho cng vic vi cc phn chi phi nh nht l nhim v hon thnh.Cui cng, sau hn 2 pht, cc kch c nhim v ca c hai cng vic thay i mt ln na: CPU h1, 7 GBI cho cng vic v 1 CPU h1, 4 GBI cho cng vic 2. ti nguyn thng tr ca c hai vic lm by gi l b nh, do DRF c gng cn bng tr nh ca h c phiu. L do cc c phiu khng chnh xc bng nhau l do s phn mnh ti nguyn (xem mc 6.2).7.2 DRF vs. Alternative Allocation Policies.Tip theo chng ti nh gi DRF i vi hai phng n thay th vi: lp lch cng bng khe da trn (mt chnh sch chung trong cc h thng hin ti, chng hn nh Hadoop Fair Scheduler [34] v Quincy [18]) v (max-min) chia s cng bng ch p dng cho mt ngun duy nht (CPU). Vi th nghim, chng ti chy mt 48-nt Mesos cluster trn EC2 vi 8 li CPU v 7 GB RAM mi. Chng ti cu hnh mesos phn b 8 CPU v RAM 6 GB trn mi nt, li 1 GB min ph cho h iu hnh. Chng ti thc hin cc chnh sch lp lch trnh ba l module phn b Mesos.Chng ti chy mt khi lng cng vic vi hai lp ngi dng, i din cho hai n v t chc vi workloads.One khc nhau ca cc i tng bn ngi dng gi cc cng vic nh vi nhim v i hi CPU h1, 0,5 GBI. Cc khc en-tity c bn ngi dng gi cc cng vic ln vi CPU nhim v demandsh2, 2 GBI. Mi cng vic bao gm 80 tasks.As ngay sau khi hon thnh mt cng vic, ngi s dng s khi ng mt cng vic khc c nhu cu tng t. Mi th nghim chy cho mi pht. Cui cng, chng ti tnh ton s lng cng vic hon thnh ca tng loi, cng nh thi gian phn ng ca h.

i vi cc n phn b khe-based, chng ti thay i s lng cc khe trn mi my 3-6 xem lm th no n nh hng n hiu sut. Hnh 9 n 12 cho thy kt qu ca chng ti. Trong hnh 9 v 10, chng ta so snh s lng vic lm ca mi loi hon thnh cho mi chng trnh lp k hoch trong mi pht. Trong hnh 11 v 12, chng ta so snh thi gian p ng trung bnh.Mt s xu hng ny l r rng t d liu. u tin, vi lch khe da trn, c hai thng lng v p ng cng vic thi gian c ti t hn vi DRF, khng ph thuc vo s lng cc khe. iu ny l bi v vi mt s khe thp, chc nng lch c th undersubscribe nt (v d,, khi ng ch c 3 nhim v nh trn mt nt), trong khi vi mt s khe ln, n c th oversubscribe chng (v d, chy 4 nhim v ln vo mt nt v gy ra trao i bi v mi cng vic cn 2 GB v cc nt ch c 6 GB). , Vi th hai chia s cng bng cp ca CPU, s lng vic lm nh thc hin tng t nh DRF, nhng c rt t vic lm ln thc hin, bi v b nh hn cam kt trn mt s my mc v dn n hot ng km hiu cho tt c cc nhim v b nh cao chy . Nhn chung, vic lp lch DRFbased l nhn thc ca c ngun lc c thi gian p ng thp nht v thng lng tng th cao nht.7.3 Simulations using Facebook Traces.Tip theo chng ta s dng du vt ng nhp t mt cm nt 2000 ti Facebook, cha d liu cho mt khong thi gian mt tun (thng 10 nm 2010). Cc d liu bao gm vic lm Hadoop MapReduce. Chng ti gi nh thi gian cng tc, s dng CPU v b nh tiu th l ging ht nh trong cc du vt ban u. Cc du vt c m phng trn mt cm nh hn 400 nt t c mc s dng cao hn, nh vy l cng bng tr thnh c lin quan. Mi nt trong cluster gm 12 khe cm, 16 li, v b nh 32 GB. Hnh 13 cho thy mt on ngn 300 th hai ph mu hnh dung nh th no CPU v b nh s dng tm kim cc khi lng cng vic tng t khi s dng DRF so vi lch cng bng Hadoop ca (slot). Nh th hin trong hnh, DRF cung cp s dng cao hn, v n c thHnh 14 cho thy vic gim trung bnh thi gian hon thnh cng vic cho DRF so vi lch cng bng Hadoop. Khi lng cng vic kh nng v vic lm nh,m khng c kinh nghim ci tin (v d, -3%). iu ny l bi v cc cng vic nh thng bao gm mt giai on thc thi duy nht, v thi gian hon thnh c chi phi bi cc nhim v di nht. Nh vy thi gian hon thnh l kh ci thin cho cc cng vic nh nh vy. Ngc li, thi gian hon thnh cc cng vic ln gim ca nhiu nh 66%. y l vic lm becausethese bao gm nhiu giai on, v do h c th c hng li t vic s dng cao hn t c bng DRF.8. Related Work.Chng ti cng xem xt cc cng vic lin quan trong khoa hc my tnh v kinh t.Trong khi nhiu bi bo v khoa hc my tnh tp trung vo s cng bng nhiu ngun, chng ta ch xem xt nhiu trng hp ca cc ngun ti nguyn thay th cho nhau cng, v d, CPU [6, 7, 35], v bng thng [10, 20, 21]. Khng ging nh cc phng php ny, chng ti tp trung vo vic phn b cc ngun lc ca cc loi khc nhau.

Quincy [18] l mt lch trnh pht trin trong bi cnh khun kh Dryad tnh ton cluster [17]. Quincy t c s cng bng bng cch m phng vn cng bng lch trnh nh mt vn dng min-chi ph. Quincy hin khng h tr cng bng nhiu ngun lc. trong thc t, nh cp trong phn tho lun ca bi bo [18, tr. 17], n xut hin kh khn kt hp cc yu cu a ti nguyn vo vic xy dng dng chy min-chi ph.Hadoop hin ang cung cp hai b k hoch chia s cng bng [1, 2, 34]. C hai b k hoch phn b ngun lc ti granularity khe, ni mt khe cm l mt phn c nh cacc ngun ti nguyn trn mt my. Kt qu l, cc b k hoch c th khng phi lc no cng ph hp vi vic phn b ti nguyn vi nhu cu ca cng vic, c bit l khi nhng nhu cu l khng ng nht rng ri. Nh chng ti trnh by trong phn 7, khng ph hp ny c th dn n vic s dng mt trong hai cm thp hoc hiu sut km do ti nguyn trn thu bao.Trong cc ti liu kinh t vi m, cc vn ca equityhas c nghin cu trong v ngoi khun kh ca l thuyt tr chi. Nhng cun sch ca tr [33] v Moulin [22]l hon ton dnh ring cho cc ch v cung cp gii thiu tt. Cc phng php a thch ca b phn cng bng trong kinh t vi m l CEEI [3, 33, 22], nh gii thiu bi Varian [30]. Do chng ti dnh s quan tm ng k cho n trong phn 5.2. Nhc im chnh ca com CEEI b DRF l n khng phi l chin lc chng. Kt qu l, ngi dng c th thao tc cc Scheduler bng cch ni di v nhu cu ca h.Nhiu ngi trong s cc chnh sch phn chia cng bng xut trong cc ti liu kinh t vi m da trn khi nim v tin ch v, do , tp trung vo cc thc o duy nht ca tin ch. Trong cc ti liu kinh t, cng bng max-min c bit n nh s sp t t t in [26, 25] (leximin) ca tin ch.Cu hi t ra l nhng g cc cng c ngi dng ang trong khung cnh multiresource, v lm th no so snh tin ch nh vy. Mt cch t nhin l xc nh tin ch nh s lng cc nhim v c giao cho mt ngi s dng. Nhng m hnh ha cc tin ch theo cch ny, cng vi leximin, vi phm nhiu tnh cht cng bng, chng ti xut. Nhn trong nh sng ny, DRF lm hai ng gp. u tin, n cho thy bng cch s dng phn vt tri nh l mt proxy cho tin ch, c cn bng bng cch s dng t hng leximin chun. Th hai, chng ti chng minh rng chng trnh ny l chin lc chng cho cc chc nng tin ch nh vy. Lu rng vic t hng leximin l mt phin bn t t in ca KalaiSmorodinsky (KS), gii php [19]. Nh vy, kt qu ca chng ti cho thy rng KS l chin lc chng cho cc tin ch nh vy.

Kt lun v bi hcChng ti gii thiu Dominant Resource Cng bng (DRF), mt m hnh chia s cng bng m khi qut cng bng max-min cho nhiu loi ti nguyn. DRF cho php lp lch cluster a vo ti khon cc nhu cu khng ng nht ca cc ng dng trung tm d liu, dn n c vic phn b cng bng hn cc ngun ti nguyn v s dng cao hn so vi cc gii php hin rng vic phn b ngun lt ging ht nhau (khe) cho tt c cc tasks.DRF tha mn mt s tnh cht mong mun. c bit, DRF l chin lc chng, ngi dng c incentivized bo co nhu cu ca h mt cch chnh xc. DRF cng incentivizes ngi dng chia s ti nguyn bng cch m bo rng ngi dng thc hin t nht cng trong mt cm chia s nh h s nh hn, cm ring bit. Lp lch khc m chng ti nghin cu, cng nh cc khi nim khc v s cng bng t cc ti liu kinh t vi m, khng p ng c tt c cc thuc tnh.Chng ti nh gi DRF bng cch thc hin n trong qun l ti nguyn Mesos, v cho thy rng n c th dn n hiu sut tng th tt hn so vi lp lch cng bng khe da trn l thng s dng hin nay.C nhiu hng khc nhau th v cho research.First trong tng lai, trong mi trng cluster vi nhim v ri rc, mt vn th v l gim thiu s phn mnh ti nguyn m khng nh hng s cng bng. Vn ny cng tng t nh bin-ng gi, nhng mt trong nhng ni phi ng gi nh nhiu mt hng (nhim v) c th chu p DRF. Mt hng th hai lin quan n vic xc nh s cng bng khi cc nhim v c nhng hn ch v tr, chng hn nh my preferences.Given cc xu hng hin ti ca my a li, mt hng nghin cu th v th ba l khm ph vic s dng ca DRF nh mt lch trnh h iu hnh. Cui cng, t gc kinh t vi m, mt hng t nhin l iu tra DRF l ch c chnh sch strategyproof c th cho cng bng nhiu ngun lc, nht nh c tnh mong mun khc hiu qu Pareto nh vy.

Ti liu tham kho