复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic...
Transcript of 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic...
![Page 1: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/1.jpg)
复旦大学大数据学院School of Data Science, Fudan University
DATA130008 Introduction to Artificial Intelligence
Constraint Satisfaction Problems魏忠钰
March 27th,2019
![Page 2: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/2.jpg)
Constraint Satisfaction Problems
![Page 3: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/3.jpg)
Constraint Satisfaction Problems
§ Planning:sequencesofactions§ Thepathtothegoalistheimportantthing
§ Pathshavevariouscosts(e.g.depths)
§ Heuristicsgiveproblem-specificguidance
§ Identification:assignmentstovariables§ Thegoalitselfisimportant,notthepath
§ E.g.Nqueens,Sudoku
§ Allpathsatthesamedepth(forsomeformulations)
§ ConstraintSatisfactionProblems(CSPs)arespecializedforidentificationproblems
![Page 4: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/4.jpg)
Constraint Satisfaction Problem
§ Standardsearchproblems:§ Stateisa“blackbox”:arbitrarydatastructure
§ Goaltestcanbeanyfunctionoverstates
§ Constraintsatisfactionproblems(CSPs):§ Stateisrepresentedasafeaturevector
§ Goaltestisasetofconstraints toidentifytheallowablefeaturevector
§ 2
![Page 5: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/5.jpg)
Feature Vector
§ Wehave§ Asetofkvariables (orfeatures)§ Eachvariablehasadomain ofdifferentvalues.§ Astateisspecifiedbyanassignmentofavalueforeachvariable.§ height={short,average,tall},§ weight={light,average,heavy}
§ Apartialstateisspecifiedbyanassignmentofavaluetosomeofthevariables.
§ Acompletestateisapossiblesolutiontotheproblem.
![Page 6: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/6.jpg)
CSP Examples
![Page 7: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/7.jpg)
Example: Map Coloring
§ Variables:
§ Domains:
§ Constraints:adjacentregionsmusthavedifferentcolors
§ Solutionsareassignmentssatisfyingallconstraints,e.g.:
Implicit:
Explicit:
![Page 8: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/8.jpg)
Example: N-Queens
§ Formulation1:§ Variables:§ Domains:§ Constraints
![Page 9: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/9.jpg)
Example: N-Queens
§ Formulation2:§ Variables:
§ Domains:
§ Constraints:
Implicit:
Explicit:
![Page 10: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/10.jpg)
Constraint Graphs
![Page 11: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/11.jpg)
Constraint Graphs
§ BinaryCSP:eachconstraintrelates(atmost)twovariables§ Binaryconstraintgraph:nodesarevariables,arcsshow
constraints
![Page 12: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/12.jpg)
Example: Cryptarithmetic
§ Variables:
§ Domains:
§ Constraints:
![Page 13: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/13.jpg)
Example: Sudoku
§ Variables:§ Each(open)square
§ Domains:§ {1,2,…,9}
§ Constraints:
9-wayalldiff foreachrow
9-wayalldiff foreachcolumn
9-wayalldiff foreachregion
(orcanhaveabunchofpairwiseinequalityconstraints)
![Page 14: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/14.jpg)
Transform N-nary CSPs to Binary constraints one
§ ShowhowasingleternaryconstraintsuchasA+B=Ccanbeturnedinto3binaryconstraintsbyusingauxiliaryvariables.Youmayassumefinite domains.Nextshowhowconstraintswithmorethan3variablescanbetreatedsimilarly.
§ Step1:IntroduceanewvariableZ.§ Domainisthesetofallpossible3-tuplessatisfyingA+B=C.{(a1,b1,c1),(a2,b2,c2),…}
§ Step2:CreatenewconstraintsbetweenZandthreeoriginalvariablesrespectively(thepositioninthetupleislinkedtoanoriginalvariable).§ < 𝑍, 𝐴 >∈ {(((𝑎1, 𝑏1, 𝑐1)𝑎1), …}§ < 𝑍, 𝐵 >∈ {(((𝑎1, 𝑏1, 𝑐1), 𝑏1), …}§ < 𝑍, 𝐶 >∈ {((𝑎1, 𝑏1, 𝑐1))𝑐1), …}
![Page 15: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/15.jpg)
Varieties of CSPs and Constraints
![Page 16: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/16.jpg)
Varieties of CSPs
§ DiscreteVariables§ Finitedomains
§ SizedmeansO(dn) completeassignments§ Infinitedomains(integers,strings,etc.)
§ E.g.,jobscheduling,variablesarestart/endtimesforeachjob
§ Continuousvariables§ A + B = C(A,B,Carerealnumbers)
![Page 17: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/17.jpg)
Varieties of Constraints
§ VarietiesofConstraints§ Unaryconstraintsinvolveasinglevariable:
§ Binaryconstraintsinvolvepairsofvariables,e.g.:
§ Higher-orderconstraintsinvolve3ormorevariables: e.g.,cryptarithmetic columnconstraints
§ Global Constraints, e.g.: AllDiff()
§ Preferences(softconstraints):§ i.e.appleisbetterthanbanana§ Oftenrepresentablebyacostforeachvariableassignment
![Page 18: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/18.jpg)
Real-World CSPs
§ Assignmentproblems:e.g.,whoteacheswhatclass§ Timetablingproblems:e.g.,whichclassisofferedwhenandwhere?§ Hardwareconfiguration§ Transportationscheduling§ Factoryscheduling§ Circuitlayout§ Faultdiagnosis§ …lotsmore!
![Page 19: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/19.jpg)
Solving CSPs
![Page 20: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/20.jpg)
Standard Search Formulation
§ StandardsearchformulationofCSPs
§ Statesdefinedbythevaluesassignedsofar(partialassignments)§ Initialstate:theemptyassignment,{}§ Successorfunction:assignavaluetoanunassignedvariable§ Goaltest:thecurrentassignmentiscompleteandsatisfiesallconstraints
§We’llstartwiththestraightforward,naïveapproach,thenimproveit
![Page 21: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/21.jpg)
Search Methods
§WhatwouldBFSdo?
§WhatwouldDFSdo?
§Whatproblemsdoesnaïvesearchhave?§ Ordering ofvariables.§ Choicesofvalues.§ Theutilizationofconstraints.
![Page 22: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/22.jpg)
Backtracking Search
![Page 23: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/23.jpg)
Backtracking Search
§ BacktrackingsearchisthebasicuninformedalgorithmforsolvingCSPs
§ Idea1:Onevariableatatime in fixed ordering§ Variableassignmentsarecommutative,sofixordering§ I.e.,[WA=redthenNT=green]sameas[NT=greenthenWA=red]
§ Idea2:Checkconstraintsasyougo§ I.e.consideronlyvalueswhichdonotconflictpreviousassignments§ Mighthavetodosomecomputationtochecktheconstraints§ “Incrementalgoaltest”
§ Depth-firstsearchwiththesetwoimprovements§ iscalledbacktrackingsearch
![Page 24: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/24.jpg)
Backtracking Example
![Page 25: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/25.jpg)
Backtracking Search
• Backtracking=DFS+variable-ordering+fail-on-violation• Whatarethechoicepoints?
![Page 26: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/26.jpg)
Improving Backtracking
§Ordering:§ Whichvariableshouldbeassignednext?§ Inwhatordershoulditsvaluesbetried?
§ Filtering:Canwedetectinevitablefailureearly?
§ Structure:Canweexploittheproblemstructure?
![Page 27: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/27.jpg)
Ordering
![Page 28: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/28.jpg)
Ordering: Minimum Remaining Values
§ VariableOrdering:Minimumremainingvalues(MRV):§ Choosethevariablewiththefewestlegalleftvaluesinitsdomain
§Whyminratherthanmax?§ Alsocalled“mostconstrainedvariable”§ “Fail-fast”ordering
![Page 29: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/29.jpg)
Ordering: Least Constraining Value
§ ValueOrdering:LeastConstrainingValue§ Givenachoiceofvariable,choosetheleastconstrainingvalue
§ I.e.,theonethatrulesoutthefewestvaluesintheremainingvariables
§ Notethatitmaytakesomecomputationtodeterminethis!(E.g.,rerunningfiltering)
§Whyleastratherthanmost?
![Page 30: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/30.jpg)
Filtering: Forward Checking
§ Forwardchecking:Crossoffvaluesthatviolateaconstraintwhenaddedtotheexistingassignment
WASANT Q
NSWV
![Page 31: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/31.jpg)
Filtering: Constraint Propagation
§ Forwardcheckingpropagatesinformationfromassignedtounassignedvariables,butdoesn'tprovideearlydetectionforallfailures:
§ NTandSAcannotbothbeblue!§ Constraintpropagation:reasonfromconstrainttoconstraint
WA SA
NT Q
NSW
V
![Page 32: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/32.jpg)
Consistency of A Single Arc
§ AnarcX® Yisconsistent iff foreveryxinthetailthereissomeyintheheadwhichcouldbeassignedwithoutviolatingaconstraint
§ Forwardchecking:Enforcingconsistencyofarcspointingtoeachnewassignment
Deletefromthetail!
WA SA
NT Q
NSW
V
![Page 33: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/33.jpg)
Arc Consistency of an Entire CSP
§ Asimpleformofpropagationmakessureallarcsareconsistent:
§ Important:IfXlosesavalue,neighborsofXneedtoberechecked!§ Arcconsistencydetectsfailureearlierthanforwardchecking§ Canberunasapreprocessororaftereachassignment§ What’sthedownsideofenforcingarcconsistency?
Remember:Deletefromthetail!
WA SANT Q
NSW
V
![Page 34: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/34.jpg)
Enforcing Arc Consistency in a CSP
• Runtime:O(n2d3),canbereducedtoO(n2d2)• How?• Checkexercise6.13formoreinformation.
![Page 35: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/35.jpg)
Limitations of Arc Consistency
§ Afterenforcingarcconsistency:§ Canhaveonesolutionleft§ Canhavemultiplesolutionsleft§ Canhavenosolutionsleft(andnotknowit)
§ Arcconsistencystillrunsinsideabacktrackingsearch!
Whatwentwronghere?
![Page 36: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/36.jpg)
K-Consistency
![Page 37: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/37.jpg)
K-Consistency
§ Increasingdegreesofconsistency
§ 1-Consistency(NodeConsistency):Eachsinglenode’sdomainhasavaluewhichmeetsthatnode’sunaryconstraints
§ 2-Consistency(ArcConsistency):Foreachpairofnodes,anyconsistentassignmenttoonecanbeextendedtotheother
§ K-Consistency:Foreachknodes,anyconsistentassignmenttok-1canbeextendedtothekth node.
§ Higherkmoreexpensivetocompute
![Page 38: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/38.jpg)
Strong K-Consistency
§ Strongk-consistency:alsok-1,k-2,…1consistent
§ Claim:strongn-consistencymeanswecansolvewithoutbacktracking!
§ Why?1. Chooseanyassignmenttoanyvariable2. Chooseanewvariable3. By2-consistency,thereisachoiceconsistentwiththefirst4. Chooseanewvariable5. By3-consistency,thereisachoiceconsistentwiththefirst26. …
§ Lotsofmiddlegroundbetweenarcconsistencyandn-consistency!(e.g.k=3,calledpathconsistency)
§ O(n2d) forasolutiontoStrongK-Consistencyproblem.
![Page 39: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/39.jpg)
Structure
![Page 40: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/40.jpg)
Problem Structure
§ Extremecase:independentsub-problems§ Example:Tasmaniaandmainlanddonotinteract
§ Independentsub-problemsareidentifiableasconnectedcomponentsofconstraintgraph
§ Supposeagraphofnvariablescanbebrokenintosub-problemsofonlycvariables:§ Worst-casesolutioncostisO((n/c)(dc)),linearinn§ E.g.,n=80,d=2,c=20§ 280 =4billionyearsat10millionnodes/sec§ (4)(220)=0.4secondsat10millionnodes/sec
![Page 41: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/41.jpg)
Tree-Structured CSPs
§ Theorem:iftheconstraintgraphhasnoloops,theCSPcanbesolvedinO(nd2)time§ ComparetogeneralCSPs,whereworst-casetimeisO(dn)
![Page 42: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/42.jpg)
Tree-Structured CSPs
§ Algorithmfortree-structuredCSPs:
§ Order:Choosearootvariable,ordervariablessothatparentsprecedechildren
§ Checkthearcconsistencyfromchildtoparent§ Assignvaluefromroottoleaves
![Page 43: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/43.jpg)
Tree-Structured CSPs
§ Algorithmfortree-structuredCSPs:
§ Runtime:O(nd2)(why?)
![Page 44: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/44.jpg)
Tree-Structured CSPs
§ Claim1:Afterbackwardpass,allroot-to-leafarcsareconsistentØEachX®YwasmadeconsistentatonepointandY’sdomaincouldnothavebeenreducedthereafter(becauseY’schildrenwereprocessedbeforeY)
§ Claim2:Ifroot-to-leafarcsareconsistent,forwardassignmentwillnotbacktrack
ØGiven one value of a parent node, there is always a valid value in thechild node to make the assignment consistent.
§ Whydoesn’tthisalgorithmworkwithcyclesintheconstraintgraph?
![Page 45: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/45.jpg)
Improving Structure
![Page 46: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/46.jpg)
Nearly Tree-Structured CSPs
§ Conditioning:instantiateavariable,pruneitsneighbors'domains§ CycleCutset conditioning:instantiate(inallways)asetofvariablessuchthattheremainingconstraintgraphisatree
§ CycleCutset sizecgivesruntimeO((dc)(n-c)d2)
SA
![Page 47: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/47.jpg)
Cutset Conditioning
SA
SA SA SA
Instantiatethecutset(allpossibleways)
ComputeresidualCSPforeachassignment
SolvetheresidualCSPs(treestructured)
Chooseacutset
![Page 48: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/48.jpg)
Cutset Excercise
§ Findthesmallestcyclecutset forthegraphbelow.
![Page 49: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/49.jpg)
Tree Decomposition
§ Idea:createatree-structuredgraphofmega-variables§ Eachmega-variableencodespartoftheoriginalCSP§ Sub-problemsoverlaptoensureconsistentsolutions
M3
{(NT=r,SA=g,Q=b),(NT=b,SA=g,Q=r),…}
Agree:(M1,M2)Î{((WA=g,SA=g,NT=g),(NT=g,SA=g,Q=g)),…}
NSW
SA
¹Q
¹ ¹
Agreeonsharedvars
M1 M2
{(WA=r,SA=g,NT=b),(WA=b,SA=r,NT=g),…}
Agreeonsharedvars
NT
SA
¹WA
¹ ¹
Q
SA
¹NT
¹ ¹
Agreeonsharedvars
M4
V
SA
¹NSW
¹ ¹
![Page 50: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/50.jpg)
Iterative Improvement
![Page 51: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/51.jpg)
Iterative Algorithms for CSPs
§ Localsearchmethodstypicallyworkwith“complete”states,i.e.,allvariablesassigned
§ ToapplytoCSPs:§ Takeanassignmentwithunsatisfiedconstraints§ Operators reassignvariablevalues
§ Algorithm:,§ Startfromacompleteassignment(whilenotasolution)§ Variableselection:randomlyselectanyconflictedvariable§ Valueselection:min-conflictsheuristic:
§ Chooseavaluethatviolatesthefewestconstraints§ I.e.,hillclimbwithh(n)=totalnumberofviolatedconstraints
![Page 52: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/52.jpg)
Example: 4-Queens
• States:4queensin4columns(44 =256states)• Operators:movequeenincolumn• Goaltest:noattacks• Evaluation:c(n)=numberofattacks
![Page 53: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/53.jpg)
Min-Conflicts Algorithm
§ Justlikeahillclimbingapproach
![Page 54: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/54.jpg)
Summary: CSPs
§ CSPsareaspecialkindofsearchproblem:§ Statesarepartialassignments§ Goaltestdefinedbyconstraints
§ Basicsolution:backtrackingsearch
§ Speed-ups:§ Ordering§ Filtering§ Structure
§ Iterativemin-conflictsisofteneffectiveinpractice§~50 formostoftheN-queenproblems
![Page 55: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/55.jpg)
Local Search
§ Systematicsearches:keeppathsin memory,andrememberalternativessosearchcanbacktrack.Solutionisapathtoagoal.
§ Pathmaybeirrelevant,ifonlythefinalconfigurationisneeded(8-queens,ICdesign,networkoptimization,…)
![Page 56: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/56.jpg)
Local Search versus Systematic Search
§ SystematicSearch§ BFS,DFS,IDS,Best-First,A*§ Keepssomehistoryofvisitednodes§ Alwayscompleteforfinitesearchspaces,some
versionscompleteforinfinitespaces§ Goodforbuildingupsolutionsincrementally
§ State=partialsolution§ Action=extendpartialsolution
![Page 57: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/57.jpg)
Local Search versus Systematic Search
§ LocalSearch§ Gradientdescent,Greedylocalsearch,Simulated
Annealing,GeneticAlgorithms§ Doesnotkeephistoryofvisitednodes§ Notcomplete.Maybeabletoarguewill
terminatewith“highprobability”§ Goodfor“fixingup”candidatesolutions
§ State=completecandidatesolutionthatmaynotsatisfyallconstraints
§ Action=makeasmallchangeinthecandidatesolution
![Page 58: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/58.jpg)
Local search
§ HillClimbing§ Simulatedannealing§ Localbeamsearch§ GeneticAlgorithm§ Gradientdecentforcontinuesfunction
![Page 59: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/59.jpg)
Hill-climbing search
§ Generatenearbysuccessorstatestothecurrentstate§ Pickthebestandreplacethecurrentstatewiththatone.
![Page 60: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/60.jpg)
Hill-climbing search
§ Localmaximum:apeakthatislowerthanthehighestpeak,soasuboptimalsolutionisreturned
§ Plateau:theevaluationfunctionisflat,resultinginarandomwalk
§ Ridge:resultinasequenceoflocalmaximathatisverydifficultforgreedyalgorithmstonavigate.
Local maximum Plateau Ridge
![Page 61: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/61.jpg)
States Where Greedy Search Must Succeed
objectivefunctio
n
Target
![Page 62: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/62.jpg)
States Where Greedy Search Might Succeed
objectivefunctio
n
Target
![Page 63: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/63.jpg)
States Where Greedy Search Will NOT Succeed
objectivefunctio
n
Target
![Page 64: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/64.jpg)
Greedy Search Landscape
objectivefunctio
nob
jectivefunctio
n
LocalMinimum
Plateau
Target
![Page 65: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/65.jpg)
Local search
§ HillClimbing§ Simulatedannealing§ Localbeamsearch§ GeneticAlgorithm§ Gradientdecentforcontinuesfunction
![Page 66: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/66.jpg)
Simulated Annealing
![Page 67: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/67.jpg)
Simulated Annealing
§ Combinationofrandomwalkandhillclimbing.§ Generatearandomnewneighborfromcurrentstate.§ Ifit’sbettertakeit.§ Ifit’sworsethentakeitwithsomeprobabilityproportional
tothetemperatureandthedeltabetweenthenewandoldstates.
![Page 68: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/68.jpg)
Simulated Annealing
§ Thealgorithmwandersaroundduringtheearlypartsofthesearch,hopefullytowardagoodgeneralregionofthestatespace
§ Towardtheend,thealgorithmdoesamorefocusedsearch,makingfewbadmoves
![Page 69: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/69.jpg)
Local search
§ HillClimbing§ Simulatedannealing§ Localbeamsearch§ GeneticAlgorithm§ Gradientdecentforcontinuesfunction
![Page 70: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/70.jpg)
Local Beam Search
§ Keeptrackofkstatesratherthanjustone,asinhillclimbing
§ Beginswithkrandomlygeneratedstates§ Ateachstep,allsuccessorsofallkstatesaregenerated§ Ifanyoneisagoal,Done!§ Otherwise,selectsbestksuccessorsfromthecomplete
list,andrepeats
![Page 71: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/71.jpg)
Local Search in Continuous Spaces
initial state vector( ) quantity to optimized
step sizeuntil Goal_Test( ) do
( )
Sf S
SfS S SS
a
a
==
=
¶= -
¶
negativesteptominimizefpositivesteptomaximizef
![Page 72: 复旦大学大数据学院 Constraint Satisfaction Problems · Local Search versus Systematic Search § Local Search § Gradient descent, Greedy local search, Simulated Annealing,](https://reader034.fdocument.pub/reader034/viewer/2022042123/5e9e0bfae4284a7f8c229cbd/html5/thumbnails/72.jpg)
Wrap-up
§ Ifyouareinterested,refertochapter4.1– 4.2