Fuzzy inference system and learning 08 july 2014

FUZZY INFERENCE SYSTEM AND LEARNING08 JULY 2014

DataSense Digicosme | CORNEZ Laurence

PLAN

| PAGE 2DataSense Digicosme | CORNEZ Laurence

I. Brief introduction on Fuzzy Logic and Fuzzy inference system (FIS)

II. Real context and database III. Fuzzy rules and Sugeno’s classifier IV. Implementation in three stepsVI. Visualization of FIS obtainedVII. Perspectives in terms of intelligibility and

performance


Fuzzy logic and applications

• 1965, Zadeh proposes fuzzy concept : one object can be simultaneously in two different classes

• This allows the imperfections (natural language), imprecisions and uncertainties (data)

• Applications (since 1974): washing machine, ABS, autofocus camera…)

| PAGE 3


Fuzzy expert systems

• Goal: three parts to reproduce the cognitive reasoning of an expert:Rules base:

- expression of the knowledge of the expert through « If-then » inference rules.

- directly expressed by expert or learnt via databasesInputs Inference engin able to integrate these rules and these inputs to produce specific outputs.

| PAGE 4

Inference engin

Rules

inputs outputs

Fuzzy inference system: example

• Rules base (Jang97)IF temperature=low THEN cooling valve=half open

IF temperature=medium THEN cooling valve=almost open

• input moteur d’inférence

Rules

18°

low1

0

half open1

0

1

0

medium1

0

almost open

T°

T°

d%

d%

0,2

0,5

1

0 d%

Implementation

Implementation

DataSense Digicosme | CORNEZ Laurence | PAGE 5

70%


WORK POSITION / DEFINITION

| PAGE 6


Seismicity map (+/- France)

| PAGE 7

Earthquakes

Marine explosions

Quarry blasts

Rock bursts

How to class a new

event automatically with

good interpretability for

the expert ?


Database stutied

French seismic metropolitan data from 1997 and 2003

Inputs (high level features):Hour : circular variable [0;24]Latitude : quantitative variable [42;51]Longitude : quantitative variable [-5;9]Magnitude : quantitative variable [0.7;6.0]Date : qualitative variable with 3 modalities

{Working day, Saturday, Sunday and bank holiday}

Classification output (3 possible classes):Earthquakes (9349 events)Quarry blasts (3485 events)Rock bursts (1075 events)

| PAGE 8


Model proposed

| PAGE 9

Aggregation of rules (Sugeno order 0)

- If magnitude is middle and event is nocturnal then event is earthquake - If magnitude is high then event is surely earthquake

How to generate these rules automatically ?

NbRules

kkk

NbRules

kkkk

x

zx

xZ

1

1

)(

)(

)(

An example of the input space

Weight of the rule k

Membership degree of x to the rule k

Issue of the rule k (unit vector)

Sugeno’s classifier (normalized) defined as:


MODEL IMPLEMENTATION

| PAGE 10


Model implementation: first step (1/2)

| PAGE 11

Soft Clustering = modelling class density by gaussian mixture

Mountain clustering (Chiu 94)

The algorithm learns :• Gaussian number • Location of gaussian

centers

magnitude

hour


Model implementation: first step (2/2)

| PAGE 12

Results:Good classification rate with « winner takes all » method5-fold cross-validation databases

Method Learning rate (%) Test rate (%) Cluster number

5 quantitatives(quali. va. misused)

85.40 +/- 0.87 84.84 +/- 1.47

32 ; 36 ; 35 ; 38 ; 37

4 quantitatives 85.66 +/- 0.85 84.65 +/- 1.76

25 ; 26 ; 28 ; 28 ; 29

What about the qualitative variable ?

Similar good

classification

rates

Less clusters


Model implementation: second step (1/2)

| PAGE 13

Probability estimations ofeach modality for each cluster

cXki

cXki

QLi

k

i

i

A

AmXInd

Xmp,

,)(

)(

)()()(1

mXIndXmpXp QLi

NbModes

mk

QLik

2

1 ,

,,

1,

,

)(

2

1exp

2

1 NbInputs

l lk

lkQTli

NbInputs

llk

NbInputski

CXdA

With:

NbRules

kNbRules

k

QLikkik

QLikkikk

i

XpA

XpAzXZ

1

1,

,

)(

)()(

Associated Sugeno’s

classifier


Model implementation: second step (2/2)

| PAGE 14

Method Learning rate (%) Test rate (%)

4 quanti. + 1 quali. 86.99 +/- 0.48 86.02 +/- 0.91

Results after step II:

Good classification

rates not significantly

improved

Well classified point

Ill classified point

One cluster

Semi optimal(cluster juxtapositions)

Not optimal (absence of clusters)


Model implementation: third step (1/2)

| PAGE 15

Improvement of parameters with EM « Expectation-Maximization » (Jordan et Jacobs 1993)

Input space is virtually augmented by adding a hidden variable, the cluster of interest

EM garantees improvement after each step

Computation of new

parameters:

={weights, centers and

standard deviations }

N

iN

ii

inewlkli

newlk

N

iN

ii

ili

newlk

N

ii

newk

Xkp

XkpCXd

Xkp

XkpXC

XkpN

1

1

2,,,

1

1

,,

1

),(

),(),(

),(

),(

),(1


Model implementation: third step (2/2)

| PAGE 16

Results after step III: 50 iterations for EM The same 5-fold cross-validation database

Method Learning rate (%) Test rate (%)

4 quant.+ 1 qual. 93.67+/- 0.60 93.12 +/- 1.66

Improvement for good

classification rate

significant improvement for cluster locations


Visualization

| PAGE 17

X

sum

X

X

X

Estim. ProbaGaussians Rule outputWeight

Product

One

rule

One example Class of the

exampleClasse decided

EQ[ 92.70% 0.00% 7.32%]


PERSPECTIVES IN TERMS IN INTELLIGIBILITY AND PERFORMANCE

| PAGE 18


FIS 95,19%

DT 94,88%

Fuzzy DT 95,19%

Comparison with previous works

| PAGE 19

1998 : S. Muller• fuzzy Controller codage • MLP• 92.5% well classified

1999 : F. Gravot• FIS

- Mixture of gausians- Gradient-based descent

• 90,5% well classified

2005 : R. Quach et D. Mercier• fuzzy controller codage• MLP : 95,9% well classified• SVM : 96,5% well classified

Intelligibility

Performance

cNF+RN92,5%

cNF+MLP/SVM~96%

FIS90,5%2006 : L. Cornez

• DT• 94,88% well classified• 95,19% well classified

2007 : L. Cornez• FIS 3 steps• 95,19% well classified

Objective


How improve intelligibility ?

| PAGE 2020

According to fold cross validation database, the coverage is

different


Improve intelligibility and stability

| PAGE 21

More the model is stable and more the model fit with cognitive representation more the expert can accept it

Generative Gaussian Graph (M. Aupetit) to identify complex clusters

CEA Tech

Département Métrologie,

Instrumentation et Information

Laboratoire d’Analyse de Données et

Intelligence des Systèmes

Commissariat à l’énergie atomique et aux énergies alternatives

Institut Carnot CEA LIST

Centre de Saclay | 91191 Gif-sur-Yvette Cedex

T. +33 (0)1 69 08 18 00

Etablissement public à caractère industriel et commercial | RCS Paris B 775 685 019

| PAGE 22


THANKS ! QUESTIONS ?


Best fuzzy decision tree

| PAGE 23

Fuzzy inference system and learning 08 july 2014

Documents

Transcript of Fuzzy inference system and learning 08 july 2014