Prediction Modeling for Personalization &

Recommender Systems

Bamshad MobasherDePaul University

Bamshad MobasherDePaul University

2

What Is Prediction?

i Prediction is similar to classification4 First, construct a model4 Second, use model to predict unknown value

i Prediction is different from classification4 Classification refers to predicting categorical class label (e.g., “yes”, “no”)4 Prediction models are used to predict values of a numeric target attribute

h They can be thought of as continuous-valued functions

i Major method for prediction is regression4 Linear and multiple regression4 Non-linear regression4 K-Nearest-Neighbor

i Most common application domains:4 Personalization & recommender systems, credit scoring, predict

customer loyalty, etc.

3

Personalization

i The Problem4 Dynamically serve customized content (books, movies, pages, products,

tags, etc.) to users based on their profiles, preferences, or expected interests

i Why we need it?4 Information spaces are becoming much more complex for user to navigate

(huge online repositories, social networks, mobile applications, blogs, ….)4 For businesses: need to grow customer loyalty / increase sales4 Industry Research: successful online retailers are generating as much as

35% of their business from recommendations

i Recommender Systems the most common type of personalization systems

4

Recommender Systems: Common Approaches

i Collaborative Filtering4 Give recommendations to a user based on preferences of “similar” users4 Preferences on items may be explicit or implicit4 Includes recommendation based on social / collaborative content

i Content-Based Filtering4 Give recommendations to a user based on items with “similar” content in the

user’s profile

i Hybrid Approaches

5

The Recommendation Task

i Basic formulation as a prediction problem

i Typically, the profile Pu contains preference scores by u on some other items, {i1, …, ik} different from it4 preference scores on i1, …, ik may have been obtained explicitly (e.g.,

movie ratings) or implicitly (e.g., time spent on a product page or a news article)

Given a profile Pu for a user u, and a target item it, predict the preference score of user u on item it

Given a profile Pu for a user u, and a target item it, predict the preference score of user u on item it

6

Example: Recommender Systems

i Content-based recommenders4 Predictions for unseen (target) items are computed based on their

similarity (in terms of content) to items in the user profile.

4 E.g., user profile Pu contains

recommend highly: and recommend “mildly”:

7

Content-Based Recommender Systems

8

Content-Based Recommenders :: more examples

i Music recommendationsi Play list generation

Example: Pandora

Content representation & item similarities

i Represent items as vectors over features4 Features may be items attributes, keywords, tags, etc.4 Often items are represented a keyword vectors based on textual descriptions

with TFxIDF or other weighting approachesh Has the advantage of being applicable to any type of item (images,

products, news stories, tweets) as long as a textual description is available or can be constructed

4 Items (and users) can then be compared using standard vector space similarity measures

Content-based recommendation

i Basic approach

4 Represent items as vectors over features4 User profiles are also represented as aggregate feature vectors

h Based on items in the user profile (e.g., items liked, purchased, viewed, clicked on, etc.)

4 Compute the similarity of an unseen item with the user profile based on the keyword overlap (e.g. using the Dice coefficient)

4 sim(bi, bj) =

4 Other similarity measures such as Cosine can also be used4 Recommend items most similar to the user profile

11

Collaborative Recommender Systemsi Collaborative filtering recommenders

4 Predictions for unseen (target) items are computed based the other users’ with similar interest scores on items in user u’s profileh i.e. users with similar tastes (aka “nearest neighbors”)h requires computing correlations between user u and other users according

to interest scores or ratingsh k-nearest-neighbor (knn) strategy

Star Wars Jurassic Park Terminator 2 Indep. Day Average PearsonSally 7 6 3 7 5.75 0.82Bob 7 4 4 6 5.25 0.96Chris 3 7 7 2 4.75 -0.87Lynn 4 4 6 2 4.00 -0.57

Karen 7 4 3 ? 4.67

K Pearson1 62 6.53 5

Can we predict Karen’s rating on the unseen item Independence Day?Can we predict Karen’s rating on the unseen item Independence Day?

12

Collaborative Recommender Systems

Many examples in real world applications

Don’t need a representation for items, but compare user profiles instead

13

Collaborative Filtering: Measuring Similarities

i Pearson Correlation4 weight by degree of correlation between user U and user J

4 1 means very similar, 0 means no correlation, -1 means dissimilar

4 Works well in case of user ratings (where there is at least a range of 1-5)4 Not always possible (in some situations we may only have implicit binary

values, e.g., whether a user did or did not select a document)4 Alternatively, a variety of distance or similarity measures can be used

Average rating of user Jon all items.2 2

( )( )

( ) ( )UJ

U U J Jr

U U J J

14

Collaborative Filtering: Making Predictions

i In practice a more sophisticated approach is used to generate the predictions based on the nearest neighbors

i To generate predictions for a target user a on an item i:

4 = mean rating for user a

4 u1, …, uk are the k-nearest-neighbors to a

4 ru,i = rating of user u on item I

4 sim(a,u) = Pearson correlation between a and u

i This is a weighted average of deviations from the neighbors’ mean ratings (and closer neighbors count more)

k

u

k

u uiuaia

uasim

uasimrrrp

1

1 ,,

),(

),()(

ar

15

Example: User-Based Collaborative Filtering

Star Wars Jurassic Park Terminator 2 Indep. Day Average PearsonSally 7 6 3 7 5.75 0.82Bob 7 4 4 6 5.25 0.96Chris 3 7 7 2 4.75 -0.87Lynn 4 4 6 2 4.00 -0.57

Karen 7 4 3 ? 4.67

K Pearson1 62 6.53 5

Star Wars Jurassic Park Terminator 2 Indep. Day Average PearsonSally 7 6 3 7 5.75 0.82Bob 7 4 4 6 5.25 0.96Chris 3 7 7 2 4.75 -0.87Lynn 4 4 6 2 4.00 -0.57

Karen 7 4 3 ? 4.67

K Pearson1 62 6.53 5

prediction

Correlation to KarenCorrelation to KarenPredictions for Karen on Indep. Day based on the K nearest neighbors

Predictions for Karen on Indep. Day based on the K nearest neighbors

i Build a content-based recommender for4 News stories (requires basic text processing and indexing of

documents)4 Blog posts, tweets4 Music (based on features such as genre, artist, etc.)

i Build a collaborative or social recommender4 Movies (using movie ratings), e.g., movielens.org4 Music, e.g., pandora.com, last.fm

h Recommend songs or albums based on collaborative ratings, tags, etc.h recommend whole playlists based on playlists from other users

4 Recommend users (other raters, friends, followers, etc.), based similar interests

16

Possible Interesting Project Ideas

Prediction Modeling for Personalization &

Recommender Systems

Bamshad MobasherDePaul University

Bamshad MobasherDePaul University