Miguel Jiménez, Prashanti Angara, Harshit Jain, Kirti Agarwal, Roshni Jain, Hausi Müller, Ulrike Stege, Joanna Ng
IBM Canada LabWeek - CASTLE 2017University of Victoria
Miguel Jiménez, Prashanti Angara, Harshit Jain, Kirti Agarwal, Roshni Jain, Hausi Müller, Ulrike Stege, Joanna Ng
IBM Canada LabWeek - CASTLE 2017University of Victoria
Cognitive IoT Recipe MavenDigital Expertise in the Kitchen
Cognitive IoT Recipe MavenDigital Expertise in the Kitchen
Picture from http://foodindustryexecutive.com/2016/04/the-internet-of-things-and-the-future-of-food/
Motivation● Food wastage is currently 40% in North America [1]● Obesity rate is increasing globally [2]● Limited context integration in existing applications
○ Nutrient information and dietary goals○ Effective use of expiry dates○ Knowledge of users’ taste, preferences, allergies and diseases
● Limited knowledge and time to cook healthy● Sparse and duplicate data across multiple devices and applications
[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3228640/[2] https://www.hsph.harvard.edu/obesity-prevention-source/obesity-trends/obesity-rates-worldwide/
2
Outline1. Overview of the IoT Recipe Maven2. Key stakeholders3. Maven components
a. CAPRecipes: context-aware personalized recipes recommenderb. Foodie Fooderson: a conversational agent for the smart kitchenc. Smidge: Smart Refrigerator and Groceryd. SmartGrocer: a profit-aware store path optimizer
4. Challenges5. Conclusion
3
Key Stakeholders
Grocerystores
FridgeMakers
Users
❖ Smidge❖ SmartGrocer❖ CAPRecipes❖ Foodie
4
Cognitive IoT Recipe Maven
Context Sphere
Web app FoodieMobile app
SmartGrocer CAPRecipes
SpoonacularGrocery Store
Smidge Kitchen data
Personal data
Heart monitor
diabetes monitor
Weighing scale
Fridge camera
Smartermats
Food data
5[1] https://spoonacular.com/food-api
CAPRecipes: Context-Aware Recipe Recommendation
6
Collaborative filtering
Content-based filtering
Recommender
Personalized Recipes
Personal health information: allergies and diseases
Cuisine and ingredient preferences
Preferences of family and friends
Browser search history and social interactions
Time of day, location (general context)
Expiry date, quantity and missing ingredients
Context
Recipe preferences and ratings
Ratings
+Reasoning
CAPRecipes: Hybrid interfaceMobile ApplicationWeb UI
7
Foodie: A Cognitive Kitchen Assistant
A cognitive conversational assistant connected to our recipe recommendation engine that:
○ Assists people in finding a suitable recipe○ Describes step-by-step cooking instructions○ Provides nutrition information○ Helps people in creating and tracking dietary goals
I’m hungry
Would you like to make Lemon Garlic Chicken?
I don’t like garlic
How about Grilled Cheese
Pizza?
Sounds good!
Cuisine
Ingredients 8
Sugar intake
Body Weight
Foodie, I’d like to reduce my intake of sugar
[...] how much fat is in thatrecipe?
Under the Hood● #Intents
○ Determine the purpose of arbitrary user input○ Example: “I’m feeling hungry” is classified under the intent #start_cooking
● @Entities○ Keyword identification○ Example: “I want to eat a french breakfast”
■ @cuisine = french
■ @mealType = breakfast
● Dialog○ Possible flows of a conversation via nodes○ Nodes are triggered by conditions
● Context○ Mechanism for passing information between the dialog and the application
Foodie: A Cognitive Kitchen Assistant (2)
9
Smidge: Smart Refrigerator and Grocery
10
Smarter Mats
FridgeCam
Grocery Store
● Update ingredient inventory through grocery receipts
● Personalize recommendations for coupons to increase savings
● FridgeCam: Analyze contents of the fridge
● Smarter Mats: Track weights for contents in the fridge
●
Cashier
Smart Grocer: Profit-Aware Store Path Optimization
Promote products by incentivizing customers:
● Guide users through the store to purchase a list of items
● Indicate additional items in or near the customer’s path that the store wants to promote or offer discounts on.
11
List itemPromoted itemOptimal PathSuggested Detour
Entry
Smart Grocer: Profit-Aware Store Path Optimization
12
Grocery List
User’s preferences
Context Sphere
Kitchen data
Personal data
Smidge
Smart Grocer
Optimal Path
Optimal Path + Suggested Detours
Optimal Path + Extra Detours
ChallengesDiscovering agents in a shared environment to accomplish tasks collaboratively.
Effective context management
Foodie
Building a richer conversational agent is difficult. Platforms are not sophisticated enough yet.
Accurate voice recognition services for people with accents and environments with noise
Smidge
Accurately identifying products inside the fridge
13
Key TakeawaysCognitive IOT Recipe Maven integrates food-related applications and uses context to collaboratively enhance the user experience.
It includes CAPRecipes, a context-aware personalized recipes recommender, Foodie: a conversational agent for the smart kitchen, Smidge: An IoT Enabled Fridge and SmartGrocer: a profit-aware store path optimizer
There is great potential for building systems which cross context barriers and enhance user experiences.
14
Thanks
15
Miguel Jiménez, [email protected]
Prashanti Angara, [email protected]
Hausi Müller, [email protected]
Joanna Ng, [email protected]
Top Related