Recommender systems: When you have no idea

Yesterday, I went to dinner with a group of my friends. I was an invitee, so I had no choice but accepting their invitation and go to the restaurant they had already reserved. I was completely new to that restaurant and had no idea about what were the best options for me. I went through the menu and found few familiar dishes I have had before from other places. We had 10 people sitting around the table and 7 out of them had already decided what to get for the dinner as they were members of the restaurant. 3 of us were completely lost :). I was switching between dishes as I had no idea. What did I do?

 

I decided to ask for help from my friends to take the decision on dishes. I selected 3 friends who have similar taste to me. We know each other from the childhood. I asked the best options from those 3 folks. 2 of them recommend me fried noodles with chicken and other one suggested me pasta instead of noodles. Thanks to my friends, I could reduce the list to 2 dishes. Then, I decided to take the final decision considering my past experience. I like noodles more than pasta and I took fried noodles with chicken dish. It was a great choice and their recommendation worked for me. I would like to recommend you that dish if you go to that restaurant one day. Just let me know :)

Sometimes we face these kinds of situations where we need suggestions, advice or recommendations from experts on specific subjects. It may be a movie to watch, book to read, dish from a restaurant, whom to vote, doctor to visit, song to listen, hotel to stay and much more. By nature, we would like to ask it from our friend or a similar person who has expertise in the topic or with previous experience. Also, we consider our own experience or preferences in such situations. In real life, sometimes we have to rely on other's recommendations or our previous experiences/choices or take the decision considering both. This is the foundation of Recommender Systems.

Recommender systems are tools and technologies which recommend items/concepts/services/actions or solutions to users. Today, Recommender Systems are heavily and widely used in social networks, e-commerce websites, recommendation software which recommends movies, books, songs. Let's look at the technical concepts and methods under the hood.

We can use several different methods to generate recommendations. We can recommend the most popular items to users. If we look at our restaurant example, the restaurant can recommend most popular dishes to new(or existing) customers. It can collect data from purchasing history and identify a list of popular items which can be recommended later. This method is called Popularity Based Recommendations. Do you see any problem in this method? While this is very easy to implement, we are missing personalization in generated recommendations. Personalization is the process of tailoring items/solutions to individual users' characteristics or preferences. Systems based on popularity based recommendations provide a fixed set of items for any user in a given time interval. Can't we think the problem as a classification problem? Yes, we can. We can consider an item and predict whether the user is going to like it or not. Here we look at the attributes of the item and take decisions based on those attributes. We can consider user attributes too. Identifying recommendation problem as a classification problem is called Classification Based Recommendations method. Normally, recommendation problems deal with a very large number of users and items. So, we are facing a problem here. If we are going forward with classification approach, we have to work with a large number of features(from user's perspective, items are features and vice versa). Can't we find a solution for the problem of generating recommendations based on our real life experiences? Can't we use our neighbor's experiences or our own experiences to solve this? Yes, we can. Neighborhood Based Recommendations method addresses the problem using our neighborhood. Content Based Recommendations method uses user's past experiences to generate recommendations. Let's discuss these two methods it in detail.

There are two commonly used methods in RS.

1. Collaborative filtering
2. Content-based filtering

 

Collaborative Filtering(CF)

Collaborative Filtering(CF) is the most popular and widely adapted method among two. CF relies on past ratings of the active user and other users. It generates recommendations based on the concept of 'similar users prefer similar items'. So, based on similarity of users, CF generates recommendations for the active user.

If we peep into our restaurant experience through Collaborative Filtering window, First, I selected people who have similar taste to me. Then, they suggested me few dishes according to their ratings and I accepted their choices considering the similarity between us.

Content-based filtering

CBF generates recommendations based on user's past behavior or preferences. This method retrieves the list of items the user has used/rated previously and try to find new items which have similar features/attributes with past items. Here, we are walking through user profile considering attributes of the user to generate recommendations.

This can be related to my past experience of taking dinner from a new restaurant. At the moment of getting the decision of dishes, I considered my preferences and past experience. I was looking for similar dishes to my favorite choices.

*Hybrid method

Another blooming research area is using a hybrid method combining Collaborative filtering and Content-based filtering methods. The goal is to mitigate weaknesses of individual methods while taking advantages of strengths of both.

When I get my final decision on the dish, I used concepts of the hybrid method. I got suggestions from my friends and then used my preferences/past experience to select one.

 

According to what we have discussed so far, it is obvious that the science of Recommender Systems is developed imitating human rational decision-making process.

Let me wind up our recommendation story with a true story of Touching the Void and Into the Air books.

In 1988, a British mountain climber Joe Simpson wrote a book called Touching the Void, a harrowing account of near-death in the Peruvian Andes. It got good reviews, only a modest success, it was soon forgotten. Then, a decade later, a strange thing happened. Jon Krakauer wrote Into Thin Air, another book about a mountain-climbing tragedy, which become a publishing sensation. Suddenly, Touching the Void started to sell again. Amazon’s recommendation system noticed a few people who bought both books, and started recommending Touching the Void to people who bought, or were considering, Into Thin Air. Had there been no on-line bookseller, Touching the Void might never have been seen by potential buyers, but in the on-line world, Touching the Void eventually became very popular in its own right, in fact, more so than Into Thin Air.

Let's meet again.