Statistical Challenges in Recommeder System for Web Applications

Abstract

In this talk, I will begin with an overview of statistical challenges that arise in recommender system problems for web applications like content optimization, online advertising. I will then describe some modeling solutions for a content optimization problem that arises in the context of the Yahoo! Front Page. In particular, I will discuss time series models to track item popularity, explore-exploit/sequential design schemes to enhance performance and matrix factorization models to personalize content to users. For some of the methods, I will present experimental results from an actual system at Yahoo!. I will also provide examples of other applications where the techniques are useful and end with discussion of some open problems in the area.

Speaker Biography

Dr. Deepak Agarwal is currently a principal research scientist at Yahoo! Research. Prior to joining Yahoo!, he was a member of the statistics department at AT&T Research. His current research interests are on scalable statistical models for recommender systems and online advertising. In particular, he is interested in hierarchical modeling, sequential designs and matrix factorization methods. His other research interests include mining massive graphs, statistical models for social network analysis, anomaly detection using a time series approach and spatial scan statistic for detecting hotspots.

Deepak has been a co-author on three best paper awards (Joint Statistical Meetings, 2001; Siam Data Mining 2004; KDD 2007) in the past, he is currently associate editor for Journal of American Statistical Association and regularly serves on program committees of data mining and machine conferences like KDD, WWW, SDM, ICDM, WSDM, ICML and NIPS.