RubyConf Session 3 - Recommendation Engines

I’m going to be honest here… my notes from session 3 are a little light for two reasons. First, I know Evan and he’s amazing and I know that if I had additional questions, I could always ask him after the conference and second because when you’re new, concentrating on so many high level talks in a day can be a little tiring. But still, I learned some great stuff! The nice thing about the presentation, as well, was that I had actually had a fair amount of exposure to the concepts he was speaking about through both projects I’ve built and through my shadowing opportunities.

Evan was talking about recommendation engines. His example was that he was building a social network for soccer. The client he was working with wanted to have a “feed” function a la facebook where people could see what was going on. Specifically, they wanted the users to see what was popular and relevant in real time using tags. Each post has polymorphic tags associations. When looked at a recommendation engine, he mentioned that the engines are based on approximation and are based in statistical methods. The trick was to quantify posts based on popularity and relevancy.

He then spoke about Redis and how Redis stores a key/value pair in memory essentially making it a glorified hash in the data process. One interesting thing about this is that the value doesn’t have to be a string. It can be a data structure (like a hash, list, set, or sorted set) which is pretty interesting and means you can have more flexibility with the data. Finally in Redis there are few persistence options but the ones that exist are adequate.

He then talked about using ActiveRecord::observers which pushed events to resque. Finally, in order to actually determine importance weight, he used different calculators. One was trendingness. A second was user interest. And the third was the post score calculator. The methods themselves were really interesting to look at, think about, and dissect how they interacted with one another, but we’ll need to wait for the slides to be posted to really pick through those. Finally, I think in order to limit the amount of data being processed, he set a small TTL. TTL means time to live. Basically, if you set a record to a specific time, it means you can read that record for x amount of time and after that it gets deleted and you can’t read that record anymore.

Evan’s talk also taught me that there are two famous Kobayashis… the major league eater and one related to Star Trek (more on this in the “random things I learned at RubyConf” post).