Kifi is an innovative Silicon Valley startup that is creating a new way to search the Internet and store or share information. Kifi allows you to easily keep and tag anything you find online – an article, video, picture, email, – then quickly find it on top of your favorite search engine results. Kifi also surfaces at the top of results relevant that your friends kept, as well. Kifi layers social tools on top of what you keep by including messaging to friends and highlighting and annotation of information found on pages. The Kifi search infrastructure also lets you tag and organize information you are saving, just like in a bookmarking system.
Kifi CTO and co-founder Eishay Smith knew that he was undertaking a significant challenge in building an application that would attempt to perform multiple computationally intensive tasks in an extremely limited time window. What’s more, the architecture had to support this level of performance at all scales and be flexible enough to scale up and scale down instantly to meet often unpredictable customer usage patterns. Therefore, the Kifi team built a Reactive application structure to enable rapid iteration, handle massive number of simultaneous users and processes, and maintain a diverse range of services. In short, Kifi needed:
“We need the ability to run Web applications that change very frequently and need to be very flexible in a true Service-Oriented-Architecture. Lightbend's Play framework and its interaction with Akka handles this beautifully and gives us a very Reactive system. Since we are data heavy application and we have a lot of different services and data types, we also rely heavily on multi-threading that is very well integrated into Scala. Support for things like WebSockets makes it much easier for us to maintain performance and a broad set of features and services in a small number of virtual machines.”
Eishay SmithCTO and co-founder - Kifi
The original engineers building Kifi had previously worked with Scala-based application architectures, and knew the Lightbend stack would meet these requirements. For the core application, Scala and Play are used for all web services. “Our AngularJS website, browser extensions, and mobile clients call our Play API, speaking JSON,” says Stephen Kemmerling, a Kifi engineer who has worked extensively with Play building the messaging service. Kifi also uses the native WebSockets capability in Play to enable low-latency, two-way client communication such as for real-time messaging and updates to clients in browser extensions, iOS, and Android.
On the backend Kifi is built using several MySQL databases with a multi-caching layer in between the primary database and the actual application using Memcached and a custom in-memory cache. The cache was built to improve response times and run more of the application and tasks in memory. Yasuhiro Matsuda, based on experience as the Chief Architect of LinkedIn distributed search, led development of Kifi Search using Lucene components to adaptively return highly personal results. Additionally, Kifi has a custom Graph engine, built completely in Scala, to generate user recommendations.
For distributed work and task scheduling, Kifi uses Lightbend’s Akka. “Our scraping service downloads every page that someone keeps so we can index and search with it real-time,” explains Raymond Ng, an engineer working on Kifi. “We also use Akka’s actors for work such as syncing social network updates, system checks, and link imports.” Coordination of running services and service topology is managed using Zookeeper, a service discovery and management program that is commonly used in service oriented architecture applications.
Additionally, engineers at Kifi have fostered a strong Scala and Play community. The Kifi Engineering blog frequently posts about machine learning, Scala, Play, and Slick. Andrew Conner, who has built applications using Play for over two years, started the SF Bay Area Play Framework Meetup group last year. It has attracted high interest from local Scala/Play enthusiasts and people interested in building modern web applications. “We’ve been pleasantly surprised with the community response to the meetup group. More and more companies are embracing quick development and reactive architectures, and Play meets the needs of small startups to large enterprises exceptionally,” Andrew added.
Kifi launched its app and has thus far experienced excellent performance and continued rapid development. The team's design for a Reactive infrastructure leveraging Scala, Play and Akka has allowed them to perform dozens of deployments per day in a highly aggressive continuous deployment environment. With the Lightbend Scala components Kifi has:
“We built Kifi from the ground up as a Reactive application and the Lightbend tools gave us the best combination of scalability, agility and reliability. We also got the added benefit of a great recruiting tool because great engineers want to learn Scala.”
Eishay SmithCTO and co-founder - Kifi
Inspired by this story? Contact us to learn more about what Lightbend can do for your organization.