Lei Gong

Programmers love side-projects. It’s a great way to stay on top of trending technologies and, sometimes, make some extra dollars. The success of a side-project, however, often has nothing to do with the specific shinny technology you are using. It is about actually FINISHING it. To me, that means starting a project with the end in mind. You have to have some purpose in order to squeeze out the nights and weekends to build it. To that extent, what would be better motivations than some extra incom ...

In a previous post, we talked about caching Spark models in memory for a web service so that the prediction latency is reduced. As with any web applications, caching strategy can get very interesting, but the patterns of caching a machine learning model are relatively straightforward, since the model is likely to be static unless there are updates. In particular, I find Guava provides some handy in-memory cache solutions for our use case. In the rest of this post, I am going to walk you through ...

We had talked about various ways to serve machine learning results in production (see an ealier post Predictive Model Deployment with Spark for example). That article outlines three architectures of model serving systems. In particular, I found using Spark’s internal serialization logic to persist/load models to be both flexible and reliable. As a follow-up, in this post, I am going to show case how to serve a simple Spark MLLib machine learning model, i.e. Naive Bayes classifier as an example, ...

Front-end development is an interesting beast of its own. It mostly deals with user experience, which requires constant tuning of page layout and experience flow. Do you find yourself refresh that page for the 18 times? Do you find yourself run the script every time before deployment to uglify the page? There is a ton of room for automation here! Most famously, the Front-end world seems to be using Gulp or Grunt for this kind of task (this statement may be false very soon with the speed of tool ...

As part of the task to automate our production modeling pipeline, I was exploring options for storing offline training metrics on AWS. The goal is two-fold: 1) to keep tracking of training evaluation metrics and 2) for our internal web app to consume and display this data. AWS provides various storage options such as file storage system S3 and NoSQL database DynamoDB. This post summarizes the differences between these two services and how/why we choose one over the other. Bear in mind that the c ...

JSON is a widely used protocol for communication between services. Recently, I was tasked to write some serialization/deserialization logic in Scala, so I did a bit research and find json4s to be an effective option for my use case. Here are some simple code snippets to get started with this library. BasicsJSON AST is used to model the structure of a JSON document, which is the fundamental building block of json4s. 123456789101112sealed abstract class JValuecase object JNothing extends JValue / ...

It’s been a few month since I first set up my blog using Hexo and Github Page (more details are in an earlier post How To Build A Blog With Hexo On Github Page). As a bonus for myself updating a new post every week, I decided to upgrade my blog with more features. It is an experiment to learn how the SaaS ecosystem is structrued around blogging community. In this post, I am going to introduce some of the handy SaaS solutions out there with freemium plans and walk through a few how-to’s. Share co ...

Are you someone who is highly motivated, always keen to learn new skills, but not feeling particularly accomplished on a day-to-day basis or, even worse, consistently anxious about missing some buzz words from Hacker News? Read on. You seriously need changes. Learner’s SymptomI felt exactly the same way and I came to realize these characteristics actually characterize, what I’d like to call, the Learner’s Symptom. Been in the tech industry, everything is moving super fast with new programming la ...

Yep, you guessed it right. This is one of those retrospective articles flying all over the Internet at the end of the year. But, what’s a better timing than now to look back and learn from the past, or, more importantly, to look forward and plan the next big one? 2016 is an exciting year for me both professionally and personally. Professionally, Steve Jobs’ famous quote from the 2005 Stanford Commencement has always been inspiring to me, and I think I finally have found what I love to work on t ...

In an earlier post, I talked about predictive model deployment with Spark. That post mainly addresses the issues around how to persist a trained model and serve it online in real-time to users. However, there are many other concerns when one set out to build a predictive model system, especially around the robustness and reliability of the system, since predictive models can go terribly wrong without a single error message in the log. In this post, I am going to discuss, what I’d like to call, t ...