Lei Gong

Yep, you guessed it right. This is one of those retrospective articles flying all over the Internet at the end of the year. But, what’s a better timing than now to look back and learn from the past, or, more importantly, to look forward and plan the next big one? 2016 is an exciting year for me both professionally and personally. Professionally, Steve Jobs’ famous quote from the 2005 Stanford Commencement has always been inspiring to me, and I think I finally have found what I love to work on t ...

In an earlier post, I talked about predictive model deployment with Spark. That post mainly addresses the issues around how to persist a trained model and serve it online in real-time to users. However, there are many other concerns when one set out to build a predictive model system, especially around the robustness and reliability of the system, since predictive models can go terribly wrong without a single error message in the log. In this post, I am going to discuss, what I’d like to call, t ...

ExpressJS is an unopinionated, minimalist web framework for NodeJS. Among many others, middleware is an important concept to build any ExpressJS applications. Its significance is clear from the following paragraph in the official ExpressJS tutorial, Express is a routing and middleware web framework that has minimal functionality of its own: An Express application is essentially a series of middleware function calls. As part of my own learning, we are going to look at a few examples in this pos ...

Recently, David Heinemeier Hansson, a.k.a @dhh, the creator of the Ruby on Rails web development framework and a successful entrepreneur, was interviewed on the Tim Ferriss Show. I have been following him on Twitter for a while and it was always interesting, sometimes enlightening, to learn his perspective on habit, business and other matters. Over the 3-hour show, he touches on many of these aspects and I resonate many with ...

My background was primarily pure science with four years in college studying mathematics and another four years in a PhD program doing statistics. Trainings in these two subjects taught me how to think/reason rigorously and mine information out of datasets. Those are valuable skills that I really enjoyed learning. At the same time, I am passionate about the growing tech scene in the Bay Area, mainly because of the fast speed things are moving and the massive impacts an individual can have. With ...

Predictive models, a.k.a. machine learning models (if you prefer the buzzword), play huge parts in today’s technology companies. They are used to uncover valuable patterns from mass datasets that are just too complicated for humans to consume directly. Spark is a large-scale data processing engine that handles large datasets efficiently in a distributed way. Given Spark’s already high adoption as the data process engine and its rapid development in machine learning modeling, it is natural for ma ...

If necessity is the mother of invention, then laziness is the father. Recently, I came across a great story behind park.io, a service helps you to backorder domain names. At the 1000-feet level, the guy wrote a script that automatically gets any expired domains of interest. Simple idea, right? This company now generates $125K revenue per month with only one employee - the founder himself. I bet many people had similar ideas before. We can talk a lot around why ideas are cheap and only executio ...

Code review is a big part in daily software development here at Valley. Every production change requires at least a +2, ideally from someone who knows the project or the particular part of the codebase well, before it can be merged. Code review serves multiple purposes. On a very high level, it helps to Enforce high coding standards. Find bugs at lower cost. Share knowledge between developers. With all these benefits, code review comes at the cost of short-term development speed, although it ...

I love working on side projects. They are great opportunities to try out new ideas, technologies and, who knows, they may become something big one day. Up till now, I have been hosting most of my side projects on Github. It has been huge in the open source and enterprise world as de facto git hosting service. But, lately, I came across Gitlab, which seems to bring the goodies from Github and Bitbucket all together. As I am planning my new side project (more on this later :-)), I want to revisit ...

Apark Spark, the allegedly hottest open source cluster computing project, recently released a major upgrade to its 2.0 version. With new changes and updates, the performance of its computing engine sees significant improvements. To help people get started or transition to this new version, I started a series of blog posts to cover Spark 2.0 related stuff. This article in particular talks about one of its major API updates; that is, the introduction of a new interface Dataset (Well, technically, ...