On scalability and why it’s so important

by Vladimir Piskarev

I’m not going to say anything really new here. Rather, it’s a way to summarize thoughts on that, quite important to me, matter.

I have a deep interest in object-oriented programming; more in the original idea than in any specific implementation (though I find Smalltalk to be an execution par excellence) or methodology (however helpful design patterns can be). I was immensely impressed by Alan Kay’s talk at OOPSLA 97. Here’s the link. I continue to enjoy it every time I’m reading it. It’s influential, to say the least. Without doubt, Alan is one of the most important thinkers of our time, probably of all time.

Of course, the central idea underlying object-oriented programming is how we can deal with large-scale structures effectively. It’s all about scalability. As Alan Kay excellently put it: “As size and complexity increase, architecture always dominates materials”. When ARPANET was being designed in the sixties, its designers thought deeply about scalability of “the intergalactic network” — the result is, of course, the Internet as we know it — probably, the largest structure ever made by the human.

Now, many seem to be concerned more about performance than scalability. Some may even believe it’s the same thing. It’s hard to say why this syndrome is so deeply embedded in our nature, as programmers. Indeed, system programmers may be thought of as “high priests of a low cult”. The cult is, of course, optimization. But the irony is that staying in the “pink plane”, strongly focused on optimization, we are often optimizing something that can’t be optimized very much. While such optimization can bring tangible tactical benefits, it can also bring much less tangible, but strategic losses (skewed design, closed options, hard to maintain inflated code base, etc.). In contrast, thoroughly thinking about scalability is strategically important, though perhaps less perceivable tactically. When a first Smalltalk took a while to compute “3 plus 4”, Alan could not care less. He knew about the Moore’s law and designed for scalability, not for performance per se. The rest is history.

“If you think programming is small, that’s why your programs are so big.” Our industry as a whole could benefit if we begin to think more deeply about all aspects of scalability when doing system design. And that’s the main lesson from Alan Curtis Kay, to me.

P.S. Once you start blogging, it’s hard to stop. That’s why I’ve long resisted creating a personal blog. Now it’s frightening to think what will happen when I discover Facebook and Twitter.