I recently attended a talk by Andrew McAfee (Harvard Business School) at the famed Palo Alto Research Center (PARC). Andrew is one of the more eloquent speakers from business schools on matters of technologies. His talk at PARC was on Enterprise 2.0 or how the recent evolution of the Web is infiltrating inside enterprises and what enterprises can do to take advantages of new Web innovations.
While Andrew's talk was delivered passionately, most of what he discussed was not new to me, and I imagine not new to anyone monitoring the valley or Web 2.0. However, Andrew mentioned one thing that caught my attention and made me decide to post this entry...
Prospect Theory and TiVo
After discussing the various ways in which Web 2.0 is impacting enterprises, e.g., blogs, wikis, social networking, and virtual markets, Andrew highlighted some challenges that still remain. In particular, he explained that Web 2.0 technologies will only slowly become mainstream in enterprises. This is because, in most decision makers' minds, Web 2.0 innovations are not 10x (or more) better than current alternatives. The 10x figure is not magic, it comes from Prospect Theory.
Developed by Daniel Kahneman (Princeton University and Hebrew University) and Amos Tversky (1937 - 1996), Prospect Theory (PT), for which Kahneman would win the Nobel prize in economics in 2002, tries to explain the psychology of decision-making of agents that are faced with deciding between alternatives in the presence of risks. In particular the theory tries to explain an agent's decision-making in the face of the prospect of "better" alternatives (B).
One interesting result of the theory, confirmed many times empirically, is that when agents are faced with the decision to move (or change) to alternative B from a known, workable alternative (A), they tend to overestimate alternative A as being three times better than what it is and at the same time underestimate the prospective B as being three times less valuable than what it is... This implies that in order for alternative B to become mainstream (thereby displacing A), assuming no additional changes to the market and rounding the numbers, the new prospect B should be at least 10x more "valuable" than the alternative it is displacing, i.e., A.
Andrew explained that the 3x figures are supported by various empirical data drawn from many experiments. He illustrated his point by making a quick (unscientific) poll with the audience. He asked who had, loved, and evangelized TiVo. Interestingly, about 10 to 15 percent of the audience raised their hands and never let it down... The point is that while TiVo has a good, passionate, following, however, it is not generally seen as being 10x better than current TV experience, and thus is not causing a mainstream change in current TV viewers.I am not an economist, but I immediately saw the importance of Andrew's point and how he may be right applying PT's results to the infiltration of Web 2.0 tools into the enterprise. I also quickly thought of applying the same to one of my research area: Web computing.
The future of Web programming
Web programming has evolved from CGI scripts written mostly Perl, to server-side applications running on complicated containers written in Java and .NET, to now increasingly using various agile and lightweight server-side frameworks using dynamic scripting languages such as JavaScript, Python and Django, Project Zero, and Ruby and Rails. This evolution is not accidental, but rather, I believe the result of the need for agile reaction to changes in requirements for such applications as well as the new complexities of Web applications; and finally the increase performance of networks and of the devices that we are using to connect to the Web.
It is well documented that individual and teams of programmers can quickly build Web 2.0 Web sites in Ruby on Rails much faster than they can with Java EE or .NET or Perl. Naturally, there will be cases where this productivity is not applicable, say areas where lots of legacy code is to be reused, or where skilled programmers cannot be found, or where scalability (to many millions of concurrent connections) is required. However, on average, it is fair to assume that building Web applications using modern scripting frameworks, such as Rails, allows a huge advantage to programmers.The DSL hypothesis and the prospect of Ruby and Rails
The way to understand this increase in productivity, I believe, is in what I call "the domain-specific language (DSL) hypothesis." Dynamic languages, especially Ruby, allows the creation of libraries and mini-languages (DSLs) that allows one to represent domain-level concepts in a way that they can be programmed directly. For instance, in Rails programmers can represent the data model of their application using the ActiveRecord DSL. Using ActiveRecord constructs, programmers directly represent the relationships amongst the models and their "shape". Using metaprogramming the constructs are translated into code that is executed by the framework and the underlying database. This meta-level expression capability saves the programmers a huge burden and increases her productivity.
However, while these dynamic languages and DSL are increasing productivity of programmers everywhere, they are not making as much inroads into enterprises... Why is that? Maybe Prospect Theory is the answer; or maybe, we are just too early in this wave change and we need to give enterprises and enterprise programmers more time to catch up with the rest of the community. I don't have enough evidence to suggest that languages and frameworks like Ruby and Rails provide 10x increase in productivity over Java EE or .NET, but from my own experience I believe the figure is close...To help make my point and to encourage naysayers to look seriously into these alternative Web computing paradigms, I have created a list of 10 reasons Why I love Ruby. I posted reason #1 last year and promised to continue posting the subsequent ones over the next few months. I am not claiming that together these reasons will prove that Ruby and Rails will become mainstream, however, I hope they will encourage enterprise programmers to give strong considerations to trying these alternative paradigms, as I believe once they get a taste of the productivity boost, they will raise their hands like these TiVo passionate in Andrew's lecture.
Change log
24 February 2008 - first posted
