Node.js meet IBM PureApplication System – Part 1 of 3

This is a re-post from the on on the Expert integrated systems blog.  Please go there for discussions and feedback.

In this three-part series I will explore how you can create IBM PureApplication Systems patterns for the “hot” open source web technology: Node.js.  In doing so, I will revisit the basics and the details behind creating PureApplication System plug-ins (extensions) as well as how to use them to create cloud application patterns.

The paramount goal of this series is to create more example plug-ins for the PureApplication System and contribute to the IBM PureSystem’s growing ecosystem.  Essentially, this is reinforcing the points we have been making in this blog about the openness of IBM PureSystems, its extensibility, its ease of use, and simply how it is transforming enterprise IT into the cloud.

Extending IBM PureApplication Systems

The IBM PureApplication System is the platform as a service (PaaS) platform for the IBM PureSystems family of products.  As I mentioned in a previous post (Should you virtualize your pattern or not?) PureApplication System has two means to define workloads: virtual applications (vApp) and virtual systems (vSys).  Each model has a set of extensibility points.  In this post I focus on vApp since that is the one with the most flexibility.

To create a vApp pattern one has to reuse or create plug-ins that represent the cloud components, cloud services, and the links constituting the pattern.  The IBM PureApplication System comes with a multitude of plug-ins supporting the patterns that are available right of the box, such as J2EE applications, web-based applications, transactional applications, and other enterprise-oriented patterns.  There is also a growing ecosystem of patterns available in the IBM PureSystems Marketplace, where third party business partners have created plug-ins for additional components and made them available.

For this post we will focus on creating a plug-in for Node.js to support typical patterns of deployments of this new web application stack.  This will be done in three parts and will cover all aspects of the designing, implementing, building and deploying plug-ins as well as using them to create patterns, and of course using the resulting patterns.  With that, let’s jump right into the details, starting with a refresher overview of vApp plug-ins and then an overview of Node.js so we can have a good idea of what type of patterns we want to create, which will help define the design goals for our plug-in.

Anatomy of a vApp plug-in

In the IBM PureApplication System, patterns are composed (visually) using cloud components, cloud services, and links.  Cloud components represent a part of a software stack, such as the WebSphere Application Server, the DB2 database, or the Ruby on Rails application server, and others.  A cloud service is a component that is shared across deployments, for instance, the IBM MQ service is a cloud service that you can add to your pattern.  Finally, links represent connections among cloud components and between cloud components and cloud services.

Creating a new plug-in amounts to deciding where it logically fits into the categorization above.  Sometimes this means creating multiple plug-ins to address the fact that a component might need to be connected to other components; in this case, you would create a link plug-in to connect the components.

Once decided, the next important step is to decide the attributes that the component (or link) exposes and the type and values that each attribute can take.  For instance, for a Node.js component, a sensible set of initial attributes might be:

• the name of the application
• the version of Node.js to use
• the URL to the git repository containing the Node.js application

Naturally, many more attributes are possible; however, these three make for an easy start.

The IBM PureApplication System plug-in model is very systematic and uses a strict structure for all aspects of the plug-in.  In short, the structure captures the following four key items:

• metadata.json – meta data information about the plug-in, such as: name, description, help files, image files, and the list of attributes exposed along with the attributes’ types.
• config.json – contains additional configuration information to optionally give hints to the PureApplication System deployment process.
• scripts – contains the scripts that will configure and install the cloud component that this plug-in represents.  Additional management scripts include those to stop, restart, and so on.
• files – all supporting files for the cloud component, for instance, binary installation files and configuration files.

The plug-in’s UI is represented by the icon image set in the metadata.json.  The IBM PureApplication System pattern designer web application also automatically creates the appropriate remaining UI for the plug-in using the meta data information.  This includes providing property sheets for the plug-in’s attributes as well as showing help files and enabling policies and links, if these are enabled and loaded.

The final part of a plug-in are the optional links and policies.  Links represent the connections between a cloud component and another cloud component or a cloud component and a cloud service.  When a cloud component supports a link then it is added to its metadata.  Similarly, quality of service (QoS) policies are used to provide non-functional capabilities to a cloud component.  For instance, specifying how a cloud component scales by horizontal replications or by growing its instance resources, such as CPU or memory, vertically.

Links and policies constitute advanced features of the IBM PureApplication System plug-in architecture and I may discuss them in a future post.  For now let’s focus on getting Node.js and the IBM PureApplication System acquainted.

Introduction to Node.js

Node.js is the “new hotness” in web application development.  But what makes Node.js interesting to web developers and what is the craze all about? And how does it compare to mature platforms such as JavaEE and Ruby on Rails?

Implementing an application server, in its simplest form, amounts to implementing a daemon process that executes client requests as they arrive.  The requests need to be parsed, routed to appropriate services, executed, and responded to.  The faster this cycle can be achieved, the more requests can be managed, and thereby the better performant is the resulting application server.

Naturally, requests sometimes fail and also need to be secured and isolated.  These complications, and others, imply that designing an application server usually


results in using OS-level resources that are designed for concurrency control while providing some level of security and isolation.  As such, most application servers use OS processes or more frequently OS threads as a basis for their core implementation.  Such is the architectures of most Java-based application servers.

However, while OS processes and threads make it easy to create heavily concurrent applications and servers, they also have drawbacks.  Primarily, thread-based application servers (and even more so for process-based servers) tend to be heavyweight, that is, they require lots of system resources: CPU, memory, and storage.  This can become a significant issue when running application servers that have to deal with significant number of concurrent requests or when these requests last a long time or perform tricky computation, such as complicated database queries.

Node.js is designed to deal specifically with this “heavyweight” implementation consequence of threads-based servers.  The distinguishing characteristic of the Node.js application server, which gives it its “lightweight” implementation claim is that instead of using OS-level mechanism to deal with multiple requests, it instead uses application-level callbacks and non-blocking IO libraries.  The JavaScript language uses callbacks as an essential mechanism for modularization and extension, and Node.js, as it is implemented in JavaScript, uses the same callback mechanism.

Another interesting aspect of Node.js is that it embeds the highly performing Google V8 JavaScript engine.  The V8 engine has revolutionized JavaScript virtual machines by utilizing advanced techniques pioneered in other interpreted languages such as Smalltalk and Self. Some of this includes aggressive optimization like generational incremental garbage collector, Just in Time compilation (JIT), and inline caching of functions. The result is a virtual machine that makes JavaScript a worthy competitor to other interpreted languages and even (in some cases) compiled languages.

Additional benefits of Node.js are that as web application programming interfaces (APIs) moved to using JavaScript Object Notation (JSON) as their primary data interchange format, most web services now include a back end that exposes a JSON-based API with a front end that uses the JSON data to create the user interface.  Creating the application using Node.js allows both the back end and front end to be in the same language, thus potentially simplifying the codebase and associated assets, such as, data validators, tests code, tests data, and so on.

Node.js plugin goals

The primary goal for our Node.js PureApplication System plug-in is to allow Node.js applications to be simply and quickly installed, ran, and managed on an IBM PureApplication System setup.  We will assume that the Node.js application is located on a Git repository, such as Github.com, and that it uses N
ode 
Package Manager (NPM) to manage dependencies. The latest version of Node.js will be added into the plug-in and with a version attribute, a different version can be selected and used instead.

To keep the plug-in simple, we will not deal with typical scaling features of Node.js applications such as using aNginx HTTP/reverse proxy server, a load balancing service, or data caching service
s
.  These additional features can be added to this current plug-in in the future and are left as an exercise to the reader.

What next?

In part 2 of this post, I will complete the Node.js plug-in and deploy a non-trivial open source Node.js application found on Github.com.  I will also discuss how you can build the plug-in and install it into an IBM PureApplication System setup.

Open 24x7x365: The IBM open PaaS and private cloud platform

This is a re-post from the on on the Expert integrated systems blog.  Please go there for discussions and feedback.

In this post, I briefly highlight the features of the IBM PureSystems offering, and particularly focus on why I claim that it is an “open” platform.  I give a high-level overview of its plug-in mechanism and point to several example extensions that are available now for PureSystems customers.

What is IBM PureSystems?

IBM PureSystems is a turnkey private cloud solution that for the first time integrates a flexible, expandable hardware platform with layers of software to manage the system itself along with the workloads that it runs.  Essentially, the IBM PureSystems offering combines an infrastructure-as-a-service (IaaS) substrate with a platform-as-a-service (PaaS) layer that also provides means to codify workload components into patterns.  With these patterns, users can effectively deploy workloads and applications in a predictable and repeatable fashion.

Additionally, IBM PureSystems embodies all of the seven characteristics, which I mentioned in a previous post (“Stormy forecast”), to make this private cloud a modern incarnation of previous private clouds.  Features, such as extensibility, built-in support, autonomic self-healing, and customization, are all part of every IBM PureSystems.

However, one additional feature of IBM PureSystems places it above and beyond typical private IaaS and PaaS solutions — it’s based on an open architecture and design. Specifically it includes:

• Open extensible hardware platform
• Open plug-in mechanism to extend its software
• Open marketplace for patterns and software you can run on IBM PureSystems

Open ecosystem and APIs

First, it’s important to distinguish between open source and open ecosystem and open APIs.  As an analogy, in the mobile world there are two dominant platforms today: the Android and iOS mobile platforms.  Arguably, they are both open, however, while Android is fully open (source code, APIs, and ecosystem-open), the iOS platform is open in its ecosystem and APIs.  Anyone can join both platforms (for a small fee in the iOS case) and different rules apply to participants of each of the platforms.

Is it valid to say that having an open ecosystem and open APIs without open source is providing an open platform?  Well, it’s a matter of semantics and business model.  Because by open, we usually mean any participants (within some rules) can contribute and can create applications for the platform; using this definition, both models (Android and iOS) are open.

Providing an open platform that is based on open ecosystem and open APIs is not an iOS novel innovation.  Microsoft Windows has done exactly this for years and before that the IBM OS 360 had pioneered this model.  Although the participants don’t have the source code of the entire platform as in other models, in this participatory open platform, the APIs are fully published and the system provides key points of extensions and contributions.

IBM’s PaaS platform falls into the category of an open ecosystem and open APIs.  It does so by having open hardware architecture and an open plug-in mechanism to create extensions.

Open hardware

First, IBM PureSystems is based on standard industry server architecture.  Although the hardware architecture is sophisticated to include fault-tolerance attributes, nothing in there is proprietary.  The various units that make up an IBM PureSystems are designed with commercial off-the-shelf (COTS) parts that can be replaced.  For instance, each unit contains an Intel Sandy Bridge CPU that anyone can purchase from Intel.

However, beyond the hardware openness, what really sets the IBM PureSystems apart from other private cloud solutions is in its open plug-in mechanism, which includes open APIs and results in an open ecosystem.

Open plug-in mechanism

IBM PureSystem workloads are defined using patterns.  Think of a pattern as the definition of a template using cloud components and linking these components.  The components on a template represent the resources (typically software) and capture the necessary details to set up and configure that component.

For instance, a component representing the IBM WebSphere Application Server is part of the J2EE pattern and can be modified when users specify an EAR file for the application they want to add in their workload.  Patterns are built visually by using the IBM Pure Application System Designer and the library of components available on the left side of the interface.

A pattern can be saved and parametrized by leaving certain aspects unspecified, for example the EAR file name and location.  A completed pattern, which is one that is valid and has all of its missing parameters filled, can be deployed on an IBM Pure Application System instance.  Because patterns can be instantiated many times and be shared and customized, this process allows the predictable and repeatable means that enterprises need to manage their workloads.

The components and links library are defined in plug-ins that can be added dynamically to the system.  The plug-in architecture is open and allows a wide range of cloud components to be defined and connected together.  Each plug-in defines the metadata of the component or link including the attributes (required and optional) and also the help data, icons, and so on.  Additionally, each plug-in follows a strict structure that allows various parts (binaries) to be defined, and custom scripts to install and configure the processes that constitute the plug-in.

One key aspect of cloud components and links defined as plug-ins is that they do not dictate the exact cloud resources that they need; instead, they suggest this information by giving hints and limits to the IBM Pure Application System deployment process.  Using this information, the system can make appropriate placements and resource provisioning decisions resulting in an optimal configuration for your workload.

Ready for PureSystems: Plug-ins available now

IBM has been working with hundreds of independent software vendors (ISVs) to create various cloud components.  By the time you read this post, many of these plug-ins will either be included as part of every IBM Pure Application System or be available in the marketplace.

We also welcome your contribution to this open marketplace by downloading the Plug-in Development Kit and looking at the examples and tutorials available online.  In a future post, I will describe in simple steps how a common popular open source framework can be added to the IBM Pure Application System platform in the few hours that is required to create a plug-in for it.