Node.js meet IBM PureApplication System – Part 3 of 3

This is a re-post from the on on the Expert integrated systems blog.  Please go there for discussions and feedback.

In this final of three posts about our Node.js plug-in for IBM PureApplication System I will cover testing and debugging plug-ins and deployments.  Additionally, I briefly discuss some advanced features that could be useful moving forward and point to some references as well as listing some tips and tricks using IBM PureApplication System and the Node.js platform.

Recap: Part 1 and 2

In part 1 of this series I discussed the plug-in model for IBM PureApplication System along with details on how to create one.  Since the series is about demonstrating how open this platform is, I also picked and discussed the hot web application framework Node.js.  After a short background, I discussed how to design a plug-in for Node.js that could support simple patterns deploying Node.js applications from a Git repository such as Github.com.

The subsequent post (part 2) covered how to create the plug-in designed in part 1.  Some of the details covered in part 2 involved deep diving into how to create, build and install a new plug-in.  It also included discussions of the metadata.json and config.json files as well as the scripts to configure, install and start the Node.js server.  In that same post I also showed how to install the plug-in and use it in a simple pattern on an IBM PureApplication System environment.

In this post I will cover what you can consider as pitfall avoidance and tips and tricks, as well as pointers for advanced features.  In particular I also have a brief discussion of where to go from here.

Testing and debugging plug-ins

Testing any plug-in amounts to deploying it into a running PureApplication System setup, creating a pattern with the plug-in components and using that pattern.  However, such a test cycle can be time consuming and error prone.  It is therefore recommended that you individually test the different parts of the plug-in before doing the eventual complete tests.  I give three primary approaches to testing your plug-ins and debugging common issues that may arise:

1. Build / Deploy / Instantiate / Test

In this scenario, you build your plug-in according to the Plug-in Development Kit (PDK), and deploy it using the IBM PureApplication System dashboard (see part 2 for details).  Create a plug-in, and test it by deploying an instance. While this approach should always be included in your test plan, it is usually the most error prone, and it can be long.  Any issues with the plug-in will be known at a later phase and thus requires you to restart.

2. Build / Test / Deploy /Instantiate

In this approach we move the test phase earlier—before even deploying your plug-in.  This requires you to create tests for the various parts of your applications.  Using testing frameworks like PyUnit you create unit tests for the various Python files in your plug-in and make sure that while they can be built into a plug-in, they are also passing your tests.  You might need to isolate your code from the PDK files or stub or mock any dependencies.

3. Test / Build /Deploy / Instantiate

In this final approach we move the tests even earlier.  The idea is to create your Python scripts even before packaging them into a plug-in and testing (through PyUnit, for instance).  Once your configure.py, install.py, and start.py files work fine, you can then retrofit them to follow the PDK format.  In this case you are using a set of Python scripts that you are sure are able to configure and install your component, prior to even packaging them in a plug-in.

In any of these strategies, I want to highlight two common pitfalls that occur with plug-in developers.  First, once you build your plug-in, is verifying the IBM PureApplication System dashboard correctly shows and list your plug-in.  Most added plug-ins are shown under the “Other Components” in the Virtual Application Builder tool.  There you should see an icon matching the image you used in your metadata.json.

Second, if after successful installation of your plug-in you do not see your component then it’s likely that your IBM PureApplication System does not have the pattern type for your plug-in enabled.  Please refer to part 2 of this series on the steps to follow to enable your plug-in.

As one can easily notice, the main difference between the three approaches is when test is introduced.  Since I am of the school of thought that testing early and frequently is usually a great idea and a worthwhile investment, my primary recommendation is to move your tests as early as possible.  Finally, it’s worth noting that most pitfalls and debugging approaches discussed here are generally applicable to any approach used.

Node.js tips and tricks

In this section I give a quick primer with pointers on how to get started with Node.js.  This is not intended to be a complete overview of the subject but rather a set of reference links that I have found useful as I myself got started in Node.js and in creating this three-part blog post series.

1. Installing Node.js

The Installation wiki page for Node.js on Github.com contains what is the official set of instructions for getting Node.js running on your platform.  Most of the instructions assume that you have root access to the system you are using and that you use one of the following popular operating systems: Mac OS X, Windows 7, and various flavors of Linux.

Current Node.js releases depend on Google’s V8 engine.  So, one aspect of the installation for Node.js is getting the V8 engine installed onto your machine.  This might involve building it.  Generally, this step works fine for most operating systems.  However, you need to make sure you have correct Python interpreter as well as correct C/C++ compilers and libraries.

For the Mac OS X these come with installing the latest Xcode development package, for Windows the MS Visual Studio should contain the correct dependencies, and for Linux the latest GNU C/C++ compiler and libraries should suffice.

2.Adding packages via NPM

Like most modern software frameworks, Node.js’ architecture is modularized.  That is, while the basic Node.js installation comes with full features, it also lacks various components which are then added (as needed) after the fact.  This allows your Node.js installation to be easily customized and extended.  For instance, if you want to use the language CoffeeScript (a JavaScript-compatible language) then you simply add the coffee package.

To install and manage these modules, Node.js uses the Node Package Modules (NPM).  NPM is a standalone package manager that usually needs to be installed separately from Node.js itself.  However, once installed, you can use NPM to easily add new modules (or packages) to a Node.js installation as well as updating existing packages.  Even Node.js itself can be installed and updated using NPM!  Finally, the NPM web site (https://npmjs.org/) also constitutes a repository of various OSS modules you can readily access, as of this writing more than 18,744 modules were available.

3.Troubleshooting and debugging

I have found three areas that cause issues when getting started with Node.js.  First, the installation process, while usually flawless, can be painful for some users.  Primarily this has to do with not having the correct development environment when building Node from sources and setting up the V8 JavaScript engine.  Carefully following theInstallation wiki page for the operating system you are targeting is your best solution.

Second, while installing Node.js modules through NPM is as easy as issuing the command: $npm install coffee it has a couple of pitfalls.  First, NPM allows a user to have multiple module package directories (where the modules are installed) as well as a global one.  To install modules in the global directory, you must use the -g option when installing.  Also, since the global module directory usually defaults to: /usr/local/lib/node_modules in most UNIX compatible system, accessing this directory will require root privileges.  So all installation command must be done with that access:  $sudo npm -g install coffee.  It’s also recommended that the module directory be exported from the shell where the application will be executed, this is achieved with: $export NODE_PATH=/usr/local/lib/node:/usr/local/lib/node_modules

Finally, when running Node.js applications it’s usually important to run $sudo npm -g install in the application directory.  The application’s package.json file is then used to determine the dependencies and what packages need to be updated and/or installed.   The application can then be ran using the $node command.

Plug-in advanced features

Creating or re-using an existing cloud component plug-in is the first step to creating patterns for IBM PureApplication System.  While, as we demonstrated, you can use a cloud component to create a simple pattern, anything more complicated requires other cloud components as well as linking these components together and adding quality of service (QoS or policy) features to the current components.

While a thorough discussion of any of these features (linking, QoS) would warrant its own post, I want to highlight each here as well as monitoring and scaling.  The goal is to give hints at what is possible with IBM PureApplication System and in some cases point to where more information can be found.  Also, the paramount goal is to re-iterate the fact that the IBM PureApplication System plug-in model is flexible, open, and enables modeling and deployments of complex workloads.

Adding links

The most common advanced feature for creating comprehensive patterns is to create links between components.  For instance, one might want to create a pattern for Node.js applications and relational databases such as DB2,MySQL, or Postgres or a graph database like Neo4j.  For each of these cases, you would need to create a link plug-in that specifies the Node.js plug-in as its source and the datastore plug-in as its target (this assumes that you reuse or create a plug-in for the target datastore).

Link plug-ins are created like component plug-ins, they have a similar structure, however, they use “link” as their type and have additional parameters in their metadata.json such as “source” and “target”.  Like a component plug-in, a link plug-in can also include attributes.  For instance, connecting to a database you may want to include the database name as a link plug-in attribute as well as its JDBC JNDI URL or other types of URL references.  The link plug-in uses the attributes’ values (specified by the user) to configure the source and target components correctly.  Visually, this is represented by the screenshot above showing how the WebSphere component connects to the DB2 component in the J2EE pattern that ships with the IBM PureApplication System.

 QoS policies

Another important class of advanced features that could complement the Node.js                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 plug-in created in this series of posts is to modify it to support QoS policies.  An example of using QoS policies for a cloud component is illustrated in the figure below for the WebSphere Application Server (WAS).

The QoS policies for WAS are extensive.  In the screenshot above we show one aspect which allows the user to specify various options for the Java virtual machine that WAS will run on.  For instance, using the JVM policy one can specify the initial size of the heap as well as its maximum value.

For the Node.js plug-in an obvious set of QoS policy would be around how to fine tune Node.js and the V8

JavaScript engine.  For instance, specifying the max V8 stack size or passing options directly to the V8 JavaScript engine to fine-tune its performance.

Monitoring and scaling

The final set of features that would be needed for our Node.js cloud component to address most of the needs of modern production-level workloads is monitoring and scaling.  Most cloud environments suffer from failures.  These failures, while infrequent, are inevitable.  As such, any production-level pattern needs a means to recover from some failures and therefore some level of monitoring capability to know when failures occur or are about to occur.

The IBM PureApplication System comes with monitoring features than can be added to any plug-in.  The monitoring is done in the form of a plug-in that can be required in your own plug-in.   Without going into too much detail, some of its primary features are to help monitor processes on virtual machines (VMs) and trigger actions and aggregate the results into the IBM PureApplication System dashboard.  Additionally, the IBM PureApplication System comes built with some basic monitoring features readily available from the dashboard.  Some of these features are viewing current VM status as well remotely accessing logs for any VM and component.

A feature that works side-by-side to monitoring is scaling.  One of the advantages of using a cloud environment for your workload is the ability to quickly scale (up and down) the workload to address the immediate demand.  While scaling is not an easy feature and can be tricky and be workload-specific, some general scaling strategies such as replicating services and using a load balancer service to spread the load across a pool of services is a tried and true way to scale.

Adding scaling to a plug-in is another advance topic that can be achieved by adding a QoS policy to capture the scaling requirements of the user and then pass that information to the plug-in and use it to customize your startup scripts.  One example of using a scaling policy for a cloud component is how the WAS plug-in QoS scaling policy allows users to specify scaling rules to scale up and down based on the current HTTP request volume.

Wish list

While creating a new plug-in is relatively straightforward when you have some examples, as we discussed, there exists the possibility of various pitfalls during the development.  Simplifying and streamlining the plug-in development process is needed.  In particular the following list of potential improvements could help:

1. A plug-in generator.  This is a tool that would take some basic input like name, attributes list, image, and so on, and generate the scaffolding for a working plug-in that, of course, would have scripts that do not do anything.  The point is to get the user moving fast and have a working plug-in that they can modify and iterate multiple times after.

2. A plug-in simulator that could simulate the lifecycle of a plug-in during deployment.  The goal of this simulator is to reduce the time it takes to test the deployment of plug-ins during development.  Instead of using a real IBM PureApplication System and installing and testing a plug-in, developers would use the simulator to execute these steps in a simulated fashion in seconds versus minutes.

4. A plug-in mock testing environment. Along with the simulator, we also need a mock testing environment mimicking the real environment where plug-in lives.  Ideally, this could be extended to support full pattern development with mocked VM resources.

5. A developer plug-in catalog and browser so that all plug-ins created by a developer can be collected in one place.  The various versions of each plug-in can also be displayed along with notes for each plug-in.  This type of browser would facilitate long term development of plug-ins along with their maintenance.

Since we are always looking to improve the process of using developing and using our IBM PureApplication System environment, this list of wishful items has been communicated to the research and development teams.  Future versions of the PDK and the IBM PureApplication System platform might include all or some of these features.  If you have an opinion on these or have other development tools that you would like to see then please comment on this post.

What next?

In this three part series we did a deep dive investigation on how to support the Node.js web application stack in the IBM PureApplication Systems environment.  The investigation was thorough and took us from design to implementation and test of the plug-in.  Additionally, we discussed advanced features for expanding the current plug-in to support quality of service as well as features like scaling and monitoring.

• Preparing for IBM PureApplication System: a five part series
• Create and customize virtual application patterns
• Video demo: Install and configure the Virtual Pattern Kit for Developers
• Develop patterns for IBM PureApplication System
• Should you virtualize your pattern or not?
  
• PureApplication integration with existing monitoring
• IBM PureSystems Center
• IBM Virtual Pattern Kit for Developers

I hope you have enjoyed this series and that you now have enough information to start building your own set of plug-ins as well as consider contributing them to our IBM PureSystems Marketplace.

Node.js meet IBM PureApplication System – Part 2 of 3

This is a re-post from the on on the Expert integrated systems blog.  Please go there for discussions and feedback.

In this second of three posts on Node.js I will complete the implementation for the plug-in we designed in part oneand build and test it on an IBM PureApplication System.  In doing so I will cover the details of the process of creating PureApplication System plug-ins, as well as using them.

Recap

In part one of this series, we looked at an overview of the plug-in mechanism of IBM PureApplication Systems and how we could add support of the new OSS web application stack: Node.js.

As I noted, Node.js has some interesting architectural and implementation decisions which might make it more performant for some classes of workloads.  I’ve also identified a minimal set of attributes (three) needed to create a Node.js cloud component so that we can support Node workloads in IBM PureApplication systems.

If you did not read or don’t recall the details from the previous post now might be a good time to go and peruse it.  In this post I am going to complete the plug-in realizing this Node.js component and build and test it on an installation of IBM PureApplication System.

Creating the Node.js plug-in

Now that we have an idea of how we want to structure our Node.js pattern, as well as which plug-in should we create and what attributes it needs to have; let’s deep dive and create the plug-in.

Directory structure

As mentioned in the previous post, PureApplication System’s plug-ins have a strict form as far as directory structure and some of the files located herein.

At the root of each plug-in are two manifest directories with files: META-INF/MANIFEST.MF and OSGI-INF/node_jsXForm.xml describing the plug-in.  Additionally, in the root directory there is a build.plugin.xml file which is an Apache ANT file to build the plug-in.  We will not concern ourselves with these files as they contain mostly boiler plate content that are similar to all other IBM PureApplication System plug-ins.

From the various files and directories in the resulting plug-in, we mainly should be concerned with two directories: “plugin” and “src.”  The “plugin” directory contains the files for the plug-in of which two files and two directories are mainly important: plugin/appmodel/metadata.json, plugin/config.json, plugin/parts/node_js.scripts, and the binary files in the directory plugin/parts/**.  The “src” directory optionally contains Java source files and Apache Velocity template files for further customization, in this plug-in I will not use this feature.

The remaining files can be mostly copied from existing example plug-in with minimal modifications.  Let’s take each one of the files I modified in turn and discuss its content and the changes needed to make for Node.js.

plugin/appmodel/metadata.json

The metadata.json file located in the “plugin” directory is the key file providing most of the plug-in’s description.  This includes its unique identification and all associated information such as name, description, images and help files.

The bottom section of the metadata.json includes the list of attributes that this plug-in supports.  For each attribute you not only list its name, description, but also it’s type (like string

or integer or Boolean) as well as additional information such as help hover string and an optional regular expression to validate input and a message string when the input data is incorrect.

As I mentioned last time, the IBM PureApplication System pattern designer web tool parses the metadata.json for a plug-in to create a UI stylesheet containing the attributes listed for the plug-in.  This stylesheet lists all attributes and using the attribute’s type and other information can provide help to the user as far as the possible valid values for the attribute.

 The picture above shows the stylesheet for the Node.js plug-in.  Note how the version attribute includes a drop-down list for the values since these version values are fixed and noted as a list in the metadata.json section for the Node.js version attribute.

plugin/config.json

When a plug-in is used to create a pattern in IBM PureApplication System, the pattern can be used and instantiated as a concrete deployment.  Typically the user must provide missing information.  For the Node.js plug-in most patterns using it will need to be concretized by the user providing the URL to the Git project where the application to deploy resides. 

The other attributes having defaults means that the patterns can typically be instantiated using defaults.  However, how does the system know which virtual machine (VM) should be created, how many, and where each plug-in should be deployed on?

While the IBM PureApplication System is usually able to automate the deployment of most patterns, it uses hints from the plug-in’s plugin/config.json file as its primary means to find the affinity of the plug-in for various VM parameters.  For instance, if the plug-in requires some specific CPU or memory lower limits then this can be specified in the config.json file.

Other information such as the image for the VM as well as the VM type can also be specified.  For our Node.js plugin, we will simply require the typical image which contains the default Linux operating system and most dependencies we need for Node.js.

plugin/parts/node_js.scripts

Every plug-in will eventually be deployed when used as part of a pattern.  The deployment process is where the work of most of the plug-in is executed.  The deployment is essentially a flow that includes well known sequences where each plug-in gets to be configured, installed and started.  As you may have now guessed, these plug-in steps are achieved by executing the pattern’s configure.py, install.py and optionally the start.py and stop.py when stopped.

The configure.py script is usually reserved to setup, configure and download dependencies for the plug-in.  In our case, this is where we install PCRE (Perl Compatible Regular Expression) which is a library that we need to run Node.js on the Linux VM that is created.  In the install.py we divide the installation into three parts:

• 1.Download and install the appropriate version of Node.js directly from Github.com.  This is shown in line 37 which calls a shell script to complete the installation (shown on screenshot below).  In the shell script we get the parameters and download the correct Node.js version in lines 21 to 23.
• 2.Similarly, in the Python script we download and install Git and NPM (Node Package Manager) are lines 29 and 39
• 3.Download the application from the Git repository, lines 44 to 46.

The start.py is used as expected to start the application and the app might have various NPM packaged dependencies, we first run NPM  and then start the Node.js application server in the app directory.  Deciding if NPM should be run is done using the Run NPM attributes of the plug-in.  The basic command is something like:

exec /usr/bin/nohup /usr/local/bin/node $WORKING_DIR/$NODE_APP_NAME 2>&1 > /dev/null &

This assumes that environment variables are set for the working directory and the Node.js application name—passed as a user parameter from the plug-in.

plugin/parts/**

In the plugin/parts directory and subdirectories we keep actual installation tarball files for PCRE, NPM and Node.js.  This allows the plug-in to install in case the VM does not have access to the Internet, or usually, if a failure occurs when the download is attempted.  In general, it’s a good idea to keep all dependent files that are needed for the plug-in to correctly setup and install.  In our case, the correct version of PCRE is contained therein since the rest of the Node.js setup depends on it to be installed on the Linux environment.

Building the Node.js plugin

Building the plug-in requires Java 6 and Apache ANT.  Java is needed since Apache ANT is a Java tool.  Running the build is easily done with one command:

$ant -f plugin.build.xml

A correct installation of the Plugin Development Kit (PDK) means that the dependencies and other associated Java JARs will be on your CLASSPATH.  The ANT command, if successful, results in a tarball file created in the export directory.  This is the plug-in.  In our case this file is export/node_js-1.0.0.0.tgz.  The version is controlled by the value in the plugin/config.json.  Changing that will result in a new version for the plug-in.

Deploying and testing the Node.js plugin

Now that we’ve created our Node.js plug-in and also discussed how to build the plug-in, one thing remains: how to use it?  In this section we aim to answer this question by first describing how to install the plug-in into an IBM PureApplication System and use the plug-in to create a simple Node.js pattern.  From there we will test the plug-in by deploying the pattern.

Adding plug-in to IBM PureApplication System

Once your plug-in is built the next step, before you can create patterns to test and use it, is to deploy the plug-in into an IBM PureApplication System.  This can be achieved in one of two ways:

1.If you have access to the IBM PureApplication System controller node via SSH then you can add

and remove plug-in by executing the shell scripts: /opt/IBM//plugin_add.sh and /opt/IBM//plugin_remove.sh.  The add_plugin command takes as argument the tarball file of the plug-in.  For the remove_plugin command you pass the name and version of the plug-in to remove.

2.Alternatively, there is a web UI interface to manage plug-ins.  This can be accessed in the admin section of the IBM PureApplication System dashboard.  The plug-ins are listed and new plug-ins are added by using the web interface and uploading the tarball file for the plug-in.

When the plug-in is successfully added to the IBM PureApplication System plug-in catalog it will then show up in the tool palette on the left hand side when you attempt to create a new pattern or edit an existing pattern.

Creating a Node.js pattern

Now that the plug-in is installed the next step is to create a simple pattern to deploy Node.js application.  Doing so is simple using the drag-and-drop interface of the IBM PureApplication System pattern designer application.  You launch the application clicking on Pattern -> Virtual Applications -> New (+ button) and selecting the “Blank Application” option so that the pattern starts with a blank sheet.

Once the pattern builder application is loaded, you simply need to drag-and-drop the Node.js cloud component from the left hand side list to the canvas.  Selecting the component shows its stylesheet.  There you can select the default Node.js version to use as well as that NPM should be run to download dependencies.  Save the resulting as a pattern giving it the name: Node.js pattern.

Testing a non-trivial Node application

Testing this new pattern is relatively easy.  You now need to create a new cloud application using the Node.js pattern which shows up in the list of available patterns.  What remains is to give the application a name and point to the Git repository.  For testing purposes we will deploy the Nodejistsu Prenup, a collaborative Behavior Driven Development (BDD) application found on Github.com at: https://github.com/nodejitsu/prenup.

Prenup is a pure Node.js OSS application that facilitates Behavior-Driven Development (BDD) by creating a collaborative engagement between developers and clients.  Once the cloud application is saved, it will show up in the list of cloud applications.  All that remains is to click deploy.  Once deployed, you can then access the details of the deployment, including the status of the VM, its log files and also a link to the deployed application.  From there you can manage the deployment, such as stopping or deleting it.

What Next?

In this post we completed the Node.js plug-in that we designed in the first post of this series.  In doing so we covered in detail aspects of the plug-in and also showed how to build and install this plug-in in an IBM PureApplication System.  Using the plug-in we created a simple Node.js pattern and used it to deploy the OSSNodejistsu Prenup Node.js  app on Github.com.

Next we will complete this series by discussing what can go wrong in the process so far and give hints for debugging and testing your plug-ins.  Additionally, we will also briefly highlight some advanced features, such as quality of service (QoS) and link plug-ins which can be used to extend this current plug-in and make it more functional, for instance, using a link plug-in to create a connection between Node.js and a database or adding scaling QoS properties to the current plug-in.

Node.js meet IBM PureApplication System – Part 1 of 3

This is a re-post from the on on the Expert integrated systems blog.  Please go there for discussions and feedback.

In this three-part series I will explore how you can create IBM PureApplication Systems patterns for the “hot” open source web technology: Node.js.  In doing so, I will revisit the basics and the details behind creating PureApplication System plug-ins (extensions) as well as how to use them to create cloud application patterns.

The paramount goal of this series is to create more example plug-ins for the PureApplication System and contribute to the IBM PureSystem’s growing ecosystem.  Essentially, this is reinforcing the points we have been making in this blog about the openness of IBM PureSystems, its extensibility, its ease of use, and simply how it is transforming enterprise IT into the cloud.

Extending IBM PureApplication Systems

The IBM PureApplication System is the platform as a service (PaaS) platform for the IBM PureSystems family of products.  As I mentioned in a previous post (Should you virtualize your pattern or not?) PureApplication System has two means to define workloads: virtual applications (vApp) and virtual systems (vSys).  Each model has a set of extensibility points.  In this post I focus on vApp since that is the one with the most flexibility.

To create a vApp pattern one has to reuse or create plug-ins that represent the cloud components, cloud services, and the links constituting the pattern.  The IBM PureApplication System comes with a multitude of plug-ins supporting the patterns that are available right of the box, such as J2EE applications, web-based applications, transactional applications, and other enterprise-oriented patterns.  There is also a growing ecosystem of patterns available in the IBM PureSystems Marketplace, where third party business partners have created plug-ins for additional components and made them available.

For this post we will focus on creating a plug-in for Node.js to support typical patterns of deployments of this new web application stack.  This will be done in three parts and will cover all aspects of the designing, implementing, building and deploying plug-ins as well as using them to create patterns, and of course using the resulting patterns.  With that, let’s jump right into the details, starting with a refresher overview of vApp plug-ins and then an overview of Node.js so we can have a good idea of what type of patterns we want to create, which will help define the design goals for our plug-in.

Anatomy of a vApp plug-in

In the IBM PureApplication System, patterns are composed (visually) using cloud components, cloud services, and links.  Cloud components represent a part of a software stack, such as the WebSphere Application Server, the DB2 database, or the Ruby on Rails application server, and others.  A cloud service is a component that is shared across deployments, for instance, the IBM MQ service is a cloud service that you can add to your pattern.  Finally, links represent connections among cloud components and between cloud components and cloud services.

Creating a new plug-in amounts to deciding where it logically fits into the categorization above.  Sometimes this means creating multiple plug-ins to address the fact that a component might need to be connected to other components; in this case, you would create a link plug-in to connect the components.

Once decided, the next important step is to decide the attributes that the component (or link) exposes and the type and values that each attribute can take.  For instance, for a Node.js component, a sensible set of initial attributes might be:

• the name of the application
• the version of Node.js to use
• the URL to the git repository containing the Node.js application

Naturally, many more attributes are possible; however, these three make for an easy start.

The IBM PureApplication System plug-in model is very systematic and uses a strict structure for all aspects of the plug-in.  In short, the structure captures the following four key items:

• metadata.json – meta data information about the plug-in, such as: name, description, help files, image files, and the list of attributes exposed along with the attributes’ types.
• config.json – contains additional configuration information to optionally give hints to the PureApplication System deployment process.
• scripts – contains the scripts that will configure and install the cloud component that this plug-in represents.  Additional management scripts include those to stop, restart, and so on.
• files – all supporting files for the cloud component, for instance, binary installation files and configuration files.

The plug-in’s UI is represented by the icon image set in the metadata.json.  The IBM PureApplication System pattern designer web application also automatically creates the appropriate remaining UI for the plug-in using the meta data information.  This includes providing property sheets for the plug-in’s attributes as well as showing help files and enabling policies and links, if these are enabled and loaded.

The final part of a plug-in are the optional links and policies.  Links represent the connections between a cloud component and another cloud component or a cloud component and a cloud service.  When a cloud component supports a link then it is added to its metadata.  Similarly, quality of service (QoS) policies are used to provide non-functional capabilities to a cloud component.  For instance, specifying how a cloud component scales by horizontal replications or by growing its instance resources, such as CPU or memory, vertically.

Links and policies constitute advanced features of the IBM PureApplication System plug-in architecture and I may discuss them in a future post.  For now let’s focus on getting Node.js and the IBM PureApplication System acquainted.

Introduction to Node.js

Node.js is the “new hotness” in web application development.  But what makes Node.js interesting to web developers and what is the craze all about? And how does it compare to mature platforms such as JavaEE and Ruby on Rails?

Implementing an application server, in its simplest form, amounts to implementing a daemon process that executes client requests as they arrive.  The requests need to be parsed, routed to appropriate services, executed, and responded to.  The faster this cycle can be achieved, the more requests can be managed, and thereby the better performant is the resulting application server.

Naturally, requests sometimes fail and also need to be secured and isolated.  These complications, and others, imply that designing an application server usually


results in using OS-level resources that are designed for concurrency control while providing some level of security and isolation.  As such, most application servers use OS processes or more frequently OS threads as a basis for their core implementation.  Such is the architectures of most Java-based application servers.

However, while OS processes and threads make it easy to create heavily concurrent applications and servers, they also have drawbacks.  Primarily, thread-based application servers (and even more so for process-based servers) tend to be heavyweight, that is, they require lots of system resources: CPU, memory, and storage.  This can become a significant issue when running application servers that have to deal with significant number of concurrent requests or when these requests last a long time or perform tricky computation, such as complicated database queries.

Node.js is designed to deal specifically with this “heavyweight” implementation consequence of threads-based servers.  The distinguishing characteristic of the Node.js application server, which gives it its “lightweight” implementation claim is that instead of using OS-level mechanism to deal with multiple requests, it instead uses application-level callbacks and non-blocking IO libraries.  The JavaScript language uses callbacks as an essential mechanism for modularization and extension, and Node.js, as it is implemented in JavaScript, uses the same callback mechanism.

Another interesting aspect of Node.js is that it embeds the highly performing Google V8 JavaScript engine.  The V8 engine has revolutionized JavaScript virtual machines by utilizing advanced techniques pioneered in other interpreted languages such as Smalltalk and Self. Some of this includes aggressive optimization like generational incremental garbage collector, Just in Time compilation (JIT), and inline caching of functions. The result is a virtual machine that makes JavaScript a worthy competitor to other interpreted languages and even (in some cases) compiled languages.

Additional benefits of Node.js are that as web application programming interfaces (APIs) moved to using JavaScript Object Notation (JSON) as their primary data interchange format, most web services now include a back end that exposes a JSON-based API with a front end that uses the JSON data to create the user interface.  Creating the application using Node.js allows both the back end and front end to be in the same language, thus potentially simplifying the codebase and associated assets, such as, data validators, tests code, tests data, and so on.

Node.js plugin goals

The primary goal for our Node.js PureApplication System plug-in is to allow Node.js applications to be simply and quickly installed, ran, and managed on an IBM PureApplication System setup.  We will assume that the Node.js application is located on a Git repository, such as Github.com, and that it uses N
ode 
Package Manager (NPM) to manage dependencies. The latest version of Node.js will be added into the plug-in and with a version attribute, a different version can be selected and used instead.

To keep the plug-in simple, we will not deal with typical scaling features of Node.js applications such as using aNginx HTTP/reverse proxy server, a load balancing service, or data caching service
s
.  These additional features can be added to this current plug-in in the future and are left as an exercise to the reader.

What next?

In part 2 of this post, I will complete the Node.js plug-in and deploy a non-trivial open source Node.js application found on Github.com.  I will also discuss how you can build the plug-in and install it into an IBM PureApplication System setup.