Experiences with Node.js: Porting a RESTful Service Written in Java

This is the second post in a series of 3 about my experiences with Node.js. In this post I will discuss my experiences porting a RESTful service from Java and some basic benchmarks, comparing the Node.js service and the Java service.

If you haven’t read the first article in this series, you can view it here: Experiences with Node.js: Researching Node. My final post in this series Experiences with Node.js: Final Thoughts is now available, where I summarize my experiences, provide some feedback, and discuss my next steps with Node.js.

Since the writing of the first article in this series, there have been a few new releases to the Node.js project. You can read about these updates on the Node Blog.

Porting a Java Project

The ultimate goal in this Node.js research is to successfully port an existing RESTful service written in Java using Jersey (JAX-RS) running on Tomcat to JavaScript running on Node.js. Not only could the experience be gained of building a useful service with Node, but some basic performance testing could gain some insights to its power and the results compared. If things go well here, more time and energy could be invested into Node.

The port will use the ExpressJS library as the base framework. ExpressJS is a great web application framework for Node.js. It is very powerful and offers some great plugin to extend its functionality. In this example, the Express Resource plugin is used to handle resource routing. This too is very convenient and simplified creating individual routes. The ORM layer uses Sequelize, a powerful ORM on top of MySQL.

Module installation

First, the modules need to be installed for use in this project. This example uses the Node Package Manager (NPM) to install the modules. The module files are copied to the local node_modules directory in the current path, making a convenient directory structure for packaging a project.

$ cd project
$ npm install express
$ npm install express-resource
$ npm install sequelize

$ cd project

$ npm install express

$ npm install express-resource

$ npm install sequelize

app.js

The app.js file contains the core instructions for the application. It defines the application, sets global variables, initializes the ORM, routes files, and starts our HTTP server. There are many good resources available on the web for creating this core file. This example uses a combination of many different techniques from multiple sources. It also includes added support for pulling in command line parameters. There are modules available that add helper methods for this, but it is pretty simple to handle without them.

// Module dependencies.
var express = require('express');

// Create Application Server
var app = module.exports = express.createServer();

// Set Route Init Path
var routes = "./routes.js";

// Set Database Init Path
var database = "./database.js";

/**
* app
* @type {Express}
*
* The Singleton of Express app instance
*/
GLOBAL.app = app;

/**
* Retrieve Command Line Arguments
* [0] process : String 'node'
* [1] app : void
* [2] port : Number 8010
*/
var args = process.argv;

/**
* port
* @type {Number}
*
* HTTP Server Port
*/
var port = args[2] ? args[2]: 8010;

// Database Connections

var database_options = {
    schema: "inventory",
    user: "inventory_user",
    password: "password1", //High Security FTW.
    host: "localhost",
    port: "3306"
};
// Configuration
app.configure(function(){
    app.set('views', __dirname + '/views');
    app.set('view engine', 'jade');
    app.use(express.bodyParser());
    app.use(express.cookieParser());
    app.use(express.methodOverride());
    app.use(app.router);
    app.use(express.static(__dirname + '/public'));
});

app.configure('development', function(){
    app.use(express.errorHandler({ dumpExceptions: true, showStack: true }));
    database_options.logging = true;
});
app.configure('production', function(){
    app.use(express.errorHandler());
    database_options.logging = false;
});

/**
 * db (database)
 * @type {Object}
 * @param Object [database_options] the database options
 */
GLOBAL.db = require(database)(database_options);

// Routes
var routes = require(routes);

// HTTP Server
app.listen(port);
console.log("Express server listening on port %d in %s mode using Sequelize ORM.", app.address().port, app.settings.env);

// Module dependencies.

var express = require('express');

// Create Application Server

var app = module.exports = express.createServer();

// Set Route Init Path

var routes = "./routes.js";

// Set Database Init Path

var database = "./database.js";

/**

* app

* @type {Express}

* The Singleton of Express app instance

GLOBAL.app = app;

/**

* Retrieve Command Line Arguments

* [0] process : String 'node'

* [1] app : void

* [2] port : Number 8010

var args = process.argv;

/**

* port

* @type {Number}

* HTTP Server Port

var port = args[2] ? args[2]: 8010;

// Database Connections

var database_options = {

schema: "inventory",

user: "inventory_user",

password: "password1", //High Security FTW.

host: "localhost",

port: "3306"

};

// Configuration

app.configure(function(){

app.set('views', __dirname + '/views');

app.set('view engine', 'jade');

app.use(express.bodyParser());

app.use(express.cookieParser());

app.use(express.methodOverride());

app.use(app.router);

app.use(express.static(__dirname + '/public'));

});

app.configure('development', function(){

app.use(express.errorHandler({ dumpExceptions: true, showStack: true }));

database_options.logging = true;

});

app.configure('production', function(){

app.use(express.errorHandler());

database_options.logging = false;

});

/**

* db (database)

* @type {Object}

* @param Object [database_options] the database options

GLOBAL.db = require(database)(database_options);

// Routes

var routes = require(routes);

// HTTP Server

app.listen(port);

console.log("Express server listening on port %d in %s mode using Sequelize ORM.", app.address().port, app.settings.env);

routes.js

This is the routing file that defines what URL’s will use which resource. Using the express-resource module, simple routes were created for the RESTful resources with very minimal lines of code. In this example, the URL is passed in and the Order resource module to the resource to define that relationship.

/**
 * Resource Routes
 */
module.exports = function() {
    var Resource = require('express-resource');
    app.resource('services/orders', require('./resource/Order'));
...
};

/**

* Resource Routes

module.exports = function() {

var Resource = require('express-resource');

app.resource('services/orders', require('./resource/Order'));

...

};

database.js

In the database.js file, the database object is created and a utility method for pushing the data from the model to an object is appended. After the ORM has created it’s database connection, the models can be built.

/**
 * Database Connection
 */
module.exports = function(options) {
    var database = {
        options: options
    };

    var Sequelize = require('sequelize');
    database.module = Sequelize;
    database.client = new Sequelize(options.schema, options.user, options.password, {
            host: options.host,
            port: options.port,
            logging: options.logging,
            dialect: 'mysql',
            maxConcurrentQueries: 100
        });

    /**
     * @type {Object}
     * Map all attributes of the registry
     * (Instance method useful to every sequelize Table)
     * @this {SequelizeRegistry}
     * @return {Object} All attributes in a Object.
     */
    database.map = function() {
        var obj = new Object(),
            ctx = this;
        ctx.attributes.forEach(function(attr) {
            obj[attr] = ctx[attr];
        });

        return obj;
    };

    database.models = require('./models.js')(database);

    return database;
};

/**

* Database Connection

module.exports = function(options) {

var database = {

options: options

};

var Sequelize = require('sequelize');

database.module = Sequelize;

database.client = new Sequelize(options.schema, options.user, options.password, {

host: options.host,

port: options.port,

logging: options.logging,

dialect: 'mysql',

maxConcurrentQueries: 100

});

/**

* @type {Object}

* Map all attributes of the registry

* (Instance method useful to every sequelize Table)

* @this {SequelizeRegistry}

* @return {Object} All attributes in a Object.

database.map = function() {

var obj = new Object(),

ctx = this;

ctx.attributes.forEach(function(attr) {

obj[attr] = ctx[attr];

});

return obj;

};

database.models = require('./models.js')(database);

return database;

};

models.js

Using the Sequelize ORM model structure, the individual model Java classes are converted to a singular JSON object that represented all the database tables with validations, constraints and relationships. In this models.js example, there is a mapAttributes is linked to the db.map function to return only the data in a specific model. For this example, the Order model is shown.

/**
 * Sequelize ORM Models
 */
module.exports = function(db) {
    /**
     * @type {Object}
     * All models we have defined over Sequelize, plus the db instance itself
     */
    var self = {
        Order: db.define('ORDERS', {
            ID: { type: Sequelize.STRING, allowNull: false, primaryKey: true, validate: { max: 36 } },
            TENANT_ID: { type: Sequelize.STRING, allowNull: false, validate: { max: 36 } },
            VERSION: { type: Sequelize.INTEGER, allowNull: false },
            CREATED: { type: Sequelize.INTEGER, defaultValue: null },
            CREATEDBY: { type: Sequelize.STRING, defaultValue: null, validate: { max: 50 } },
            MODIFIED: { type: Sequelize.INTEGER, defaultValue: null },
            MODIFIEDBY: { type: Sequelize.STRING, defaultValue: null, validate: { max: 50 } },
            LOCATION_ID: { type: Sequelize.STRING, allowNull: false, validate: { max: 36 } },
            SOURCE_ID: { type: Sequelize.STRING, allowNull: false, validate: { max: 36 } },
            TREATMENT_ID: {type: Sequelize.STRING, validate: { max: 36 } },
            PONUMBER: { type: Sequelize.STRING, defaultValue: '', validate: { max: 50 } },
            NOTES: { type: Sequelize.STRING, defaultValue: '', validate: { max: 255 } },
            CLOSED: { type: Sequelize.BOOLEAN, allowNull: false, defaultValue: false },
            CLOSEDON: { type: Sequelize.INTEGER, defaultValue: null },
            CLOSEDBY: { type: Sequelize.STRING, defaultValue: null, validate: { max: 50 } }
        },
        {
            timestamps: false,
            freezeTableName: true,
            instanceMethods: {
                mapAttributes: db.map
            }
        }),
        ...
    };

    return self;
};

/**

* Sequelize ORM Models

module.exports = function(db) {

/**

* @type {Object}

* All models we have defined over Sequelize, plus the db instance itself

var self = {

Order: db.define('ORDERS', {

ID: { type: Sequelize.STRING, allowNull: false, primaryKey: true, validate: { max: 36 } },

TENANT_ID: { type: Sequelize.STRING, allowNull: false, validate: { max: 36 } },

VERSION: { type: Sequelize.INTEGER, allowNull: false },

CREATED: { type: Sequelize.INTEGER, defaultValue: null },

CREATEDBY: { type: Sequelize.STRING, defaultValue: null, validate: { max: 50 } },

MODIFIED: { type: Sequelize.INTEGER, defaultValue: null },

MODIFIEDBY: { type: Sequelize.STRING, defaultValue: null, validate: { max: 50 } },

LOCATION_ID: { type: Sequelize.STRING, allowNull: false, validate: { max: 36 } },

SOURCE_ID: { type: Sequelize.STRING, allowNull: false, validate: { max: 36 } },

TREATMENT_ID: {type: Sequelize.STRING, validate: { max: 36 } },

PONUMBER: { type: Sequelize.STRING, defaultValue: '', validate: { max: 50 } },

NOTES: { type: Sequelize.STRING, defaultValue: '', validate: { max: 255 } },

CLOSED: { type: Sequelize.BOOLEAN, allowNull: false, defaultValue: false },

CLOSEDON: { type: Sequelize.INTEGER, defaultValue: null },

CLOSEDBY: { type: Sequelize.STRING, defaultValue: null, validate: { max: 50 } }

{

timestamps: false,

freezeTableName: true,

instanceMethods: {

mapAttributes: db.map

}

}),

...

};

return self;

};

resources/Order.js

A basic set of CRUD functions are constructed for the model and replicated it for each resource. The Sequelize ORM made this task very simple and allowed for querying the database with a few very simple function calls and callbacks. After querying the ORM, the success callback will call the mapAttributes function and return a JSON object to the response object. In the case of an error, the error callback function fires returning the error to the response object. For this example, the List and the Show methods are shown as they will be used for the first performance tests. Other methods can be created for a RESTful implementation. Documentation for the Express Resource module can be found on the project site.

var model = models.Order;

/*
 * List
 * @GET /model/
 * @returns JSON list of records
 */
exports.index = function(request, response) {
    model.findAll()
        .success(function(models) {
            response.json(models.map(function(record) {
                return record.mapAttributes();
            }));
        })
        .failure(function(error) {
            response.send(error);
        });
};

/*
 * Show by Id
 * @GET /model/:id
 * @returns JSON requested record
 */
exports.show = function(request, response) {
    model.find(request.params.order)
        .success(function(record) {
            response.json(record.mapAttributes());
        })
        .failure(function(error) {
            response.send(error);
        });
};

...

var model = models.Order;

* List

* @GET /model/

* @returns JSON list of records

exports.index = function(request, response) {

model.findAll()

.success(function(models) {

response.json(models.map(function(record) {

return record.mapAttributes();

}));

})

.failure(function(error) {

response.send(error);

});

};

* Show by Id

* @GET /model/:id

* @returns JSON requested record

exports.show = function(request, response) {

model.find(request.params.order)

.success(function(record) {

response.json(record.mapAttributes());

})

.failure(function(error) {

response.send(error);

});

};

...

After preparing the modules, it was time to fire up the server and see if there are any errors. Fortunately, effort was put into later versions of Node.js to support larger, more detailed stack traces for troubleshooting.

node app.js
Express server listening on port 8010 in development mode using Sequelize ORM.

1 2	node app.js Express server listening on port 8010 in development mode using Sequelize ORM.

The server fired up without issue because this code is awesome… (Ok, so in reality, there were issues, but they were worked out before writing this article, of course.)

Performance Testing

Now that there has been success in the goal of successfully creating a JavaScript port of the CRUD resources of a Java service, it is time to performance test and compare the results between the technologies. The test plan is simple and fairly unscientific: use JMeter to capture throughput results of Node.js’s single threaded, non-blocking I/O model against the traditional Java application on Tomcat. JMeter is running These tests are running locally from a MacBook Pro running 2.2GHz Intel Core i7, 8GB of 1333MHz DDR3 on OS X Lion 10.7.2.

The first tests against Node and the Tomcat server were hitting 7 of the resource’s list methods simultaneously with up to 128 users (threads), 50 times. The Node.js server and application were solid. Neither crashed or leak memory. The throughput and bites processed was consistent no matter how many users pegging the application. From a system resources standpoint, the Node process never topped 63MB of Memory, but a full processor core was consistently pegged at 100%. The Java Application also handled itself very well, but consumed a lot more system resources.

In these summaries, the standard deviation of the sample’s elapsed time, the total average requests per second (throughput), and the throughput in kilobytes per second have been recorded. The peak CPU and peak Memory usage are also noted during the tests.

JavaScript on Node.js

Users	Standard Deviation	Throughput	Bandwidth	Peak CPU	Peak Memory
16	15.480	212.531 req/sec	446.085 KB/sec	99.6%	57.9MB
32	21.798	211.456 req/sec	443.828 KB/sec	99.7%	58.7MB
48	30.709	208.421 req/sec	437.457 KB/sec	99.6%	59.0MB
64	49.849	205.131 req/sec	430.553 KB/sec	99.7%	59.7MB
128	98.737	218.719 req/sec	406.021 KB/sec	99.5%	61.0MB

Java on Tomcat

Users	Standard Deviation	Throughput	Bandwidth	Peak CPU	Peak Memory
16	120.065	502.197 req/sec	1245.193 KB/sec	135.5%	270.5MB
32	189.948	642.680 req/sec	1593.522 KB/sec	206.6%	271.1MB
48	280.655	685.770 req/sec	1700.362 KB/sec	278.4%	276.3MB
64	286.626	671.825 req/sec	1665.785 KB/sec	340.2%	279.0MB
128	320.229	526.198 req/sec	1304.609 KB/sec	582.7%	304.1MB

The results here are shocking. There was a dramatic difference between the Node.js service and the Java service. At the very least, the Node application should have performed almost as well as the Java version. There had to be something wrong here.

After more research and swapping out modules, the bottleneck was narrowed down to the Sequelize ORM. A direct correlation was identified between the number of records and the throughput from the resource. By increasing the number of records in a table, it exponentially decreased the throughput of data when using Sequelize. Although no research was put in to this, based on this finding it could be assumed the problems were in the logic that create the model objects. The models and database code were rewritten using the node-mysql library by writing select statements and the tests were ran again using the new code. The results were much better using node-mysql and the system resources were also lower.

JavaScript on Node.js with node-mysql

Users	Standard Deviation	Throughput	Bandwidth	Peak CPU	Peak Memory
16	94.879	525.624 req/sec	1089.159 KB/sec	81.0%	53.8MB
32	159.954	686.779 req/sec	1423.093 KB/sec	83.0%	54.1MB
48	210.742	769.301 req/sec	1594.089 KB/sec	83.3%	54.6MB
64	197.575	703.849 req/sec	1458.464 KB/sec	86.0%	55.7MB
128	208.428	562.785 req/sec	1166.163 KB/sec	89.2%	57.1MB

In the end, the results were better by dropping Sequelize and using node-mysql. The throughput was pretty much on-par with the Tomcat server, consuming much less resources. There may be some amazing opportunities for optimization here that could easily push the Node.js performance beyond Tomcat’s. The results reasonably show that Node.js can perform as well as Java in this scenario.

Be sure to visit Experiences with Node.js: Final Thoughts, where I summarize my experiences, provide some feedback, and discuss my next steps with Node.js.

Trackbacks/Pingbacks

exemples / implémentations | Pearltrees - [...] Experiences with Node.js: Porting a RESTful Service Written in Java - ZiggyTech app . use ( express . errorHandler…
Experiences with Node.js: Final Thoughts - ZiggyTech - [...] This is the final post in a series of 3 about my first experiences with Node.js. In this post…
Experiences with Node.js: Porting a RESTful Service Written in Java « async I/O News - [...] Read full article Tagged as: Java, JavaScript, Node.JS, REST Comments Off Comments (0) Trackbacks…
Experiences with Node.js: Final Thoughts « async I/O News - [...] benchmarks I completed in the previous post put a single threaded Node.js up against a multi-threaded Java service and the results…

Experiences with Node.js: Porting a RESTful Service Written in Java

Porting a Java Project

Module installation

app.js

routes.js

database.js

models.js

resources/Order.js

Performance Testing

JavaScript on Node.js

Java on Tomcat

JavaScript on Node.js with node-mysql

Trackbacks/Pingbacks

Leave a reply Cancel reply

About Me

Strava

Instagram

Recent Tweets

Tag cloud

Pin It on Pinterest

Experiences with Node.js: Porting a RESTful Service Written in Java

Porting a Java Project

Module installation

app.js

routes.js

database.js

models.js

resources/Order.js

Performance Testing

JavaScript on Node.js

Java on Tomcat

JavaScript on Node.js with node-mysql

Related Posts

Ace Your Face!

Experiences with Node.js: Researching Node

How Big is TOO BIG for JSON?

Experiences with Node.js: Final Thoughts

Trackbacks/Pingbacks

Leave a reply Cancel reply

About Me

Strava

Instagram

Recent Tweets

Tag cloud

Pin It on Pinterest