Deploy your nodeJS/Express app to AppFog Paas

AppFog is a cloud based "platform as a service" (PaaS) that let you run your web applications, even on private clouds.
It supports different technologies and offers a lot of installable plugins or services. Therefore no matters if you have to deploy a Python, Django, Java, NodeJS, PHP app based on MongoDB, MySQL, Redis and so on.

Appfog does not force you to change your devel habits: you can develop locally your app as usual and then deploy it with just one command as we will see. The main philosophy is: "work on code, not management".

At this time appfog supports the following versions:

• Python (2.7.3)
• NodeJS (0.8.14, 0.6.17, 0.4.12)
• PHP (5.3.10)
• Ruby (1.8.7, 1.9.3, 1.9.2)
• Java (1.7.0)

You can choose the supported environment you prefer. Obviously for the most reliable development experience, make sure you have the same version on your local development environment.

Can't wait for trying AppFog?

Here it is the basic getting started playing with the console offline:

1. sign up to http://console.appfog.com/signup 
2. install the appfog af command line interface https://docs.appfog.com/getting-started/af-cli 

Now you can play as well with the AppFog console online as shown here https://docs.appfog.com/getting-started/jumpstarts but I suggest to have a look at the command line syntax.

All you need to know is covered in the docs area https://docs.appfog.com but we we'll see the most useful commands right now.

In the next sections we will consider a NodeJS / Express app as example, so let's start with NodeJS, Express and AppFog now.

NodeJS installation

Talking about NodeJS, you might use the Node Version Manager (NVM) to manage multiple Node.js versions as described in this article http://codetheory.in/using-node-version-manager-nvm-to-manage-multiple-node-js-versions/. Installing the right version of NodeJS is as simple as typing the command:

$ nvm install NODE_VERSION (for example: 0.8.14)

Create your app NodeJS/Express app

Now you can develop locally your app as usual with Node and Express.
If you want to create a hello world application you can start using the express executable or starting from scratch. Both of the above options are described in this guide http://expressjs.com/guide.html

Or if you prefer you can start with an app generated by one of the available yeoman's generators. You can see the full list of the yeoman's community generators: http://yeoman.io/community-generators.html.
You can use a specific yeoman's express generator or choosing another one that fits better your needs and then add the Express dependency by hand. For this post I'll use as reference the hello world app based on Yeoman/Node/Express/AngularJS/generator-angular shown in this previous post:

• http://davidemoro.blogspot.com/2013/08/yeoman-express-and-angularjs.html

Once created your NodeJS/Express app as usual you need to perform some small changes.

Make your app compatible with AppFog

It's quite simple.

Firstly make sure you have provided the start command under the scripts key in your package.json file, for example:

  "scripts": {
    "test": "grunt test",
    "start": "node app.js"
  }

If you don't provide the start command AppFog runs the first of the following files it finds:

• server.js
• app.js
• index.js
• main.js
• application.js

After that you need to change the listen port of your app: AppFog passes the listen port for your app in an environment variable, accessed by process.env.VCAP_APP_PORT. So depending on how it is organized your app.js, you need to change the listend port:

-app.set('port', process.env.PORT || 3000);
+app.set('port', process.env.VCAP_APP_PORT || 3000);

AppFog requires also a manifest file named manifest.yml, but don't worry: the first time you push your app there will be an interactive prompt as you will see in the next section.

Now you are ready to deploy your app to AppFog.

Deploy your app to AppFog

Since AppFog supports npm shrinkwrap (see https://npmjs.org/doc/shrinkwrap.html for more details), before pushing your app you might want to lock down the versions of your package's dependencies: this way you can control exactly which versions of each dependency will be used when your package is installed in the AppFog cloud.

It requires you just one command:

$ npm shrinkwrap
wrote npm-shrinkwrap.json

Now we are close to pushing our app. Basically you have to login using the credentials typed during the sign up process. It is very simple as you can see:

$ af login

Attempting login to [https://api.appfog.com]
Email: YOUREMAIL
Password: *********************
Successfully logged into [https://api.appfog.com]

Type the push command with the interactive prompt:

$ af push
Would you like to deploy from the current directory? [Yn]: Y
Application Name: YOURAPP
Detected a Node.js Application, is this correct? [Yn]: Y
1: AWS US East - Virginia
2: AWS EU West - Ireland
3: AWS Asia SE - Singapore
4: HP AZ 2 - Las Vegas
Select Infrastructure: 2
Application Deployed URL [YOURAPP.eu01.aws.af.cm]:
Memory reservation (128M, 256M, 512M, 1G, 2G) [64M]:
How many instances? [1]:
Create services to bind to 'YOURAPP'? [yN]: N
Would you like to save this configuration? [yN]: y
Manifest written to manifest.yml.
Creating Application: OK
Uploading Application:
  Checking for available resources: OK
  Processing resources: OK
  Packing application: OK
  Uploading (800K): OK 

Push Status: OK
Staging Application 'YOURAPP': OK                                         
Starting Application 'YOURAPP': OK

The generated manifest.yml file is similar to the following:

$ cat manifest.yml
---
applications:
  .:
    mem: 64M
    framework:
      info:
        description: NodeJS, Express, AngularJS Application
        exec:
        mem: 64M
      name: node
    instances: 1
    url: ${name}.${target-base}
    infra: eu-aws
    name: YOURAPP

If you need to do some future changes you should run the af update command instead.

Done, your app is online at http://YOURAPP.eu01.aws.af.cm!

If you have some tips or suggestions let me know please.

Additional notes

Errors during push or upload

Sometimes you may experiment this error trying to update or pushing your app:

HTTP exception: RestClient::ServerBrokeConnection:Server broke connection:

After a quick search on the net it seems a recurring problem and, if it happens, it may persist for a lot of hours or several days. Probably it is due to overload issues. I don't think it's related to a code problem because sometimes it works.

Fortunately it was only an annoying problem in my case because it was just a hello world app but... you can image if you are in a hurry and you have to update your business app what a problem!

AppFog guys, please let me know if I'm doing something of wrong!

Anyway if you experiment similar problems see https://www.appfog.com/support/

Update 20130825: the problem seems to be fixed for me using the --no-resource option (for example: af update --no-resource). This option is not listed at all in the af --help command. Now the update command works fine, good! Hope it might help other people this experience.
If you have other troubles you can visit the Node section of AppFog troubleshooting page https://docs.appfog.com/troubleshooting

Environment vars

You must notice that AppFog supports environment vars.

These will be accessible to your app at runtime. You can specify data that you do not want in your source code like passwords, API keys, or differentiate between staging and production environments.

Since also Express supports arbitrary environments, like production and development, you can use the configure method to set different configurations under the different environments.

For example in production mode you may use images, css and js minified, disabling devel settings, etc.

You can add environment vars through the web using the console online or via command line:

$ af env-add YOURAPP status=production
Adding Environment Variable [status=production]: OK
Stopping Application 'YOURAPP': OK
Staging Application 'YOURAPP': OK                                      
Starting Application 'YOURAPP': OK 

Update 20130825: you can get your env app with process.env.YOURENVVAR.

  

Quickstart

If you are not familiar with Node, Express, yeoman&co and you want to have a try, you can start from this hello world I presented in this previous post:

• http://davidemoro.blogspot.com/2013/08/yeoman-express-and-angularjs.html

You should perform these actions:

1. git clone git@github.com:davidemoro/express-angular.git
2. $ cd express-angular
3. change your manifest.yml and other files that reference the old app name (grep is your friend)
4. $ npm install
5. $ bower install
6. $ af login
7. $ af push

If you want to enable the production mode you should type:

1. $ grunt build
2. $ af update
3. $ af env-add YOURAPP status=production

Let me know if you find some problems.

Links

• https://www.appfog.com 
• https://docs.appfog.com 
• https://docs.appfog.com/getting-started 
• https://docs.appfog.com/languages/node
• http://davidemoro.blogspot.com/2013/08/yeoman-express-and-angularjs.html 
• https://github.com/davidemoro/express-angular

Yeoman, Express and AngularJS

I'm a great fan of yeoman: it provides for free a lot of goodies that let you easily bootstrap and managing an application. Tests setup, minification, dependencies manager, script automation, etc: using the yeoman workflow you can save a lot of time, being more productive and following the best devel practices.

I already used yeoman for AngularJS applications: now I want to build a NodeJS/Express application powered by AngularJS with all the goodies provided by yeoman.

And... before searching for and trying an existing express-angularjs yeoman generators I'd like to do it by hand, just for fun and for diving into NodeJS/grunt & co.

AngularJS setup with yo

Start with creating and empty dir as suggested on this StackOverflow question http://stackoverflow.com/questions/14106248/yeoman-inside-expressjs:

$ mkdir express-angular
$ cd express-angular

Then launch the angular generator:

$ yo angular --minsafe

If you want you can init this folder with git.

Express initialization

After that rename the package.json because it will be replaced soon by the express init command:

$ mv package.json package_yo.json

And init this folder with express:

$ express -e -f .

The -e option stands for using the ejs template engine, the -f will force the overwrite of existing files.

Now you can launch a diff on the package.json files and merge the differences. After that you can get rid of the temporary package_yo.json.
This way we won't lost the devDependencies and other setups that let your grunt routines works fine.

Now launch the following commands

$ npm install
$ node app -> run on port 3000
Express server listening on port 3000

If you open a browser on http://localhost:3000 you will see a working Express view.

But... this is not what we want, where are the view provided by the AngularJS generator? Wait a moment.

Express integration with AngularJS

Now let's tell Express to use the AngularJS views created by yo.
Firstly you can get rid of public and views dirs needed by Express because we will use the app folder generated by yo.

$ rm -fR public
$ rm -fR views

You have also to edit the app.js:

...
 // all environments
 app.set('port', process.env.PORT || 3000);
-app.set('views', __dirname + '/views');
 app.set('view engine', 'ejs');
+app.engine('html', require('ejs').renderFile);
 app.use(express.favicon());
 app.use(express.logger('dev'));
 app.use(express.bodyParser());
...

 app.use(express.methodOverride());
 app.use(app.router);
-app.use(express.static(path.join(__dirname, 'public')));

 // development only
 if ('development' == app.get('env')) {
+app.set('views', __dirname + '/app'); 

+  app.use(express.static(path.join(__dirname, 'app')));
   app.use(express.errorHandler());
 }
+else {

+app.set('views', __dirname + '/dist'); 

+  app.use(express.static(path.join(__dirname, 'dist')));
+}
...

Let me explain, we are doing the following things:

• get rid of the /views, we will use the app/ folder with the yeoman templates
• get rid of the public dir, we will use the app dir for serving the resources or the dist if you are on a production environment
• we are telling Express to map the EJS template engine to ".html" files. This way you can avoid to rename the app/index.html file. See http://expressjs.com/api.html#app.engine for more info

Now let's modify the app/index.html putting some EJS flavour.

...
   <head>
     <meta charset="utf-8">
     <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
-    <title></title>
+    <title><%= title %></title>
     <meta name="description" content="">
...

The title var will use what it's provided by routes/index.js.

Now we have to change the routes/index.js also in order to avoid problems (we are using index.html instead of index.ejs):

 exports.index = function(req, res){
-  res.render('index', { title: 'Express' });
-};
+  res.render('index.html', { title: 'Express AngularJS app' });
+};

And now if you re-run the "node app" you will see your Express-based angularJS application, so it works!

As you will see the work is not finished.

Fix grunt build

Well, but what about grunt tasks provided by yeoman? Let's try:

$ grunt

    Warning: Running "htmlmin:dist" (htmlmin) task
    File dist/404.html created.
    Warning: app/index.html
    Parse Error: <%= title %></title>

... ouch! What happened? The htmlmin task failed trying to parse the Ejs statements.

So, before finishing we have to apply a few changes to our Gruntfile.js:

    concurrent: {
      server: [
        'coffee:dist'
      ],
      test: [
        'coffee'
      ],
      dist: [
        'coffee',
        'imagemin',
-        'svgmin',
+        'svgmin'
+        //'htmlmin'
      ]
    },
...
    // Put files not handled in other tasks here
    copy: {
      dist: {
        files: [{
          expand: true,
          dot: true,
          cwd: '<%= yeoman.app %>',
          dest: '<%= yeoman.dist %>',
          src: [
            '*.{ico,png,txt}',
+            '*.html', 'views/*.html',
            '.htaccess',
            'bower_components/**/*',
            'images/{,*/}*.{gif,webp}',
            'styles/fonts/*'
          ]

In the changes above we just disabled the htmlmin task and add the *.html files stuff to the copy section.

Finished? Not yet!

Grunt server and livereload

Now the grunt command will work fine, even the build task. If you run "node app" all it's fine but running "grunt server" you may notice that the page title is not "Express AngularJS app" as expected but "<%= title %>".

What's the matter? The Gruntfile.js is not aware about Express so it needs some extra love and the grunt-express task can help you! See the grunt-express readme for more info https://github.com/blai/grunt-express.

Basically you'll have to do these things:

• install grunt-express with a "npm install grunt-express --save-dev"
• add a module.exports to your app.js. This is required by grunt-express if you want to avoid a "Fatal error: Server should provide a function called "listen" that acts as http.Server.listen"
• modify your Gruntfile.js replacing the connect/livereload stuff with the grunt-express task

This way the "grunt server" command will run Express with live reload enabled.
Here you can see the most significant part of Gruntfile.js, basically you'll have to comment the connect/livereload stuff and adding the following lines:

    express: {
      options: {
        port: 9000,
        // Change this to '0.0.0.0' to access the server from outside.
        hostname: 'localhost'
      },
      livereload: {
        options: {
          server: path.resolve('./app.js'),
          livereload: true,
          serverreload: false,
          bases: [path.resolve('./.tmp'), path.resolve(__dirname, yeomanConfig.app)]
        }
      },
      test: {
        options: {
          server: path.resolve('./app.js'),
          bases: [path.resolve('./.tmp'), path.resolve(__dirname, 'test')]
        }
      },
      dist: {
        options: {
          server: path.resolve('./app.js'),
          bases: path.resolve(__dirname, yeomanConfig.dist)
        }
      }
    },

For the full list of changes you can see this package as reference: https://github.com/davidemoro/express-angular. Note: I had to set the serverreload option to false because it didn't worked for me.

Done!

Here you can see the result:

Notes: I'm quite new to these arguments because I come from the Python, Zope and Plone world, so let me know if you have some hints!

Links

• http://stackoverflow.com/questions/14106248/yeoman-inside-expressjs
• https://github.com/blai/grunt-express
• https://github.com/blai/grunt-express-angular-example

Update 20143009

Similar articles I wrote afterwards:

• yeoman + Pyramid Python web framework kickstart project 
• Plone (CMS Python powered) + yeoman + Angularjs kickstart project