Showing posts with label mongodb. Show all posts
Showing posts with label mongodb. Show all posts

2014-04-07

yeoman generator mongodb interactive installer


In the previous post we have seen how to create your own yeoman generators using custom _.templateSettings rules.

However if you want to dive into yeoman generators you can play with generator-generator, have a look at the code of existing generators and... why not? Create your own whatever-you-want yeoman generator!

I chose to implement a simple generator, based on custom _.templateSettings seen in the previous post, that let you automate the mongodb installation.

Its name is generator-mongodb (https://github.com/davidemoro/generator-mongodb).

Useful or not useful, this is my first whatever-i-want-generator, made just for fun. There is no tests coverage (I got stuck on a problem but the solution is on his way) and I won't maintain it. So that's just an example, anyway I hope you'll find useful hints.

How it is implemented this generator under the hood

This generator is internally based on the buildout build system (http://www.buildout.org).

What is buildout? Buildout is a widely Python-based build system for creating, assembling and deploying applications from multiple parts, some of which may be non-Python-based. It lets you create a buildout configuration and reproduce the same software later.

Don't worry: we are not going to write any line of python code! We need python because it is required by our install system).

Buildout configurations are text-based, versionable, fine-grained reproducible environments that you can share with your team or create software installers (one example is the Plone CMS unified installer - see http://plone.org/products/plone).

There are dozens of published buildout recipes ready to be used. For example:
So with buildout you can enhance your productivity and automate your build process providind a light out of the box solution, instead of relying on virtual machines and huge downloads.

This helps when introducing multiple developers to a project: you can reduce setup times and enhance productivity. This way we can ensure that the whole team we are collaborating with has the right environment and you can save a lot of time.

Another example: if you are the creator of a nice open source project (for example a Node.js CMS based on MongoDB), you shoud consider that your download&install page will be visited by people just interested in evaluating a CMS or specific app solution without any knowledge or interest about the technology involved because they want to install and play with it locally. So you can provide simpler install instructions thanks to buildout (or any other automated system), for example you can replace "be sure you have installed node.js version x.y.z and MongoDB" with "checkout this folder and run this command: done!".

If you don't provide similar easy way to install your solution you are assuming that your users are comfortable with the underlying technologies, they know how to install the external components you need to run your software.  - If you want to explore a different approach you can have a look at http://www.vagrantup.com/ with its virtual environments.

Let's come back to the buildout. Buildout files can be extended, we can build different profiles (production, development, etc) and a very simple buildout.cfg file looks like the following:
[buildout]
parts = mongodb

[mongodb]
recipe = rod.recipe.mongodb
darwin-32bit-url = http://downloads.mongodb.org/osx/mongodb-osx-i386-1.6.5.tgz
...
How it works? Just one command and the buildout process starts, for example:
$ ./bin/buildout [-c buildout.cfg]
So thanks to buildout and existing recipes our generator workflow will be very simple:
  1. read one or more user inputs (version of mongodb, but you can add more advanced options if you want)
  2. render the buildout.cfg following the steps described here http://davidemoro.blogspot.it/2014/03/yeoman-generator-template-settings-lodash-underscore-js.html
  3. setup the right environment (it creates a local and isolated python environment that does not pollute the global python installed on your system)
  4. runs the buildout command for you
At the end of the process you'll find a local environment with all you need. Since each buildout folder is an isolated, self-contained environment, you can install also different MongoDB versions on your machine as well.

For advanced option you can see the rod.recipe.mongodb reference: https://pypi.python.org/pypi/rod.recipe.mongodb.

How to use generator-mongodb

Be sure you have installed node.js (see https://github.com/creationix/nvm), python and virtualenv (>= 1.11.4).

If you are not comfortable with python you can install it with a couple of commands:
$ sudo apt-get install python2.7
$ sudo apt-get install python-pip
$ [update 20140410:  not safe, follow the official doc] sudo pip install --upgrade virtualenv
The install yo:
$ npm install -g yo
If generator-mongodb were released, installing it it would as simple as typing:
$ npm install -g generator-mongodb
Since this package is not released, you'll have to clone the github repository:
$ git clone git@github.com:davidemoro/generator-mongodb.git
$ cd generator-mongodb
$ npm install
$ npm link
$ cd
Then move where you want to install your local mongodb environment and type:
$ mkdir yourmongodir
$ cd yourmongodir
$ yo mongodb
And you'll see something of similar.

Type the mongodb version and wait for the end of the install process (it might take a while).

Once finished you'll find the mongodb executables:
$ ls bin -1 | grep mongo
mongo
mongod
...
Update 20140415: actually ./bin/mongod doesn't work because I missed to set the db path option in buildout.cfg. Just type mkdir -p data/db in your buildout root and launch ./bin/mongod --dbpath data/db. If you configure well the mongodb recipe it will work without typing any option.

Done!

Links


2014-03-28

yeoman-generator's template() now accept a forth parameter being the _.templateSettings

We are talking about yeoman generators with custom _templateSettings rules.

Sometimes when you are using underscore.js or lodash templates you have to change how parameters and expressions will be interpolated.

You might want to do so because the syntax of your generated file does not fit the templateSettings defaults (example: you get a lodash/underscore template error like "SyntaxError: Unexpected token :") or you just want to use another syntax to interpolate vars or expressions.

How can you do that with yeoman?

Starting from 0.17.0-pre.1 prerelease version the yeoman-generator's template() accept a forth parameter being the _.templateSettings (see https://github.com/yeoman/generator/releases/tag/v0.17.0-pre.1).

Here you can find an example about how to use custom _.templateSettings rules in a custom yeoman generator created with generator-generator (see http://yeoman.io/generators.html).

Note well: this post won't talk about how to create a custom generator. You can find more information about this topic following the official yeoman guide. Anyway installing generator-generator is as simple as typing the following command:
$ npm install -g yo generator-generator
(after doing that you can have a coffee and/or brush your cat’s fur since it might requires a long time depending on your network or machine)

So let's see how it works!

app/index.js

The template method usage is similar to the older versions of yeoman-generator except a richer signature:
function template(source, destination, data, options)
where the fourth parameter is an object to be used as _.templateSettings. For example here it is a working example:
this.template('_buildout.cfg',
              'buildout.cfg',
               this,
    {
        evaluate: /\{\{([\s\S]+?)\}\}/g,
        interpolate: /\{\{=([\s\S]+?)\}\}/g,
        escape: /\{\{-([\s\S]+?)\}\}/g
    });

app/templates/_buildout.cfg

This is our _buildout.cfg template, you can populate values using the {{}} syntax. In this particular case we are building the download url of mongodb depending on the user input (mongodbVersion). For example:
[buildout]
...
# Do not remove this: <%= mongodbVersion %>
recipe = rod.recipe.mongodb
darwin-32bit-url = http://downloads.mongodb.org/...-{{=mongodbVersion}}.tgz
You might wonder why there is still a line with <%= mongodbVersion %>. It seems to be necessary because without this line this prerelease version of yeoman-generator will produce the unrendered output. See https://github.com/yeoman/generator/issues/517.

With this workaround all works fine, probably it won't be necessary in future versions.

Example usage


Results

The resulting buildout.cfg contents will be similar to the following lines, depending on the user input:
[buildout]
...
[mongodb]
...
# Do not remove this: <%= mongodbVersion %>
recipe = rod.recipe.mongodb
darwin-32bit-url = http://downloads.mongodb.org/osx/mongodb-osx-i386-2.4.9.tgz
The rendered file will be used to install and configure automatically the given version of mongodb.

Done

First of all: feedback, please!

And... I think to write a second part of this blog post talking about how this particular generator works under the hood (a yeoman based mongodb installer based internally on a widely used build system for creating, assembling, configuring and deploying applications). So if you are pretty curious stay tuned on @davidemoro for future write ups.

20140407: see http://davidemoro.blogspot.com/2014/04/yeoman-generator-mongodb-installer.html fur further information.