yeoman generator mongodb interactive installer

In the previous post we have seen how to create your own yeoman generators using custom _.templateSettings rules.

However if you want to dive into yeoman generators you can play with generator-generator, have a look at the code of existing generators and... why not? Create your own whatever-you-want yeoman generator!

I chose to implement a simple generator, based on custom _.templateSettings seen in the previous post, that let you automate the mongodb installation.

Its name is generator-mongodb (https://github.com/davidemoro/generator-mongodb).

Useful or not useful, this is my first whatever-i-want-generator, made just for fun. There is no tests coverage (I got stuck on a problem but the solution is on his way) and I won't maintain it. So that's just an example, anyway I hope you'll find useful hints.

How it is implemented this generator under the hood

This generator is internally based on the buildout build system (http://www.buildout.org).

What is buildout? Buildout is a widely Python-based build system for creating, assembling and deploying applications from multiple parts, some of which may be non-Python-based. It lets you create a buildout configuration and reproduce the same software later.

Don't worry: we are not going to write any line of python code! We need python because it is required by our install system).

Buildout configurations are text-based, versionable, fine-grained reproducible environments that you can share with your team or create software installers (one example is the Plone CMS unified installer - see http://plone.org/products/plone).

There are dozens of published buildout recipes ready to be used. For example:

• compiling and installing source distribution - https://pypi.python.org/pypi/hexagonit.recipe.cmmi
• generate a text file from a template based on configurable parameters-http://pypi.python.org/pypi/collective.recipe.template
• define maintenance cron jobs to be started periodically with no root privileges (normal users) - https://pypi.python.org/pypi/z3c.recipe.usercrontab
• downloading and installing MongoDB - https://pypi.python.org/pypi/rod.recipe.mongodb
• etc

So with buildout you can enhance your productivity and automate your build process providind a light out of the box solution, instead of relying on virtual machines and huge downloads.

This helps when introducing multiple developers to a project: you can reduce setup times and enhance productivity. This way we can ensure that the whole team we are collaborating with has the right environment and you can save a lot of time.

Another example: if you are the creator of a nice open source project (for example a Node.js CMS based on MongoDB), you shoud consider that your download&install page will be visited by people just interested in evaluating a CMS or specific app solution without any knowledge or interest about the technology involved because they want to install and play with it locally. So you can provide simpler install instructions thanks to buildout (or any other automated system), for example you can replace "be sure you have installed node.js version x.y.z and MongoDB" with "checkout this folder and run this command: done!".

If you don't provide similar easy way to install your solution you are assuming that your users are comfortable with the underlying technologies, they know how to install the external components you need to run your software.  - If you want to explore a different approach you can have a look at http://www.vagrantup.com/ with its virtual environments.

Let's come back to the buildout. Buildout files can be extended, we can build different profiles (production, development, etc) and a very simple buildout.cfg file looks like the following:

[buildout]
parts = mongodb

[mongodb]
recipe = rod.recipe.mongodb
darwin-32bit-url = http://downloads.mongodb.org/osx/mongodb-osx-i386-1.6.5.tgz
...

How it works? Just one command and the buildout process starts, for example:

$ ./bin/buildout [-c buildout.cfg]

So thanks to buildout and existing recipes our generator workflow will be very simple:

1. read one or more user inputs (version of mongodb, but you can add more advanced options if you want)
2. render the buildout.cfg following the steps described here http://davidemoro.blogspot.it/2014/03/yeoman-generator-template-settings-lodash-underscore-js.html
3. setup the right environment (it creates a local and isolated python environment that does not pollute the global python installed on your system)
4. runs the buildout command for you

At the end of the process you'll find a local environment with all you need. Since each buildout folder is an isolated, self-contained environment, you can install also different MongoDB versions on your machine as well.

For advanced option you can see the rod.recipe.mongodb reference: https://pypi.python.org/pypi/rod.recipe.mongodb.

How to use generator-mongodb

Be sure you have installed node.js (see https://github.com/creationix/nvm), python and virtualenv (>= 1.11.4).

If you are not comfortable with python you can install it with a couple of commands:

$ sudo apt-get install python2.7
$ sudo apt-get install python-pip
$ [update 20140410:  not safe, follow the official doc] sudo pip install --upgrade virtualenv

The install yo:

$ npm install -g yo

If generator-mongodb were released, installing it it would as simple as typing:

$ npm install -g generator-mongodb

Since this package is not released, you'll have to clone the github repository:

$ git clone git@github.com:davidemoro/generator-mongodb.git
$ cd generator-mongodb
$ npm install
$ npm link
$ cd

Then move where you want to install your local mongodb environment and type:

$ mkdir yourmongodir
$ cd yourmongodir
$ yo mongodb

And you'll see something of similar.

Type the mongodb version and wait for the end of the install process (it might take a while).

Once finished you'll find the mongodb executables:

$ ls bin -1 | grep mongo
mongo
mongod
...

Update 20140415: actually ./bin/mongod doesn't work because I missed to set the db path option in buildout.cfg. Just type mkdir -p data/db in your buildout root and launch ./bin/mongod --dbpath data/db. If you configure well the mongodb recipe it will work without typing any option.

Done!

Links

• https://github.com/davidemoro/generator-mongodb 

yeoman-generator's template() now accept a forth parameter being the _.templateSettings

We are talking about yeoman generators with custom _templateSettings rules.

Sometimes when you are using underscore.js or lodash templates you have to change how parameters and expressions will be interpolated.

You might want to do so because the syntax of your generated file does not fit the templateSettings defaults (example: you get a lodash/underscore template error like "SyntaxError: Unexpected token :") or you just want to use another syntax to interpolate vars or expressions.

How can you do that with yeoman?

Starting from 0.17.0-pre.1 prerelease version the yeoman-generator's template() accept a forth parameter being the _.templateSettings (see https://github.com/yeoman/generator/releases/tag/v0.17.0-pre.1).

Here you can find an example about how to use custom _.templateSettings rules in a custom yeoman generator created with generator-generator (see http://yeoman.io/generators.html).

Note well: this post won't talk about how to create a custom generator. You can find more information about this topic following the official yeoman guide. Anyway installing generator-generator is as simple as typing the following command:

$ npm install -g yo generator-generator

(after doing that you can have a coffee and/or brush your cat’s fur since it might requires a long time depending on your network or machine)

So let's see how it works!

app/index.js

The template method usage is similar to the older versions of yeoman-generator except a richer signature:

function template(source, destination, data, options)

where the fourth parameter is an object to be used as _.templateSettings. For example here it is a working example:

this.template('_buildout.cfg',
              'buildout.cfg',
               this,
    {
        evaluate: /\{\{([\s\S]+?)\}\}/g,
        interpolate: /\{\{=([\s\S]+?)\}\}/g,
        escape: /\{\{-([\s\S]+?)\}\}/g
    });

app/templates/_buildout.cfg

This is our _buildout.cfg template, you can populate values using the {{}} syntax. In this particular case we are building the download url of mongodb depending on the user input (mongodbVersion). For example:

[buildout]
...
# Do not remove this: <%= mongodbVersion %>
recipe = rod.recipe.mongodb
darwin-32bit-url = http://downloads.mongodb.org/...-{{=mongodbVersion}}.tgz

You might wonder why there is still a line with <%= mongodbVersion %>. It seems to be necessary because without this line this prerelease version of yeoman-generator will produce the unrendered output. See https://github.com/yeoman/generator/issues/517.

With this workaround all works fine, probably it won't be necessary in future versions.

Example usage

Results

The resulting buildout.cfg contents will be similar to the following lines, depending on the user input:

[buildout]
...
[mongodb]
...
# Do not remove this: <%= mongodbVersion %>
recipe = rod.recipe.mongodb

darwin-32bit-url = http://downloads.mongodb.org/osx/mongodb-osx-i386-2.4.9.tgz

The rendered file will be used to install and configure automatically the given version of mongodb.

Done

First of all: feedback, please!

And... I think to write a second part of this blog post talking about how this particular generator works under the hood (a yeoman based mongodb installer based internally on a widely used build system for creating, assembling, configuring and deploying applications). So if you are pretty curious stay tuned on @davidemoro for future write ups.

20140407: see http://davidemoro.blogspot.com/2014/04/yeoman-generator-mongodb-installer.html fur further information.

travis-lint, how to check your .travis.yml file

If you are a developer probably you already know the travis-ci.org continuous integration service and this kind of image status:

Adding CI support to your software hosted on github.com is very easy:

1. add a file named .travis.yml to your repository, it's a YAML configuration file
2. got to travis-ci and enable CI for your repo

You you need more details check the official documentation: this blog post is not focused on how to enable travis or what is CI.

The question is: how you can check the .travis.yml syntax? And deprecations or possible issues without having to lose time with failed builds?
The answer is travis-lint.

travis-lint

travis-lint is a tool that check your .travis.yml for possible issues, deprecations and so on. It's recommended for all travis-ci.org users.

How to install travis-lint

If you have already installed the gem utility it requires you just one command:

gem install travis-lint

travis-lint online

Too lazy for locally installing travis-lint? Go to the Travis WebLint service:

• http://lint.travis-ci.org/

More links

• http://en.wikipedia.org/wiki/Travis_CI
• https://github.com/travis-ci/travis-lint
• http://docs.travis-ci.com/user/travis-lint/
• https://www.npmjs.org/package/grunt-travis

Testing AngularJS directives with isolate scope (.isolateScope() is your friend)

In the previous post we have seen how to test directives with templateUrl. 

Now we are talking about how to test AngularJS directives with an isolated scope.

Let's say that you have a directive that should help you to select items from a given long list with filtering, ordering and selecting facilities:

<my-list model="exampleList" />

where exampleList is the following:

$scope.exampleList = [

    {id: '001', label:'First item', selected:false},

    {id: '002', label:'Second item', selected: false},

     ...

  ];

It sounds like a complex thing but you can implement it with Angular with few lines of Javascript code (app/scripts/directives/mylist.js): 
        'use strict';

angular.module('myListApp')

  .directive('myList', [function () {

    return {

      templateUrl: 'views/mylist.html',

      restrict: 'E',

      replace: true,

      scope: {model: '='},

      controller: ['$scope', '$element', '$attrs', function ($scope, $element, $attrs) {

        ... other logics       }]

    };

  }]);

with this directive's template (app/views/mylist.html):

<div>
  <div class="input-group filtercontrols">
    <span class="input-group-addon">Filter</span>
    <input type="text" class="form-control" placeholder="filter by" ng-model="filter">
    <span class="input-group-addon" ng-click="filter = ''">reset</span>
  </div>

  <div class="input-group ordercontrols">
    <label class="radio-inline">
      <input type="radio" value="label" ng-model="orderby"> Order by label
    </label>

    <label class="radio-inline">
      <input type="radio" value="id" ng-model="orderby"> Order by id
    </label>

    <label>
      <input type="checkbox" ng-model="reverse" />
      Reverse
    </label>
  </div>

  <div class="checkboxes">
    <div class="checkbox widgetcontrols" ng-repeat="elem in model | filter:{$:filter} | orderBy: orderby:reverse">
      <label>
        <input class="smart" type="checkbox" ng-model="elem.value" value="{{elem.value || false}}" />
        <span class="itemid">[{{elem.id}}]</span> {{elem.label}}
      </label>
    </div>
  </div>
</div>

Done!

How it works? This reusable component let you filter items and order them by label, id or reversed depending on the user input. If the user clicks on reversed, the list is reversed, if the user digits "Fir" as filter the list will be reduced and so on. Selecting one or more items, the binded $scope.exampleList will reflect the user choice.

How to test this directive

The directive has an isolate scope where model is two-way binded to $scope.exampleList (see model: '=' on the directive definition). The other ng-models you see into the template directives are just internal models that lives into the isolated scope of the myList directive (orderby, reverse, filter) and they don't affect the outer scope.

So when you are trying to change the reverse, orderby or filter properties, you'll have to do it on the right isolated scope!

'use strict';

describe('Directive: myList', function () {
  // load the directive's module
  beforeEach(module('myListApp', 'views/mylist.html'));

  var element,
    $rootScope,
    $compile,
    $element,
    scope;

  beforeEach(inject(function (_$rootScope_, _$compile_) {
    $rootScope = _$rootScope_;
    scope = $rootScope.$new();
    $compile = _$compile_;
  
  $rootScope.model = [
        {id: '001', label:'First item'},
        {id: '002', label:'Second item'}
      ];

    $element = angular.element('<my-list model="model"></my-list>');
    element = $compile($element)(scope);
    $rootScope.$apply();
  }));

  it('Labels order (reverse)', function () {
    var isolateScope = $element.isolateScope();

    isolateScope.reverse = true;
    isolateScope.orderby = 'label';
    isolateScope.$apply();

    expect($element.find('.itemid').eq(0).text()).toBe('[002]');
  });
});

So remember: .isolateScope is your friend!

Links:

• http://stackoverflow.com/questions/21488470/testing-angularjs-directive-template-not-updated-in-tests-after-changing-an-iso/21628921

Testing AngularJS directives with templateUrl generated with Yeoman's generator-angular

Are you in trouble while testing your reusable directive you are developing with AngularJS? Because are you testing your application, right?!

I had to solve some problems testing my AngularJS directive and if you are in trouble I hope you will save time reading this post. One of them is:

• testing directives that use the templateUrl statement

directives that use the templateUrl statement (and Yeoman)

The templateUrl option let you put the directive template into external html files instead of inlining html into your Javascript code.

It's a very useful option but if you try to test your directive with the templateUrl option you will get a nasty error:

Error: Unexpected request: GET views/yourdirectivetemplate.html

If you want to solve you can use use the ng-html2js Karma preprocessor. This way Karma immediately generates the js file, that puts the html into $templateCache. Alternatively you can alter the $templateCache by hand, not covered in this post.

All you need to know about testing AngularJS directives with templateUrl is described here:

• https://github.com/vojtajina/ng-directive-testing

Once followed all vojtajina's hints you will get this error if you have used the Yeoman's generator-angular because you have a different code structure (remember the app folder?):

Error: [$injector:modulerr] Failed to instantiate module views/yourdirectivetemplate.html due to: Error: [$injector:nomod] Module 'views/yourdirectivetemplate.html' is not available! You either misspelled the module name or forgot to load it. If registering a module ensure that you specify the dependencies as the second argument.

Don't worry, you only need to to adjust the karma.conf.js as suggested here (see the stripPrefix usage of ngHtml2JsPreprocessor):

• http://blog.freeside.co/post/60074672413/unit-testing-angular-directives-that-use-controller-and

Summary

Hope it might help you. Let me summarize what you need to do in order to fix your tests.

Happy testing!

[update 20140210] Install karma-ng-html2js-preprocessor

Remember to install the html2js Karma preprocessor:

$ npm install karma-ng-html2js-preprocessor --save-dev

Your package.json should contains the following line into devDependencies:

+    "karma-ng-html2js-preprocessor": "~0.1.0"

karma.conf.js

module.exports = function(config) {
  config.set({
    // base path, that will be used to resolve files and exclude
    basePath: '',

    // testing framework to use (jasmine/mocha/qunit/...)
    frameworks: ['jasmine'],

+    // generate js files from html templates+    preprocessors: {+      'app/views/*.html': 'ng-html2js'+    },
+
+    // needed if you have used the Yeoman's generator-angular or the app dir 
+    ngHtml2JsPreprocessor: {

+      stripPrefix: 'app/',
+    },
+

    // list of files / patterns to load in the browser
    files: [
      'app/bower_components/angular/angular.js',
      'app/bower_components/angular-mocks/angular-mocks.js',
+    'app/views/*.html',
      'app/scripts/*.js',
      'app/scripts/**/*.js',
      'test/mock/**/*.js',
      'test/spec/**/*.js'
    ],

test/spec/directives/yourdirective.js

describe('Directive: yourdirective', function () {

  // load the directive's module

  ...

+  beforeEach(module('views/yourdirectivetemplate.html'));

AngularJS MacGyver

And now we are talking about... AngularJS MacGyver, the duct tape and swiss army knife for AngularJS applications.

MacGyver provides a set of reusable components for AngularJS, for example:

• autocomplete
• tooltips
• menus
• and more...

You can see it in action with autocomplete tags:

How to setup MacGyver

Install it via bower:

$ bower install angular-macgyver --save

Add css:

<link rel="stylesheet" href="bower_components/MacGyver/lib/macgyver.css">

Load macgyver.js

<script src="bower_components/MacGyver/lib/macgyver.js"></script>

Just include Mac as a dependency:

angular.module(‘myModule’, [“Mac”])

And now you can use one of the provided directives, for example mac-autocomplete. If you get in trouble remember that the mac-autocomplete directive requires ng-model:

<mac-autocomplete
    ng-model="selected"
    mac-autocomplete-source="['hello', 'world']"
    mac-placeholder="'Autocomplete'">
</mac-autocomplete>

Done!

Links

• http://tech.shift.com/post/58726393361/introducing-macgyver
• http://ngmodules.org/modules/MacGyver

he duct tape and swiss army knife for AngularJS applications. - See more at: http://tech.shift.com/post/58726393361/introducing-macgyver#sthash.mw58vg1J.dpuf

he duct tape and swiss army knife for AngularJS applications. - See more at: http://tech.shift.com/post/58726393361/introducing-macgyver#sthash.mw58vg1J.dpuf

Mock python datetime.datetime.now (or built-in objects declared in C)

With the Python programming language you can patch almost everything with just few lines of code. It's quite easy, unless you are going to patch built-in objects declared in C (for example datetime.datetime.now, datetime.date.today, etc).

So what you can do if you have to mock this kind of methods in your tests?
One possible option is to use the forbiddenfruit library (https://github.com/clarete/forbiddenfruit).

This project aims to help you reach heaven while writing tests, but it may lead you to hell if used on production code.
It basically allows you to patch built-in objects, declared in C through python.

Here you can find a plone.app.testing layer definition that helps you to patch the datetime.datetime.now method:

class FakeDateTime(PloneSandboxLayer):

    defaultBases = (PLONE_FIXTURE,)

    def setUp(self):
        import datetime
        original_now = datetime.datetime.now
        first_now = original_now()
        delta = datetime.datetime(first_now.year + 1, 1, 1, 0, 0, 0) - first_now
        def fake_now(self):
            now = original_now()
            return now + delta
        curse(datetime.datetime, 'now', classmethod(fake_now))

    def tearDown(self):
        import datetime
        from forbiddenfruit import reverse
        reverse(datetime.datetime, 'now')

Links:

• https://github.com/clarete/forbiddenfruit 
• mocking datetime with mock via https://twitter.com/timostollenwerk/status/398381575995273216