Book of the month I'm reading this summer: Pylons/Pyramid (http://docs.pylonsproject.org/en/latest).
Pyramid (http://www.pylonsproject.org) is a minimal Python-based web development framework that let you "start small and finish big".It stole a lot of (good) ideas and concepts from other mature Python web frameworks and it is build with the pluggable and extensible concepts in mind. Read: no need to fork applications.
Furthermore Pyramid is database and template engine agnostic: you are free.
From the very beginning Pyramid allows you to become productive quickly. So why not start with something of useful?
Pyramid + Yeoman
The goal of this experiment is integrate yeoman with Pyramid (or other frameworks like NodeJs/Express with AngularJS or Plone as already did), preserving the yeoman's workflow.UPDATE 20140926: here you can see a Plone + AngularJS + Yeoman article (collective.angularstarter)
In this article I'll talk about what are the benefits you get integrating your Pyramid app with Yeoman, in future posts I'll discuss how they work under the hood with additional technical details omitted here (each used component deserves an entire blog post).
 |
| Yeoman |
Choosing the right tools is very important for the best develop experience and I cannot work anymore without Yeoman, especially when coding with Javascript.
 |
| Grunt |
- yo, scaffolding tool like pcreate, paster or zopeskel. It is widely adopted by a large and trasversal community
- grunt, system used for build, preview and test your software. Gulp is another popular option
- bower, used for dependency management, so that you no longer have to manually download and manage your scripts
 |
| Bower |
So with the yeoman's tools you can just code, avoid annoying repetitive tasks and don't worry about:
- javascript testing setup
- javascript code linting
- javascript/css minification and merging
- image minification
- html minification
- switch to CDN versions of you vendor plugins in production mode
- auto-reload browser
- much much more
The result will be a Pyramid starter seed project integrated with modern non Python-based web development tools.
Goals
Management of third party assets
You no longer have to manually download and manage your scripts with the Bower package manager.From http://bower.io:
"""Bower works by fetching and installing packages from all over, taking care of hunting, finding, downloading, and saving the stuff you’re looking for."""So just type something like: bower install angular-translate --save and you'll get the rigth resource with pinning support.
Tasks automation
Automation, automation, automation.From http://gruntjs.com:
"""Why use a task runner? In one word: automation. The less work you have to do when performing repetitive tasks like minification, compilation, unit testing, linting, etc, the easier your job becomes. After you've configured it, a task runner can do most of that mundane work for you—and your team—with basically zero effort."""Examples:
- grunt serve
- grunt test
- grunt build
- grunt YOUR TASK
- etc
Jslint
No more deploy Javascript code with bad indentation, syntax errors or bad code practices. |
| All syntax errors or bad practise will be found. |
Image minification
The build process will detect and minify automatically all your asset images.Uncss task
Modern (and heavy) UI frameworks like Twitter Bootstrap provide an excellent solution for prototyping your initial project, but most of the times you are using a very minimal subset of their functionalities.
This inspiring Addy Osmani's blog post helps you to remove unused css in your pages with a grunt task named grunt-uncss (https://github.com/addyosmani/grunt-uncss):The original not-minified bootstrap.css weights in at 120 kB before removing unused rule.
Css concat and minification
You can split your css code into different files and then the build process will concat and minify them creating a unique app.css file. This way you write modular and better readable css files, reducing the number of browser requests.The theme.css file is quite small but in real projects you can save more. In this case:
 |
| The configured build pipeline is concat, uncss and cssmin. 122.85 kB (original bootstrap.css) -> 4.64 kB (uncss) -> 3.45 kB (minification) |
Automatic CDN-ification
It is handy using unminified versions of third party javascript libraries during development and switch to CDN versions in production mode with well known benefits for your website.Don't worry: the cdnify task will take care about this boring issue. Automatically.
You save a boring manual and error-prone configuration.
Composable bootstrap.js version
The Pyramid starter project is based on Twitter Bootstrap. |
| Twitter Bootstrap |
As you can see the Javascript component of Twitter Bootstrap is very modular: http://getbootstrap.com/javascript. So if you don't use a particular feature, just don't include it.
This way in development mode you will have all individual plugins splitted in different files, in production it will served a unique concatenated and minified Javascript file built automatically.
So if you just need alert.js and dropdown.js you can get a 2.79 kB plugins.js:
 |
| The concatenation of alert.js and dropdown.js produces a 7.06 kB, that weight in at 2.79 kB after minification instead of the 8.9 kB (gzipped) bootstrap-min.js corresponding to not gzipped 27.2 kB. |
Html (template) minification
Since the ZPT/Chameleon templating language is an extension of HTML with xml syntax, |
| Brower are able to display unrendered ZPT/Chameleon templates |
I know, template minification can lead to potential unexpected problems due to minification issues on template files... but this is my personal playground, so let me play please!
So... why not switch to a pre-compiled minified template of your ZPT/Chameleon files when you are in "production mode"?
Obviously during development you will use the original template files.
The interesting side of this approach is that there is no overhead at response time, since the minification task runs just one time before deploying your application. It might be an option if you want just your html minified and you cannot feasibly add to your site or project additional optimization tools at web server level.
Anyway I have tried this mad experiment and... if you don't use too aggressive minification params, it seems to work fine with good results. Try it at your own risk or just disable it. Here you can the effects on the generated index.html used in production:
 |
| Template minified (7.62 kB -> 4.16 kB) |
Result: a lighter Pyramid
Same results but a lighter Pyramid app:
Let's see how it behave the standard Pyramid starter project:
 |
| Standard Pyramid starter project (production.ini) |
 |
| Pyramid starter seed (production.ini) |
Useful links
- uncss - http://addyosmani.com/blog/removing-unused-css/
- alternative installation of nodejs - http://greenfinity.hu/en/blog/installing-javascript-from-zc-buildout
- performance optimization techniques - http://www.smashingmagazine.com/2014/09/08/improving-smashing-magazine-performance-case-study
@ CSSconfeu CSS performance tooling slides (updated 20140919) - https://speakerdeck.com/addyosmani/css-performance-tooling
That's all?
No, you can do more, for example:- reduce the number or requests (for example you can merge vendor.css and app.css)
- create and keep updated css sprites with grunt (https://github.com/Ensighten/grunt-spritesmith)
- manage and upload all your assets to professional services like Amazon AWS (for example you can serve up all your images, styles and scripts from a S3 bucket + CloudFront). This way Pyramid will be able to handle more requests. Pyramid let you put static media on a separate webserver during production with static_url() in conjunction with add_static_view(), without having to change your templates code
- generate static gzipped assets with Grunt and let your webserver serve them
- install and configure dedicated performance modules at webserver level (Apache's mod_pagespeed)
- have a look at the pyramid_starter_seed repository on Github https://github.com/davidemoro/pyramid_starter_seed
- follow me on Twitter https://twitter.com/davidemoro
- keep in touch with me on Google+ (https://plus.google.com/+davidemoro) or Linkedin (https://www.linkedin.com/in/davidemoro82)
Links
- pyramid_starter_seed (github repo): https://github.com/davidemoro/pyramid_starter_seed
- part 1 (benefits of yeoman fully integrated with a web framework like Pyramid, this blob post): http://davidemoro.blogspot.com/2014/09/pyramid-starter-seed-yeomam-part-1.html
- part 2 (how to install pyramid_starter_seed ant its prerequisites) - http://davidemoro.blogspot.com/2014/09/pyramid-starter-seed-yeoman-part-2.html
- part 3 (how pyramid_starter_seed works under the hood - how to achieve these results and manage things with grunt, personalize pyramid_starter_seed and create your own starter template) - http://davidemoro.blogspot.com/2014/09/pyramid-starter-seed-yeoman-part-3.html
I think these 2 technologies angular and pyramid can do a lot for each other. pcreate -s starter gets the pyramid side started. yeoman (yo angular) gets the angular side started.
ReplyDeleteYour article has some good thoughts and ideas, particularly about minimizing pyramid templates, but I think pyramid templates might be made mostly obsolete by angular and angular templates. Most importantly can you describe how you wired angular and pyramid together?
Here is how I did it? It's not sufficient for development, but it works.
My initial and not-so-great approach is to create a ui directory under pyramid's static directory. The ui directory is then where I have yeoman generate the angular project 'yo angular'. After that I run 'grunt' which creates a dist directory with a completely minified, uglified, etc, etc...index.html file (and script, css and other files)
To get this to work with pyramid I rename dist/index.html to dist/index.html.pt (because my pyramid project is setup to use chameleon templates).
Then in the view config I set my home view to:
@view_config(route_name='home', renderer='static/ui/dist/index.html.pt')
def home(request):
return {}
This all works great at first glance, but it's totally insufficient for development. I want unminified code for development. But even if I'm able to use angular's index.html file without the grunt created 'dist' directory, which helps me in development, then it's no good in production.
What I need/want is for the renderer to be configurable based on if I'm in a dev environment and a production environment.
Thoughts?
Hi Jacob.
Deleteyou got the point: server side templating has less importance when we are talking about single page web applications, moving rendering to client side solutions (but you'll have to serve at least one, minimized or not, static html or a generated server side version).
You can achieve your goal using different techniques with pyramid. The one I applied in these articles is just one of the possible ways to switch from dev to a production environment. What you are asking for is probably explained in the part 3 of this article (http://davidemoro.blogspot.it/2014/09/pyramid-starter-seed-yeoman-part-3.html). Instead if you want to start from scratch instead of using pyramid starter seed you'll need to perform some changes to the generator-webapp (or angular) scaffold wrapped by the pyramid server side code in order to achieve a working integration (not covered by this tutorial but if you need help feel free to contact me).
I'm not yet fully satisfied with assets management (with Plone+angular I succeded in adopting a less verbose technique using named browser resources), but it works fine and it is quite easy to manage. So if you find a better solution let me know!
Thanks the custom renderer with %s was just what I needed. That helped a lot.
DeleteRegarding marking up the html file with request.static_url(...), I found that was breaking the production index.html file after it was re-written (minified, uglified, etc..).
I was considering using request.static_url() too, but given that angular will be the front-end and grunt will handle building the front-end including munging urls to whatever they need to be, I didn't use request.static_url() at all in the html files.
Besided adding the custom renderer I only needed to make one extra change. I needed to add all the static views via config.add_static_view() and then I have a special line for dev:
if not production:
config.add_static_view('bower_components', 'myproject:static/ui/bower_components', cache_max_age=3600)
to get the 'unbuilt' version of the front-end working with bower components correctly.
Not using request.static_url() in the html files might come back to bite me later, but at this point I don't see how or why, and both my dev and prod versions of index.html are working fine. Can you foresee any issues?
I think I'll run with this setup for now. Thanks the for blog entries and tips, very useful!
If you don't use request.static_url you loose the ability to serve static media on a separate webserver, without having to change your templates code.
DeleteIf you have broken minified templates I suggest to check your grunt htmlmin options (maybe they are too aggressive).
Just wanted to follow up quickly on this as I recently came across grunt-bridge.
ReplyDeleteUsing yeoman and 'yo angular' to build an angular seed project, yeoman will generate a Gruntfile.js file with tasks to concatenate and minifiy html, css and other things. The defaults leave you with a dist directory containing scripts/ and styles/ directories. These directories have the concatenated and minified html and css files.
The <script src="..."> references in the original html are removed and so are any 'static' references using request.static_url(). This makes using request.static_url(...) in the original index.html file ineffective.
However if you add the grunt-bridge task and have it execute after htmlmin, it will go through, find all your external references and replace them with a pattern of your choosing. It's in this step that I have grunt replace something like <script src="...my script..."> with <script src="${request.static_url('...my script...')}">
Maybe you'll find grunt-bridge useful too.
I Jacob,
Deletethanks for sharing!
Actually the asset management just it works in pyramid_starter_seed but it is too verbose compared to other similar solutions I made (see collective.angularstarter for example).
Do you have a working fork of pyramid_starter_seed with grunt-bridge enabled? Sounds interesting
No I don't have a fork going, but it's pretty straightforward. I just installed grunt-bridge with npm install. Then in the Gruntfile.js in the grunt.registerTask() section I register the 'bridge:dist' task right after the htmlmin task.
ReplyDeleteThe bridge:dist config in the Gruntfile.js looks like (override bridge defaults to play well with the yeoman setup):
bridge: {
dist: {
options: {
html: '<%= yeoman.dist %>/index.html',
dest: '<%= yeoman.dist %>/index.html',
pattern: '${request.static_url(\'my_project_name:static/ui/<%= yeoman.dist %>/{path}\')}'
}
}
},
Now in my original index.html file from the pyramid starter seed, I remove all request.static_url(...) calls and just leave the bare, relative urls.
This works perfect well developing, and then after doing a 'grunt' build, the minified index.html file is also perfectly setup with all static urls using request.static_url(...)