We'll see in this blog post:
- advantages of the public website decoupled (frontend) from the private content administration area (backend) pattern
- PHP vs Python comparison
- Symfony2 vs Pylons/Pyramid
- Plone vs Kotti
- Python evangelism
Update 20150718Part 2 is out! See:
The challengeThat were the requirements for an important project I joined last January:
- build a new CMS from scratch (yeah, I know, it doesn't make sense reinventing the wheel: see How to choose your CMS)
- based on PHP/Symfony
- with only two (2) developers
- end user friendly
- through the web customization for visual sections by editors/administrators (home page text introduction, main navigation, header and footer links, boxes, etc).
- with an external ecommerce integration
- custom logics and behaviours
- custom security based on roles, workflow, permission sharing
- configurable portlets system (manageable columns)
- custom navigation link managers
- elasticsearch integration
- nested urls (folders hierarchy)
Obviously there was a previous huge problem at commercial/estimation level and Peppa Pig would have said the famous "THIS IS IMPOSSIBLE!" because build a new CMS is an incredibly enormous task... but let's see why we are still alive and how it was possible to meet these impossible requirements with hard work, a bit of (italian) inventiveness and the Python/Pyramid based Kotti CMS.
1 - Does anyone already found a solution to a similar problem?Not listed here:
2 - Switch technology to Plone attemptNow I'm serious :)
The first thought was: try to change the technology stack switching to Plone (http://plone.org) but the Windows/PHP/Symfony/MySQL stack was not an option, so no Plone for this project.
3 - Have a look at existing PHP-based CMS solutionsSecond step, check if there is a decent CMS solution built with PHP with this requirements:
- easy to extend and maintain
- with a good and modern codebase
- with an intuitive backend interface for editors
In addition introducing a new framework usually means more time and a lot of pain if you have to customize almost everything if the framework it is not built with flexibility in mind.
4 - try out Sonata + Symfony2 from scratchWe tried to setup a Symfony2 project powered by Sonata and we had a look at to existing early stage "CMS"s Sonata based but it was clear that it wasn't the right tool for building a real content hierarchy aware CMS usable by end users. Trust me, CMS is another sort of thing: you cannot call CMS the ugly version of the Django admin or something that is not suitable for end users. Anyway, Sonata is a quite good solution if you are going to write a data administration area for your custom Symfony2 application. See http://symfony.com/doc/1.0/cmf/cookbook/creating_cms_using_cmf_and_sonata.html and http://demo.sonata-project.org/.
5 - The solutionIt was clear that it was impossible for us building a heavy customized CMS with almost "enterprise" requirements based on PHP for our team composition, experience, tight deadlines, etc.
So the solution was: why not adopting a good CMS based on a relational database storage powered by Python (there are very good options) and use it as a content administration backend area with a decoupled frontend built with Symfony2/PHP?
We had a look to Django and Pyramid CMS solutions and Kotti seemed the best option:
- very good code base (and its core it is damn small)
- powered by Pylons/Pyramid (quite small, flexible, easy to extend and approachable Python web framework with url traversing capabilities for free)
- clean user interface
- powered by SQLAlchemy
- very very flexible if you need to turn the whole system inside out
- many good concepts available in Kotti or Pyramid/Pylons are largely inspired by the Zope/Plone world (two decades of successful web applications and content management experience there)
- it is not only a CMS but a web development framework
- 1 PHP developer on the PHP-Symfony frontend side
- 1 Python developer on the Python Kotti CMS backend
- PHP is not Python
- Symfony2 is not Pylons/Pyramid (security, routing, traversal, view declarations with discriminators)
- Kotti is not Plone (very good solution but it is still quite minimal), so I had to implement the missing parts we need. The good news is that Kotti and Pyramid you are very productive
- but... Pylons/Pyramid is Pylons/Pyramid! Very happy and damn productive, I really enjoyed programming with this framework. Very impressed.
PHP is not PythonConcepts like decorators not available as PHP builtin: cool things like decorators are only comments that are "compiled" later. And it is more hard implementing new "decorators" and test them compared to Python.
It seems that you cannot set to a class an instance of another class (our Kotti's type_info for example).
Doctrine or Propel works fine if you are adopting the Docrine or Propel pattern. If you need something of different, no way. We wrote a custom abstraction on the top of the pattern used by Kotti and SQLAlchemy for types inheritance with joined tables (!!). Last but not least, SQLAlchemy provides things like:
- association_proxy (used in Kotti tags). See http://docs.sqlalchemy.org/en/latest/orm/extensions/associationproxy.html
- other techniques used by kotti_multilingual used for independent fields. See https://github.com/Kotti/kotti_multilingual/blob/master/kotti_multilingual/sqla.py
Bit operations math for cookies shared auth among Symfony2 frontend and Kotti CMS backend.
Generally it was a bit frustrating having to write a lot of code what you get for free with Python with just 2 lines of code.
Symfony2 is not Pylons/PyramidWhat is Symfony2: it is one of the better frameworks available in the PHP world, probably the most promising. It is not a monolithic framework, you can also use standalone symfony components as well in your non-Symfony PHP projects (many famous existing PHP projects are switching to Symfony). Symfony2 plays well with other existing components like ORMs and it tries to bring innovation to PHP.
It promotes best development practise, testing, modularity, extensibility. It has its own dependency injection system and it let you write templates with a quite good template system named Twig.
Anyway, quite good ideas with concepts stolen from other non-PHP framework, in particular dependency injection plus a configuration hooks if you want to replace existing components without having to modify an existing application (but no component adapter patterns). So with Symfony you can write good code, despite PHP.
Yeah, Symfony2 is quite good and it tries to bring innovation to the PHP world: very good. But if you feel innovative and you like so much innovation why not switching to other technologies that were already more innovative 20 years ago?! And now things are even better. Programming languages like Python and web frameworks like Pylons/Pyramid are on another planet: they really seem built by aliens compared to other PHP frameworks.
For example with Python you have:
- package managers like pip that exists since many years
- ORM with extreme flexibility (SQLAlchemy is on another planet too). Doctrine or Propel works fine just if you follow the Doctrine/Propel way, if you need alternative patterns they might not follow your needs with a lot of pain
- less security vulnerabilities (see https://plone.org/products/plone/security/overview)
- easy security setup for your applications
- component adapter patterns
- easy debugging (PDB)
- routes and above all concepts like traversal (see http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/muchadoabouttraversal.html)
- clean syntax and builtin concepts like decorators and context managers
- no random byte code cache corruption
- more productivity
or if you have complex requirements you can have a look at buildout.$ git fetch $ git checkout tags/0.1.5 $ pip install -r requirements.txt $ restart your application
Probably complex is the wrong word, it would be better verbose. For verbose I mean: composer install with a lot of options and dump-autoload, clear the Symfony2 prod cache avoiding cache corruption issues and install assets, cache warmup, be sure there are right permissions for writeable directories, etc.
So the "just copy and paste the code" myth (#1) with PHP is not true anymore and I don't like it.
And the very first PHP initial setup on your computer is not so easy. I think for a newbie it is much more easy a different one or two commands approach: install requirements and launch one command.
If you have to start with a PHP/Symfony2 setup just for developing is quite a long and error prone task. That's why my very first experiment with PHP-Symfony2 was: adopt Vagrant/Ansible for environment setup and provisioning. If you need a Vagrant/Ansible example you can have a look at here: https://github.com/davidemoro/symfony-vagrant-ansible.
Another false myth (#2): the most used technology it is better than others: no.
Yet another false myth (#3): the most used technology is more secure: no, absolutely not. See again https://plone.org/products/plone/security/overview.
Let me show other things you can do better with Python:
- "byte code cache" corruption issues
- missing traversal concept
- views registration and override
- security configuration
Symfony2: traversal... what?! You need to implement it, while in Pylons/Pyramid comes for free. See http://docs.pylonsproject.org/projects/pyramid//en/latest/narr/muchadoabouttraversal.html
Add or customize views. Just one example: register or override the default view of a Document only if its parent is a Course if damn easy with Pylons/Pyramid (just register a new view with a discriminator, one line). With other frameworks you'll need to write a lot of not generic code. See http://docs.pylonsproject.org/projects/pyramid//en/latest/narr/viewconfig.html
Debugging is a pain with PHP. Debugging things with PHP and/or Symfony is a bit frustrating if you had previous experience with the Python debugger (pdb) or with the pyramid debug toolbar and its interactive through the web exception shell. If something goes wrong you can see what's the problem interactively or why a variable is "None" without leaving the browser. See:
https://symfony.com/doc/current/book/security.html). Things can be complex but if your API lead people to be confused, probably you have a bad design problem.
Anyway it is more easy working with Pylons/Pyramid if you are going to write applications with complex security requirements (even workflow-based security thanks to the third party repoze.workflow). See http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/security.html
I hope you don't consider the above personal thoughts as not constructive rants: Symfony2 is a good and promising PHP framework and if you are interested in building web applications with PHP it is definitively a very good choice. I have been working with Symfony for 6 months and this is just my opinion compared to my previous programming experience with Zope, Plone, Grok, BFG, Pylons/Pyramid, Django and a bit of Nodejs/Express (KeystoneJS, SailsJS, etc).
So the previous Symfony2/PHP vs Python/Pyramid/Python comparison boils down to: if you are a curious web developer I think it really worth leaving your "comfort zone" and have a look at something of new. It doesn't matter if you are going to use Python (Django, Pyramid/Pylons, Flask, morepath, etc), Ruby, NodeJS or whatever else: you'll learn a lot of useful new things for sure and innovate your work!
Kotti CMS is not PloneKotti is very lightweight framework and its core is damn small because:
- a lot of things are demanded to the SQLAlchemy layer for storage, queries, indexing, etc.
- simpler stack compared to Plone/Zope/CMF/Archetypes/Dexterity/etc (just Kotti, Pylons/Pyramid, SQLAlchemy + a list of well known and widely adopted third party libraries. So if you spend time learning Pylons/Pyramid, you can easily switch to other frameworks). For example if you have to write event handlers with Kotti you'll notice that things are more simpler (no event handlers hell or having to manage duplicated events)
- tries to keeps things simple
- new programmers with no Pylons/Pyramid background are productive after a couple of days
It is very easy to start with: just a couple of days and you will be ok because it is very lightweight.
So programmers with no Pylons/Pyramid nor SQLAlchemy knowledge (or even not Python developers) will be productive very soon with Kotti. Obviously if you have a previous SQLAlchemy, Plone, Pylons/Pyramid or any other Python frameworks background it is better but it is not required at all.
Obviously Kotti is not Plone:
- community (the Plone's community is much more large)
- Plone has a lot of third party plugins
- no auto generated add/edit forms (not a problem because it is very simple to create or customize forms thanks to Colander/Deform. So no z3c.form there!)
- no portlets (topic covered in next blog post)
- no link actions (eg: portal_actions, topic covered in next blog post)
- missing permalinks and not breakable links
- no collections
- no PloneFormGen-like plugins (but we used a very good form builder solution available online as a service with integrated CRM)
Why frontend decoupled from the backend patternA couple of definitions about my frontend and backend concepts:
- frontend. It is our public website, where anonymous users browse the contents of the website. In our case it is built with PHP/Symfony2 but obviously you can also do all things with Python alone (pure Python is the best option)
- backend. It is our private content management area, built with Kotti CMS, Pylons/Pyramid and Python
If you adopt a separation among frontend and backend your work will be even more agile:
- you can start with a "blank sheet" theme (topic covered in next articles)
- you don't waste time removing or hiding features or unneeded views, just implement what you need. Probably guys with experience with big fat frameworks understand what I mean.
- you can develop new complex features like portlets (box shown in views) without having to touch at all the backend interface because the backend is just a (raw) backend area! They are just another type of content (not publishable on the frontend but rendered in views). So you can adopt workflows on portlets, you can copy/cut and paste them, use different views on portlets, assign custom views, create custom addability rules based on portlet types, etc. This way you can develop complex features like portlets in a fraction of time!
- since the frontend and the backend are served on the same domain, there is shared authentication so you can implement toolbar, live edit or view site as anonymous for editors
- less SEO headaches for non-publishable objects (portlets, tabs, collapsable sections, etc), you can store in the administration area objects that won't be published by the frontend at all. It is very handy implement these things like regular contents because you inherit workflows, same navigation/editing interface, etc. For example you can manage collapsable paragraphs as regular content type objects on the backend (yoursite.com/cms/document/paragraph1) but they will be rendered on the document view yoursite.com/document (if you try to access yoursite.com/document/paragraph1 from the frontend you'll get a 404 NotFound). Since they are not published on the public website you don't have to protect urls, no need to redirect to the main parent content, etc: all these things requires a lot of extra work if you want to take care of this kind of details
- if you use the same technology for the public website and the private area, you can reuse code already defined on the backend (for example you can reuse the breadcrumbs callable view with a completely template)
- faster. You can build the frontend with less security checks (just expose public data with less complexity due to assertion about local roles, groups, etc)
- more secure. You'll keep the administration area (backend) completely private and the frontend in our case built with PHP with a low privilege database user (readonly)
- you can rebuild the frontend without having to touch the administration area (backend)
- frontend easy replaceable
- different fulltext search policy for frontend vs frontend. On the backend there is the standard search results (you search for a text contained on a collapsable document and you'll land on that collapsable document on the backend, just what you expect from the editor perspective). If you search the same word on the public website you'll land on the parent document of the collapsable document (good for visitors)
Photoes (credits)A-team photoes from http://iqtell.com/2014/05/10-reasons-why-the-a-team-is-better-than-your-team/
All Kotti posts published by @davidemoro
End of part 1In this article we have seen that:
- decoupled is cool!
- Python is cool!
- how to build a pure Python Kotti based setup case study with a private content management area decoupled from the public website (with tips, links and screenshots, etc).