Davide Moro: February 2015

2015-02-28

Kotti CMS - how to turn your Kotti CMS into an intranet

In the previous posts we have seen that Kotti is a minimal but robust high-level Pythonic web application framework based on Pyramid that includes an extensible CMS solution, both user and developer friendly. For developer friendly I mean that you can be productive in one or two days without any knowledge of Kotti or Pyramid if you already know the Python language programming.

If you have to work relational databases, hierarchical data, workflows or complex security requirements Kotti is your friend. It uses well know Python libraries.

In this post we'll try to turn our Kotti CMS public site into a private intranet/extranet service.

I know, there are other solutions keen on building intranet or collaboration portals like Plone (I've been working 8 years on large and complex intranets, big public administration customers with thousands of active users and several editor teams, multiple migrations, etc) or the KARL project. But let's pretend that in our use case we have simpler requirements and we don't need too complex solutions, features like communities, email subscriptions or similar things.

Thanks to the Pyramid and Kotti's architectural design, you can turn your public website into an intranet without having to fork the Kotti code: no forks!

How to turn your site into an intranet

This could be an hard task if you use other CMS solutions, but with Kotti (or the heavier Plone) it will requires you just 4 steps:

define a custom intranet workflow
apply your custom worklows to images and files (by default they are not associated to any workflow, so once added they are immediatly public)
set a default fallback permission for all views
override the default root ACL (populators)

1 - define a custom intranet workflow

Intranet workflows maybe different depending on your organization requirements. It might be very simple or with multiple review steps.

The important thing is: no more granting the view permission for anonymous users, unless you are willing to define an externally published state

With Kotti you can design your workflow just editing an xml file. For further information you can follow the Kotti CMS - workflow reference article.

2 - apply your custom workflow to images and files

By default they are not associated to any workflow, so once added they are immediately public.

This step will requires you just two additional lines of code in your includeme or kotti_configure function.

Already described here: Kotti CMS - workflow reference, see the "How to enable the custom workflow for images and files" section.

3 - set a default fallback permission

In your includeme function you just need to tell the configurator to set a default permission even for public views already registered.

I mean that if somewhere into the Kotti code there is any callable view not associated to a permission, it won't be accessible by anonymous after this step.

In your includeme function you'll need to :

def includeme(config):
    ...
    # set a default permission even for public views already registered
    # without permission
    config.set_default_permission('view')

If you want to bypass the default permission for certain views, you can decorate them with a special permission (NO_PERMISSION_REQUIRED from pyramid.security) which indicates that the view should always be executable by entirely anonymous users, regardless of the default permission. See:

http://docs.pylonsproject.org/docs/pyramid/en/latest/api/security.html#pyramid.security.NO_PERMISSION_REQUIRED

4 - override the default root ACL (populators)

The default Kotti's ACL associated with the root of the site

from kotti.security import SITE_ACL

gives view privileges to every user, including anonymous.
You can override this configuration to require users to log in before they can view any of your site's pages. To achieve this, you'll have to set your site's ACL as shown on the following url:

http://kotti.readthedocs.org/en/latest/developing/advanced/close-for-anonymous.html

You'll need you add or override the default populator. See the kotti.populators options here:

http://kotti.readthedocs.org/en/latest/developing/basic/configuration.html#kotti-populators

Results

After reading this article you should be able to close your Kotti site for anonymous users and obtaining a simple, private intranet-like area.

Off-topic: you can also use Kotti as a content backend-only administration area for public websites, with a complete decoupled frontend solution.

UPDATE 20150623: now you can achieve the same goals described in this article installing kotti_backend. See https://github.com/Kotti/kotti_backend

Useful links

All posts about Kotti

All Kotti posts published by @davidemoro:

http://davidemoro.blogspot.it/search/label/kotti

2015-02-24

Kotti CMS - how to store arbitrary data with annotations

With Kotti CMS you can extend existing types inheriting from a base class (eg: Document) and obtain another type of object (eg: MyDocument) with new fields, new workflows, custom views, custom addability conditions, etc.

But sometimes you may want to add a custom field to one or more resources, without having to create a new type. For example you might want to add a colour attribute to all existing Document objects, let's imagine a simple select widget with few colours that will be used for adding a class depending on the choosen colour.

By default Kotti is shipped with an annotations column that can be used to store arbitrary data in a nested dictionary.

You can store arbitrary data in the nested dictionary with a syntax similar to the following one:

context.annotations['SOMEKEY'] = VALUE

and read annotations with:

context.annotations

All you need to do is overriding the add and edit form of your target class. With Pyramid is quite easy to extending an existing application and override views, assets, routes, etc. See http://docs.pylonsproject.org/docs/pyramid/en/latest/narr/extending.html for further info.

Here you can see one possible implementation:

from pyramid.view import view_config
import colander
from deform.widget import SelectWidget
from kotti_actions.views.edit.actions.link import (
    LinkActionAddForm as OriginalLinkActionAddForm,
    LinkActionEditForm as OriginalLinkActionEditForm,
    )
from kotti_actions.resources import (
    LinkAction
    )
...

colours = [
          ('', 'Select'),
            ('red', 'Red'),
            ('brown', 'Brown'),
            ('beige', 'Beige'),
            ('blue', 'Blue'),
            ]

def add_colour(schema):
    schema['colour'] = colander.SchemaNode(
        colander.String(),
        title=_('Colour'),
        widget=SelectWidget(values=colours),
        missing=u"",
        )

@view_config(name=LinkAction.type_info.add_view, permission='add',
             renderer='kotti:templates/edit/node.pt')
class LinkActionAddForm(OriginalLinkActionAddForm):
    """ Form to add a new instance of CustomContent. """

    def schema_factory(self):
        schema = super(LinkActionAddForm, self).schema_factory()
        add_colour(schema)
        return schema

    def add(self, **appstruct):
        colour = u''
        try:
            colour = appstruct.pop('colour')
        except KeyError:
            pass
        obj = super(LinkActionAddForm, self).add(**appstruct)

        obj.annotations['colour'] = colour
        return obj

@view_config(name='edit', context=LinkAction, permission='edit',
             renderer='kotti:templates/edit/node.pt')
class LinkActionEditForm(OriginalLinkActionEditForm):
    """ Form to edit existing calendars. """

    def schema_factory(self):
        schema = super(LinkActionEditForm, self).schema_factory()
        add_colour(schema)
        return schema

    def before(self, form):
        super(LinkActionEditForm, self).before(form)
        colour = self.context.annotations.get('colour')
        if colour:
            form.appstruct.update({'colour': colour})

    def edit(self, **appstruct):
        super(LinkActionEditForm, self).edit(**appstruct)
        self.context.annotations['colour'] = appstruct['colour']

Now our LinkAction add and edit form will have an additional select with our colours.

All posts about Kotti

All Kotti posts published by @davidemoro:

http://davidemoro.blogspot.it/search/label/kotti

2015-02-11

Python mock library for ninja testing

If you are going to write unit tests with Python you should consider this library: Python mock (https://pypi.python.org/pypi/mock).

Powerful, elegant, easy, documented (http://www.voidspace.org.uk/python/mock/)...
and standard: mock is now part of the Python standard library, available as unittest.mock in Python 3.3 onwards.

Simple example

Let's suppose you have an existing validator function based on a dbsession import used for querying a relational database. If you are going to write unit tests, you should focus on units without involving real database connections.

validators.py
from yourpackage import DBSession

def validate_sku(value):
    ...
    courses = DBSession.query(Course).\
        filter(Course.course_sku == value).\
        filter(Course.language == context.language).\
        all()
    # validate data returning a True or False value
    ...

tests/test_validators.py
def test_validator():
    import mock
    with mock.patch('yourpackage.validators.DBSession') as dbsession:
        instance = dbsession
      instance.query().filter().filter().all.return_value = [mock.Mock(id=1)]
        from yourpackage.validators import sku_validator
      assert sku_validator(2) is True

In this case the DBSession call with query, the two filter calls and the final all invocation will produce our mock result (a list of with one mock item, an object with an id attribute).

Brilliant! And this is just one simple example: check out the official documentation for further information:

http://www.voidspace.org.uk/python/mock/

2015-02-04

Kotti CMS - workflow reference

Yet another blog post about Kotti CMS (http://kotti.pylonsproject.org/): this time I'm going to talk about workflows and security.

Workflows in Kotti are based on repoze.workflow. See http://docs.repoze.org/workflow/ for further information. Basically you can use an xml file (zcml) in order to describe your workflow definition. You can see an example here: https://github.com/Kotti/Kotti/blob/master/kotti/workflow.zcml. A you can see it is quite straightforward adding new states, new transitions, new permissions, etc. You can easily turn your 2-states website workflow into a 3-states website workflow with reviewers or turn Kotti app into an intranet application.

The default workflow definition is loaded from your project .ini file settings (using the kotti.use_workflow settings). The kotti.use_workflow setting's default value is:

kotti.use_workflow = kotti:workflow.zcml

but can change change default workflow for the whole site, register new workflows related to specific content types or disable it as well.

Anyway, if you need to write a Python based CMS-ish application with hierarchical contents, custom content types, workflows, security, global and local ACL (sharing permissions), pluggable and extensible, based on relational databases, developer friendly, with a simple UI, etc... Kotti is your friend!

How to disable the default workflow

Kotti is shipped with a simple workflow implementation based on private and public states. If your particular use case does not require workflows at all, you can disable this feature with a non true value. For example:

kotti.use_workflow = 0

How to override the Kotti's default workflow for all content types

The default workflow is quite useful for websites, but sometimes you need something of different. Just change your workflow setting and point to your zcml file:

kotti.use_workflow = kotti_yourplugin:workflow.zcml

The simplest way to deal with workflow definitions is:

create a copy of the default workflow definition
customize it (change permissions, add new states, permissions, transitions, initial state and so on)

If your site already has content and you configure it use a workflow for the first time, or you use a different workflow than the one you used before, run the kotti-reset-workflow command to reset all your content's workflow.

How to enable the custom workflow for images and files

Images and files are not associated with the default workflow. If you need a workflow for these items you need to attach the IDefaultWorkflow marker interface.

You can add the following lines in your includeme function:

from zope.interface import implementer
from kotti.interfaces import IDefaultWorkflow
from kotti.resources import File
from kotti.resources import Image
...


def includeme(config):
    ...
    # enable workflow for images and files
    implementer(IDefaultWorkflow)(Image)
    implementer(IDefaultWorkflow)(File)
    ...

How to assign a different workflow to a content type

[UPDATE 20150604:
We are going to use the default workflow for standard content types and a custom workflow for content providing the IBoxWorkflow marker interface. The custom workflow is configurable via .ini configuration files.
Note well: the elector, the config.begin/config.commit and the load_zcml can be omitted. I'll update this section soon.

UPDATE 20150606:
Confirmed: things can be done in a simpler way! So no load_zcml, no config.begin/config.commit, no elector.
See the following video:
]

In this kind of situation you want to use the default workflow for all your types and a different workflow implementation for a particular content type.

You'll need to:

create the new workflow definition, with a workflow elector
write an elector function that will returns True or False depending if the workflow should be applied (otherwise will win the default default workflow, or better, the first matching workflow without an elector)
load manually your zcml file in your includeme function

.ini file (optional)

kotti_boxes.use_workflow = kotti_boxes:workflow.zcml

__init__.py

from pyramid.i18n import TranslationStringFactory
from kotti import FALSE_VALUES


def includeme(config):
    ...
    workflow = config.registry.settings.get('kotti_boxes.use_workflow', None)
    if workflow and workflow.lower() not in FALSE_VALUES:
        config.begin()

        config.hook_zca()
        config.include('pyramid_zcml')
        config.load_zcml(workflow)
        config.commit()

    ...

workflow.py
From the repoze.workflow documentation: """A workflow is unique in a system using multiple workflows if the combination of its type, its content type, its elector, and its state_attr are different than the combination of those attributes configured in any other workflow."""
Depending on how specific is your combination you may need to implement an elector (a function that returns True or False for a given context).

from kotti_boxes.interfaces import IBoxWorkflow


def elector(context):
    return IBoxWorkflow.providedBy(context)

workflow.zcml

<configure xmlns="http://namespaces.repoze.org/bfg"
           xmlns:i18n="http://xml.zope.org/namespaces/i18n"
           i18n:domain="Kotti">
  <include package="repoze.workflow" file="meta.zcml"/>

  <workflow
      type="security"
      name="simple"
      state_attr="state"
      initial_state="private"
      content_types="kotti_boxes.interfaces.IBoxWorkflow"
      elector='kotti_boxes.workflow.elector'
      permission_checker="pyramid.security.has_permission"
      >
    <state name="private" callback="kotti.workflow.workflow_callback">
      <key name="title" value="_(u'Private')" />
      <key name="order" value="1" />
      <key name="inherit" value="0" />
      <key name="system.Everyone" value="" />
      <key name="role:viewer" value="viewbox view" />
      <key name="role:editor" value="viewbox view add edit delete state_change" />
      <key name="role:owner" value="viewbox view add edit delete manage state_change" />
    </state>
    ...

  <transition
      name="private_to_public"
      from_state="private"
      to_state="public"
      permission="state_change" />
  ...

  </workflow>
</configure>

All posts about Kotti

2015-02-03

Kotti CMS - how to create a new content type with an image

If you want to create a new content type based on an existing one with Kotti you need to write few lines of code and zero html for the add and edit views: it is very simple (browse Kotti's resources.py and views code).

Basically you have to extend the existing content type shipped with Kotti and add your custom fields.

But let's suppose you need a new content type named ImageWithLink with the following fields:

title
description
image
link

In this case the implementation is more verbose compared to extend another content type (like the Document, but it is still an easy job).

resources.py

from zope.interface import implements
from kotti.resources import Image
from kotti.interfaces import IImage
from sqlalchemy import Column
from sqlalchemy import ForeignKey
from sqlalchemy import Integer
from sqlalchemy import Unicode

class ImageWithLink(Image):
    implements(IImage)

    id = Column(Integer, ForeignKey('images.id'), primary_key=True)
    link = Column(Unicode(1000))

    type_info = Image.type_info.copy(
        name=u'ImageWithLink',
        title=u'ImageWithLink',
        add_view=u'add_image_link',
        addable_to=['Document'],
        )

    def __init__(self, link=u"", **kwargs):
        super(ImageWithLink, self).__init__(**kwargs)
        self.link = link

The code is quite self-explaining: you create a new ImageWithLink class that inherits from Image. You only need to add your custom field named link and you initialize the link in the __init__ code after calling the super method.

views/content.py

import colander
from deform import FileData
from deform.widget import FileUploadWidget
from kotti.views.edit import ContentSchema
from kotti.views.edit.content import ImageEditForm
from kotti.views.edit.content import ImageAddForm
from kotti.views.form import validate_file_size_limit
from kotti.views.form import FileUploadTempStore
from kotti.views.form import AddFormView
from pyramid.view import view_config
from kotti_yourplugin import _
from kotti_yourplugin.resources import ImageWithLink
from kotti_yourplugin.validators import link_validator

def ImageWithLinkSchema(tmpstore):
    """ File schema with no set title missing binding """
    class ImageWithLinkSchema(ContentSchema):
        file = colander.SchemaNode(
            FileData(),
            title=_(u'File'),
            widget=FileUploadWidget(tmpstore),
            validator=validate_file_size_limit,
            )
        link = colander.SchemaNode(
            colander.String(),
            title=_('Link'),
            validator=link_validator,
            missing=u'',
            )

    def after_bind(node, kw):
        del node['tags']

    return ImageWithLinkSchema(after_bind=after_bind)

@view_config(name='edit', permission='edit',
             renderer='kotti:templates/edit/node.pt')
class ImageWithLinkEditForm(ImageEditForm):
    def schema_factory(self):
        tmpstore = FileUploadTempStore(self.request)
        return ImageWithLinkSchema(tmpstore)

@view_config(name=ImageWithLink.type_info.add_view, permission='add',
             renderer='kotti:templates/edit/node.pt')
class ImageWithLinkAddForm(ImageAddForm):
    item_type = _(u"Banner Box")
    item_class = ImageWithLink

    def schema_factory(self):
        tmpstore = FileUploadTempStore(self.request)
        return ImageWithLinkSchema(tmpstore)

    def save_success(self, appstruct):
        # override this method (no filename as title
        # like images)
        return AddFormView.save_success(self, appstruct)

    def add(self, **appstruct):
        # override (no tags in our form)
        buf = appstruct['file']['fp'].read()
        filename = appstruct['file']['filename']
        return self.item_class(
            title=appstruct['title'] or filename,
            description=appstruct['description'],
            data=buf,
            filename=filename,
            mimetype=appstruct['file']['mimetype'],
            size=len(buf),
            )

Here the code is more complex. There is a dynamic schema definition with the Kotti's temp store implementation. Both the add and the edit form refer to this schema, with some overrides because our object does not behave like files or images.

validators.py
UPDATE 20150211: no need to write this validator. Use the url validator provided by colander instead (colander.url). Anyway you can use all the builtin colander validators or write your own validators.

import re
import colander
from kotti_yourplugin import _

VALID_PROTOCOLS = ('http',)
URL_REGEXP = r'(%s)s?://[^\s\r\n]+' % '|'.join(VALID_PROTOCOLS)

def link_validator(node, value):
    """ Raise a colander.Invalid exception if the provided url
        is not valid
    """
    def raise_invalid_url(node, value):
        raise colander.Invalid(
            node, _(u"You must provide a valid url."))
    if value:
        if not re.match(URL_REGEXP, value):
            raise_invalid_url(node, value)

Here you can see an example of link validator based on a regular expression. This validator decorates our link field of the ImageWithLink schema.

Obviously you need to add in your kotti_configure method your ImageWithLink in the kotti.available_types settings.

__init__.py

def kotti_configure(settings):
settings['pyramid.includes'] += ' kotti_yourplugin'
settings['kotti.available_types'] += ' kotti_yourplugin.resources.ImageWithLink'

and enable your configurator in your .ini file:

kotti.configurators = mip_course.kotti_configure

And what about the default view of your content types? If you visit an ImageWithLink box it will behave like an image: it inherits the default view of the image (you should customize it adding the link on the image, very simple: not showed in this blog post), no need to deal with the image resize machinery, etc.

As you can see, Kotti is a flexible solution if you need a simple but powerful CMS solution based on Python, Pyramid and SQLAlchemy. You may consider it as a simple framework (but easy to understand, don't be scared by the word framework. It is really developer friendly). If you are curious about how to manage contents with Kotti you may play with the demo online: http://kottidemo.danielnouri.org/ (admin - qwerty).

All posts about Kotti

All Kotti posts published by @davidemoro:

http://davidemoro.blogspot.it/search/label/kotti