Developer manual

This page contains information for jdwebsite developers.

Installation

Installation is easy on a Linux-like operating system.
For Windows and Mac, we advise to create a Linux virtual machine. Instructions are given below.

Basics

$ ./clean_env.sh
$ ./build_env.sh
$ source ./env/bin/activate
$ python create_local_settings.py
$ python website/manage.py createdb
$ python website/manage.py loaddata demo_data #Optional, creates admin/admin
$ python website/manage.py runserver

Requirements

Linux
Installation of the full project, and running a test server, can be done in a few minutes on any Linux machine. Just follow the steps under 'detailed instructions'. There is no need for manual configuration.

Windows and Mac
For Windows and Mac users, it is advised to install a Linux virtual machine and use this to install the project.

  • Follow the instructions and install Ubuntu on VirtualBox.
  • Start Ubuntu in a VirtualBox, open the program called "Terminal" and install some required applications by entering the command,
    $ sudo apt-get install git python-virtualenv python3-dev

Detailed instructions

Get the code
Create a new clone of the project,

$ git clone https://github.com/jonge-democraten/website.git

Virtual environment
Delete an existing virtual environment,

$ ./clean_env.sh

Create a virtual environment in ./env/ and install all dependencies,

$ ./build_env.sh

Activate the virtual environment,

$ source ./env/bin/activate

From now on, everything you do within the project should be from a shell with activated virtual environment.

Configure Django settings
Generate a Django SECRET_KEY and a local_settings.py,

(env) $ python create_local_settings.py

Create a database
You have to create an initial database and a root user and password for the database,

(env) $ python website/manage.py createdb

Run a test server
You can run a local test server with the command,

(env) $ python website/manage.py runserver

and visit http://127.0.0.1:8000 in your browser to view the web interface.

Hostnames

Add the following line to your /etc/hosts file to enable the subdomains,

127.0.0.1 jd.local lh.jd.local ams.jd.local

Update and clean

Get the latest changes
To get the latest version of the project, type the following git command in your project directory,

(env) $ git pull

Migrate possible database changes with,

(env) $ python website/manage.py migrate

Clean project
You can remove the virtual environment and database with,

$ ./clean_env.sh

Workflow

  • New features are developed on a separate feature branch.
  • The feature branch is merged with master if the feature is finished.
  • This allows to work independently on different features and still share code.
  • Push feature branch commits often to communicate what you are working on.
  • Read more about this workflow here.

Testing

All application logic code is to be unit tested. Unit tests are ideally created before development of functionality. It supports development and documents, in real code instead of text, what classes and functions are supposed to do. Feature branches are only merged if unit tests are written and all pass.
Higher level user interface actions are tested manually.

Unit tests

  • The project uses the Django unit test framework to create unit tests.
  • This framework is based on the Python unittest module.
  • Tests are defined in tests.py of the application directory.

Run the unit tests,

(env) $ python website/manage.py test <app_label>

Automated testing

Travis is used to automatically install the environment and run tests on changes in the project.
The file .travis.yml contains the Travis commands to install and test the project.

Performance

Locust can be used for load testing. It simulates users and provides a web UI to monitor the test. Easy to install and configure. Read the locust documentation.

Logging

The Python logging module is used for logging.

Example

To add log statements, simply add the following at the top of your Python file,

import logging
logger = logging.getLogger(__name__)

and add a new log statement anywhere in this file by,

logger.debug('debug log statement')
logger.warning('warning message')
logger.error('error message')

Output

The log statements include log level, application, class, function and line number of the log statement,

[2014-12-19 22:39:11] ERROR [website.tests::test_logfile() (23)]: Cannot find anything here.

Five log levels are available: debug(), info(), warning(), error() and critical().

The log statements for the applications are written to the console, if DEBUG=True, and always to debug.log and error.log. Django errors can be found in django.log.

Configuration

Logging is configured in the Django settings.py LOGGING variables. Information about configuration can be found here. New applications have to be added before logging becomes active for those applications.

Email error notifications

All admins, as defined in the ADMINS setting, will receive email notifications of all ERROR level log messages.

Confidential information

Confidential information should not be logged. During initial development, logging of confidential information is allowed if marked with a CONF tag,

logger.debug('CONF ' + member.DNA)

These will be removed before deployment.

Documentation

  • Documentation can be found in the /docs/ directory.
  • Docs are writtin in plain text with Markdown formatting.
  • MkDocs can be used to convert the doc files into nice looking html.
  • readthedocs.org hosts a html version of the documentation, generated from the markdown files.

MkDocs

You can use MkDocs to preview the documentation in the readthedocs template.
Install mkdocs using pip, 

$ pip install mkdocs

run a preview server in the project directory,

$ mkdocs serve

and visit 127.0.0.1:8000 to preview the generated documentation.

Code documentation

  • Add comments to code that is not self-explanatory.
  • Use python docstrings to describe files, classes and functions.
  • Add docstrings to files, classes and functions if useful.

Example

"""
File description.
"""

class ExampleClass(Example):
    """ Class description. """

    def example_function(self):
        """
        Function description 
        on multiple lines.
        """

Code standards

  • The default Python and Django code style is used.
  • Write code as simple as possible and focus on readability.
  • Write code for others to understand and read.

"Always code as if the person who ends up maintaining your code is a violent psychopath who knows where you live. " - source

Code check

  • Flake8 is a Python tool to check code style.
  • It runs automatically on Travis after each commit.
  • You can find the Flake8 output in the latest Travis build log.

Database

Migrations

A database migration needs to be created after database structure changes in models.py,

$ python website/manage.py makemigrations <app_label>

The generated migration file is committed together with changes in models.py.
Migrations have to be carefully managed between different branches, so keep track of other branches and prepare for a merge.

Demo data

The code base of jdwebsite contains demo data to demonstrate functionality. To create demo data, dump a fixture in /website/fixtures/, 

$ ./create_demo_data.sh

This fixture may be loaded when initialising the development environment (see Installation).

Caveat

For some reason, the categories of blog posts in Blogs on non-default Sites are not exported correctly. To include this information, you have to add it by hand.

Versioning

Version numbers are of the form,

1.3.9 N.N.N major.minor.micro
  • major release for everthing worth a new number
  • minor release for all functional and model (database) changes
  • micro release for bugfixes and minor view/template modifications

Pre-releases get a suffix ,

1.3.9a1 N.N.N(a/b/rc)N alpha/beta/release-candidate

PEP0440 is used as a basis for the version scheme.