Lettuce documentation contents

Lettuce is a Behavior-Driven Development tool written by Gabriel Falcão G. de Moura

nutshell

install it

user@machine:~$ [sudo] pip install lettuce

describe your first feature

Feature: Manipulate strings
  In order to have some fun
  As a programming beginner
  I want to manipulate strings

  Scenario: Uppercased strings
    Given I have the string "lettuce leaves"
    When I put it in upper case
    Then I see the string is "LETTUCE LEAVES"

define its steps

>>> from lettuce import *
>>> @step('I have the string "(.*)"')
... def have_the_string(step, string):
...     world.string = string
...
>>> @step
... def i_put_it_in_upper_case(step):
...     world.string = world.string.upper()
...
>>> @step
... def see_the_string_is(step, expected):
...     '''I see the string is "(.*)"'''
...     assert world.string == expected, \
...         "Got %s" % world.string

watch it pass

user@machine:~/Projects/my-project$ lettuce features/

getting involved !

github project page

Fork it, propose features, explore the code

github.com/gabrielfalcao/lettuce.

feedback

issue tracker.

discuss

hack

Development guidelines

donate

support lettuce development

hands on!

Is this your first experience with Lettuce ?!?

So, why don’t you go straight to the quick start tutorial ?!

introduction

what is Lettuce, and what it does

integrate!

furthermore

reference and concepts

recipes

make your own salad

Warning

Disclaimer on unstable features you should NOT rely on.

For instance use the features step.behave_as and the xunit XML output are unstable and untested.

Lettuce is a software for testing other software, but I’ve been receiving some contributions of new features that are coming without proper tests.

These contributions have to be rewritten with proper tests, so that they won’t break so often, but they are a current source of issues in the bug tracker.

So until someone contributes with unit and functional tests for those features, or I find some time to do it myself, you should avoid relying on them.

Sincerely, Gabriel Falcão

Overview

On BDD

Behaviour-driven development is a very good approach for maintaining the workflow plain, so you only spend time with what really matters: business value.

The common BDD approach basically consists in:

  • writing some unit tests
  • running these tests
  • making these tests fail
  • writing code
  • making the code pass these tests (green status)

This is a very awesome practice, since you can build huge and reliable software without fearing the future. You don’t need to worry if those millions of lines of code won’t make sense in 10 years, as long as they keep passing the tests. Despite BDD, other kind of tests are very important and usually follow a similar workflow: functional, integration and acceptance.

Nevertheless, BDD brings new perspectives to you, one of them is the outside-in testing development. With this approach you can build your software starting with the most external layer, and go deeper until reach unitary tests.

Introducing Lettuce

Lettuce is a very simple BDD tool based on the Cucumber, which currently has many more features than Lettuce.

Lettuce aims the most common tasks on BDD and it focus specially on those that make BDD so fun :)

Lettuce pragma

Provide to the developers the ability of describing features in a natural language, by creating one or more scenarios

Each scenario has one possible behaviour of the feature you want to implement. To make the scenarios run python code, it is necessary to define steps.

Hands on!

This documentation will drive you through all the Lettuce features. When you feel a bit comfortable, go to the first part of the tutorial, or go further on the reference.

Installing Lettuce

Stable release

You can install the latest stable release with pip

user@machine:~$ [sudo] pip install lettuce

Using control version’s HEAD

Otherwise, if you’re a more adventurous developer, you can use the bleeding edge version of Lettuce by taking the git HEAD

If you want so, you have basically 2 options:

Build and install the egg from sources

Good for those that just want to use the latest features

user@machine:~/Downloads$ git clone git://github.com/gabrielfalcao/lettuce.git
user@machine:~/Downloads$ cd lettuce
user@machine:~/Downloads/lettuce$ sudo python setup.py install

Use the latest code to contribute with lettuce’s codebase

If it is your case, I strongly recommend a sandbox:

GNU/Linux:

  1. Fetch the code
user@machine:~/Projects$ git clone git://github.com/gabrielfalcao/lettuce.git
  1. Add to your PYTHONPATH
user@machine:~/Projects$ echo "export PYTHONPATH=$HOME/Projects/lettuce:$PYTHONPATH" >> $HOME/.bashrc
  1. Open a new terminal and enjoy!

What the f** eature ?

Unless you are used to Cucumber nomenclature, you may be wondering about the terms that surround Lettuce concepts.

If this is your case, this introduction will guide you through the very basic keywords that cover Lettuce.

Features

Since Lettuce is used to test the behavior of a project, the behavior is broken up in to features of the system.

After enumerating features, you need to create scenarios which will describe those feature. Thus, scenarios are components of a feature.

Let’s learn by example: suppose we want to create a system to manage a address book.

OK, one of the very basic features of a address book is adding contacts, which will include their name and phone numbers.

This is how Lettuce allows you to describe such feature:

Feature: Add people to address book
  In order to organize phone numbers of friends
  As a wise person
  I want to add a people to my address book

  Scenario: Add a person with name and phone number
    Given I fill the field "name" with "John"
    And fill the field "phone" with "2233-4455"
    When I save the data
    Then I see that my contact book has the persons:
      | name | phone     |
      | John | 2233-4455 |

  Scenario: Avoiding a invalid phone number
    Given I fill the field "name" with "John"
    And fill the field "phone" with "000"
    When I save the data
    Then I get the error: "000 is a invalid phone number"

In the feature above we can notice a few elements, for instance:

  • The feature name:
Feature: Add people to contact book
  • Feature headline:
In order to organize phone numbers of friends
As a wise person
I want to add a people to my address book
  • Scenarios:
Scenario: Add a person with name and phone
  Given I fill the field "name" with "John"
  And fill the field "phone" with "2233-4455"
  When I save the data
  Then I see that my contact book has the persons:
    | name | phone     |
    | John | 2233-4455 |

Scenario: Avoiding a invalid phone number
  Given I fill the field "name" with "John"
  And fill the field "phone" with "000"
  When I save the data
  Then I get the error: "000 is a invalid phone number"

Scenarios

One or more scenarios compose a feature. There are two kinds of scenarios:

Simple

The simple scenarios are composed by steps, no matter if they are simple or tabulated steps.

The feature above is composed by two simple scenarios.

Outlined

Outlined scenarios are very handy because they help you to avoid repetition.

Suppose that we need to fill the same form many times, each time with a different data set. This is how it could be done using scenario outlines:

Let’s see how it could be done with scenario outlines:

Feature: Apply all my friends to attend a conference
  In order to apply all my friends to the next PyCon_
  As a lazy person
  I want to fill the same form many times

  Scenario Outline: Apply my friends
    Go to the conference website
    Access the link "I will attend"
    Fill the field "name" with "<friend_name>"
    Fill the field "email" with "<friend_email>"
    Fill the field "birthday" with "<friend_birthdate>"
    Click on "confirm attendance" button

  Examples:
    | friend_name | friend_email         | friend_birthdate |
    | Mary        | mary@domain.com      | 1988/02/10       |
    | Lincoln     | lincoln@provider.net | 1987/09/10       |
    | Marcus      | marcus@other.org     | 1990/10/05       |

In a nutshell, the scenario above is equivalent to write the huge code bellow

Feature: Apply all my friends to attend a conference
  In order to apply all my friends to the next PyCon_
  As a lazy person
  I want to fill the same form many times

  Scenario: Apply Mary
    Go to the conference website
    Access the link "I will attend"
    Fill the field "name" with "Mary"
    Fill the field "email" with "mary@domain.com"
    Fill the field "birthday" with "1988/02/10"
    Click on "confirm attendance" button

  Scenario: Apply Lincoln
    Go to the conference website
    Access the link "I will attend"
    Fill the field "name" with "Lincoln"
    Fill the field "email" with "lincoln@provider.net"
    Fill the field "birthday" with "1987/09/10"
    Click on "confirm attendance" button

  Scenario: Apply Marcus
    Go to the conference website
    Access the link "I will attend"
    Fill the field "name" with "Marcus"
    Fill the field "email" with "marcus@other.org"
    Fill the field "birthday" with "1990/10/05"
    Click on "confirm attendance" button

As you can notice, scenario outlines are very useful and help you on avoiding text and code repetition

Steps and its definitions

Comparable with Scenarios, Steps comes in two kinds:

Simple steps

Simple steps are actually simple and they are related to the step definitions inside the scenarios.

Lettuce considers each line of a scenario as a simple step. The only exception is if the first non-blank character of the line is a pipe |. In this case, Lettuce will consider the step as a tabular step.

For instance, a simple step may look like this:

Given I go to the conference website

Tabular steps

Analog to Outlined Scenarios, the tabular steps are very useful, and avoid repetition of text.

Tabular steps are specially useful to set up some data set in a scenario, or to compare a set of data to the expected results in the end of the scenario.

However, feel free to use this whenever you find it useful.

Example:

Given I have the following contacts in my database
  | name  | phone      |
  | John  | 2233-4455  |
  | Smith | 9988-7766  |

introduction

Lettuce is an extremely useful and charming tool for BDD (Behavior Driven Development). It can execute plain-text functional descriptions as automated tests for Python projects, just as Cucumber does for Ruby.

Lettuce makes the development and testing process really easy, scalable, readable and - what is best - it allows someone who doesn’t program to describe the behavior of a certain system, without imagining those descriptions will automatically test the system during its development.

_images/flow.png

get lettuce

Make sure you’ve got Python installed and then run from the terminal:

user@machine:~$ [sudo] pip install lettuce

define a problem

Let’s choose a problem to lettuce: Given a number, what is its factorial?

Note

The factorial of a positive integer n, denoted by n!, is the product of all positive integers less than or equal to n. The factorial of 0 is 1

project structure

Build the directory tree bellow such as the files zero.feature and steps.py are empty.

/home/user/projects/mymath
     | tests
           | features
                - zero.feature
                - steps.py

lettuce it!

Lets begin to describe and solve our problem...

first round

[a] describe behaviour

Start describing the expected behaviour of factorial in zero.feature using English:

Feature: Compute factorial
    In order to play with Lettuce
    As beginners
    We'll implement factorial

    Scenario: Factorial of 0
        Given I have the number 0
        When I compute its factorial
        Then I see the number 1

Note

zero.feature must be inside features directory and its extension must be .feature. However, you’re free to choose its name.

[b] define steps in python

Now let’s define the steps of the scenario, so Lettuce can understand the behaviour description. Create the steps.py file which will contain python code describing the steps.

Python:

from lettuce import *

@step('I have the number (\d+)')
def have_the_number(step, number):
    world.number = int(number)

@step('I compute its factorial')
def compute_its_factorial(step):
    world.number = factorial(world.number)

@step('I see the number (\d+)')
def check_number(step, expected):
    expected = int(expected)
    assert world.number == expected, \
        "Got %d" % world.number

def factorial(number):
    return -1

Note

steps.py must be inside features directory, but the names doesn’t need to be steps.py it can be any python file with a .py extension. Lettuce will look for python files recursively within features dir.

Ideally, factorial will be defined somewhere else. However, as this is just a first example, we’ll implement it inside steps.py, so you get the idea of how to use Lettuce.

Notice that, until now, we haven’t defined the factorial function (it’s returning -1).

[c] run and watch it fail

Go to the tests directory and run from the terminal:

user@machine:~/projects/mymath/tests$ lettuce

As you haven’t implemented factorial, it is no surprise the behavior won’t be reached:

_images/screenshot1.png

Our only scenario failed :( Let’s solve it...

[d] write code to make it pass

Well, by definition, we know that the factorial of 0 is 1. As our only feature is this... we could force factorial to return 1.

from lettuce import *

@step('I have the number (\d+)')
def have_the_number(step, number):
    world.number = int(number)

@step('I compute its factorial')
def compute_its_factorial(step):
    world.number = factorial(world.number)

@step('I see the number (\d+)')
def check_number(step, expected):
    expected = int(expected)
    assert world.number == expected, \
        "Got %d" % world.number

def factorial(number):
    return 1
[e] run again and watch it pass

Again, run from the terminal:

user@machine:~/projects/mymath/tests$ lettuce

And you’ll be happy to see your factorial implementation passed all the behaviours expected:

_images/screenshot2.png

Great! :)

However, one test is not enough for checking the quality of our solution... So let’s lettuce it again!

second round

Let’s provide more tests so our problem is better described, and so we provide a more accurate implementation of factorial:

[a] describe behaviour

Let’s provide two new scenarios, for numbers 1 and 2:

Feature: Compute factorial
  In order to play with Lettuce
  As beginners
  We'll implement factorial

  Scenario: Factorial of 0
    Given I have the number 0
    When I compute its factorial
    Then I see the number 1

  Scenario: Factorial of 1
    Given I have the number 1
    When I compute its factorial
    Then I see the number 1

  Scenario: Factorial of 2
    Given I have the number 2
    When I compute its factorial
    Then I see the number 2
[b] define steps in python

As we haven’t changed the definition, no need to make changes on this step.

[c] run and watch it fail
user@machine:~/projects/mymath/tests$ lettuce

When running Lettuce we realize that our previous implementation of factorial works fine both for 0 and for 1, but not for 2 - it fails. :(

_images/screenshot3.png
[d] write code to make it pass

Let’s provide a solution so we get the right factorial for all scenarios, specially for number 2:

from lettuce import *

@step('I have the number (\d+)')
def have_the_number(step, number):
    world.number = int(number)

@step('I compute its factorial')
def compute_its_factorial(step):
    world.number = factorial(world.number)

@step('I see the number (\d+)')
def check_number(step, expected):
    expected = int(expected)
    assert world.number == expected, \
        "Got %d" % world.number

def factorial(number):
    number = int(number)
    if (number == 0) or (number == 1):
        return 1
    else:
        return number
[e] run again and watch it pass
user@machine:~/projects/mymath/tests$ lettuce
_images/screenshot4.png

Great! Three scenarios described and they are alright!

third round

Let’s provide more tests so our problem is better described and we get new errors so we’ll be able to solve them.

[a] describe behaviour
Feature: Compute factorial
  In order to play with Lettuce
  As beginners
  We'll implement factorial

  Scenario: Factorial of 0
    Given I have the number 0
    When I compute its factorial
    Then I see the number 1

  Scenario: Factorial of 1
    Given I have the number 1
    When I compute its factorial
    Then I see the number 1

  Scenario: Factorial of 2
    Given I have the number 2
    When I compute its factorial
    Then I see the number 2

  Scenario: Factorial of 3
    Given I have the number 3
    When I compute its factorial
    Then I see the number 6

  Scenario: Factorial of 4
    Given I have the number 4
    When I compute its factorial
    Then I see the number 24
[b] define steps in python

As we haven’t changed the definition, no need to make changes on this step.

[c] run and watch it fail
user@machine:~/projects/mymath/tests$ lettuce
_images/screenshot5.png
[d] write code to make it pass
from lettuce import *

@step('I have the number (\d+)')
def have_the_number(step, number):
    world.number = int(number)

@step('I compute its factorial')
def compute_its_factorial(step):
    world.number = factorial(world.number)

@step('I see the number (\d+)')
def check_number(step, expected):
    expected = int(expected)
    assert world.number == expected, \
        "Got %d" % world.number

def factorial(number):
    number = int(number)
    if (number == 0) or (number == 1):
        return 1
    else:
        return number*factorial(number-1)
[e] run again and watch it pass
user@machine:~/projects/mymath/tests$ lettuce
_images/screenshot61.png

forth round

All steps should be repeated as long as you can keep doing them - the quality of your software depends on these.

Syntactic sugar

Available for versions > 0.2.19

Steps sentence can now be given by function name or doc.

To take a step sentence from function name or doc, just decorate it with “@step” without argument.

These two steps below, are identicals than the example above.

from lettuce import *

@step
def have_the_number(step, number):
    'I have the number (\d+)'
    world.number = int(number)

@step
def i_compute_its_factorial(step):
    world.number = factorial(world.number)

Steps can be grouped in class decorated with “@steps”

from lettuce import world, steps

@steps
class FactorialSteps(object):
  """Methods in exclude or starting with _ will not be considered as step"""

  exclude = ['set_number', 'get_number']

  def __init__(self, environs):
    self.environs = environs

  def set_number(self, value):
    self.environs.number = int(value)

  def get_number(self):
    return self.environs.number

  def _assert_number_is(self, expected, msg="Got %d"):
      number = self.get_number()
      assert number == expected, msg % number

  def have_the_number(self, step, number):
    '''I have the number (\d+)'''
      self.set_number(number)

  def i_compute_its_factorial(self, step):
      number = self.get_number()
      self.set_number(factorial(number))

  def check_number(self, step, expected):
      '''I see the number (\d+)'''
      self._assert_number_is(int(expected))

# Important!
# Steps are added only when you instanciate the "@steps" decorated class
# Internally decorator "@steps" build a closure with __init__

FactorialSteps(world)

def factorial(number):
    number = int(number)
    if (number == 0) or (number == 1):
        return 1
    else:
        return number*factorial(number-1)

Have a nice lettuce...! ;)

handling data with tables

Let’s imagine writing a MVC application. While writing the tests you will stumble in to a situation where there is a few models that must be added to the database, maybe you will also need to check the new state of those models.

It means that as you write tests with lettuce, it can be very useful to handle data within steps.

Step tables are here for you

Feature: bill students alphabetically
  In order to bill students properly
  As a financial specialist
  I want to bill those which name starts with some letter

  Scenario: Bill students which name starts with "G"
    Given I have the following students in my database:
      | name     | monthly_due | billed |
      | Anton    | $ 500       | no     |
      | Jack     | $ 400       | no     |
      | Gabriel  | $ 300       | no     |
      | Gloria   | $ 442.65    | no     |
      | Ken      | $ 907.86    | no     |
      | Leonard  | $ 742.84    | no     |
    When I bill names starting with "G"
    Then I see those billed students:
      | name     | monthly_due | billed |
      | Gabriel  | $ 300       | no     |
      | Gloria   | $ 442.65    | no     |
    And those that weren't:
      | name     | monthly_due | billed |
      | Anton    | $ 500       | no     |
      | Jack     | $ 400       | no     |
      | Ken      | $ 907.86    | no     |
      | Leonard  | $ 742.84    | no     |

In the example above there are 4 steps, in which 3 contains tables.

Now let us imagine that we’re using Django and write a step definition that uses the table.

from lettuce import step
from school.models import Student

@step('I have the following students in my database:')
def students_in_database(step):
    for student_dict in step.hashes:
        person = Student(**student_dict)
        person.save()

What about handy functions for getting the first or the last row of the tables ?!

from lettuce import step
from school.models import Student

@step('I have the following students in my database:')
def students_in_database(step):
    person1 = Student(**step.hashes.first)
    person2 = Student(**step.hashes.last)

    person1.save()
    person2.save()

Easy, huh?!

Every step has a attribute called hashes which is a list of dicts. Each dict has represents table headers as keys and each table row as value.

In other words, lettuce will translate the table written in the first step as this equivalent dict

@step('I have the following students in my database:')
def students_in_database(step):
    assert step.hashes == [
        {
            'name': 'Anton',
            'monthly_due': '$ 500',
            'billed': 'no'
        },
        {
            'name': 'Jack',
            'monthly_due': '$ 400',
            'billed': 'no'
        },
        {
            'name': 'Gabriel',
            'monthly_due': '$ 300',
            'billed': 'no'
        },
        {
            'name': 'Gloria',
            'monthly_due': '$ 442.65',
            'billed': 'no'
        },
        {
            'name': 'Ken',
            'monthly_due': '$ 907.86',
            'billed': 'no'
        },
        {
            'name': 'Leonard',
            'monthly_due': '$ 742.84',
            'billed': 'no'
        },
    ]

multi-line strings

Now imagine you are writing an application which manipulates strings. When writing the tests, you may find yourself wanting to put multi-line strings in your steps.

Multi-line strings will do the trick

Feature: Split a string into multiple lines on spaces
  In order to make strings more readable
  As a user
  I want to have words split into their own lines

  Scenario: Split small-ish string
    Given I have the string "one two three four five"
    When I ask to have the string split into lines
    Then I should see the following:
      """
      one
      two
      three
      four
      five
      """

A line with nothing but three quotes (“””) is used to indicate the beginning and the end of a multi-line string.

Now, let’s define a step that knows how to use this.

from lettuce import step

@step('I should see the following:')
def i_should_see_the_following(step):
    assert step.multiline == """one
two
three
four
five"""

Nice and straightforward.

Notice that leading spaces are stripped, and there’s not a newline at the beginning or end. This is due to the way that the parser strips blank lines and leading and trailing whitespace.

If you need blank lines leading or trailing whitespace, you can include lines which start and/or end with double quote, and they will be concatenated with the other multiline lines, with the quotes stripped off and their whitespace preserved.

For example

Feature: Split a string into multiple lines on spaces
  In order to make strings more readable
  As a user
  I want to have words split into their own lines

  Scenario: Split small-ish string
    Given I have the string "one two three four five"
    When I ask to have the string split into lines
    Then I should see the following:
      """
     " one
     " two  "
     "  three   "
     "   four    "
     "    five     "
     "
      """

Which we can verify like so:

from lettuce import step

@step('I should see the following:')
def i_should_see_the_following(step):
    assert step.multiline == '\n'.join([
    ' one',
    '  two  ',
    '   three   ',
    '    four    ',
    '     five     ',
    ''])

Admittedly, this is a hack, but there’s no clean way to preserve whitespace in only one section of a feature definition in the current parser implementation.

Note that the first line doesn’t have any whitespace at the end, and thus doesn’t need to have a quote at the end of it.

Also note that if you want a double quote at the beginning of a line in your string, you’ll have to start your line with two double quotes, since the first one will be stripped off.

scenario outlines

On our first description file, zero.feature, all scenarios were similar. This made us repeat most of the text again and again.

Isn’t there a better way to deal with this - when several scenarios are almost equal and only some values change?

Yes, there is! :) You just need to use scenarios outlines.

An example is shown below:

Feature: Compute factorial
  In order to play with Lettuce
  As beginners
  We'll implement factorial

  Scenario Outline: Factorials [0-4]
    Given I have the number <number>
    When I compute its factorial
    Then I see the number <result>

  Examples:
    | number | result |
    | 0      | 1      |
    | 1      | 1      |
    | 2      | 2      |
    | 3      | 6      |
    | 4      | 24     |

This way, you will only need to provide the values that really change, reducing “copy & paste” work and making your tests more clear.

Note

If you overwrite zero.feature using the example above, and goto step [e], you’ll see your description expanding to the five previous scenarios:

_images/screenshot7.png

calling steps from step definitions

Our tests should be as expressive as possible. However, we also want to re-use steps that we’ve seen before. With the tools we’ve used so far, you could end up with seriously long step definitions.

Scenario: Logged-in user does something cool.
  Given I go to the home page
  And I click the login button
  And I fill in username:floppy password:banana
  And I click "Login"
  When I finally do something interesting
  Then I'm already too bored to care.

In this case, we probably had a test case (maybe several) for which it was actually valuable to express how the user interacted with the login form. That’s where we got the step definitions for our login sequence. When the login form isn’t especially interesting any more, however, these steps are just noise. We’d really like to be able to define something like this without duplicating our step definitions.

Scenario: Logged-in user does something cool.
  Given I am logged in
  When I do something interesting
  Then The world becomes a better place

Lettuce affords you the ability to write such a “step of steps” with a set of helpers matching each of the grammar terms Given, When and Then. You could accomplish the above like so.

@step('I am logged in')
def is_logged_in(step):
    step.given('I go to the home page')
    step.given('I click the login button')
    # ... and so on.

running blocks of steps

It is sometimes even desirable to run blocks of steps, copy-and-pasted directly from Feature specifications. The Step.behave_as method lets you do this, and you can use str.format to fill in parameters dynamically. For example, we can write the above step definition like so:

@step('I am logged in')
def is_logged_in(step):
    step.behave_as("""
        Given I go to the home page
          And I click the login button
          And I fill in username:{user} password:{pass}
          And I click "Login"
    """.format(user='floppy', pass='banana'))

This can be combined with step argument capture for step definitions that are both expressive and DRY.

the command line

Lettuce is used as a command line utility, it means that currently the only way to use it is through a shell.

Once in a shell, you can use lettuce in 2 ways:

Which means having the simple features/step_definitions folder somewhere in your project

The difference between them is that within Django you have more options, but both ways have these common options:

running a specific feature file

user@machine:~/projects/myproj$ lettuce path/to/some/file.feature

With this option, your feature can even be out of the default features folder.

running only some scenarios of a specific feature file

user@machine:~/projects/myproj$ lettuce path/to/some/file.feature -s 3,5,9

This will run the scenarios 3, 5 and 9 from file path/to/some/file.feature

running only some scenarios all feature files

Maybe you can find it senseless, but it works like that, and does not hurt so far :)

user@machine:~/projects/myproj$ lettuce -s 3,5,9

Yeah, guess what?

This command will run the scenarios 3, 5 and 9 of all feature files living on myproj/features folder.

verbosity levels

level 1 - dots for each feature
user@machine:~/projects/myproj$ lettuce --verbosity=1

This is lettuce’s minimum verbosity level. It shows dots for each step run, regardless of what scenario or what feature is currently running.

For example, if you have a feature that looks like:

Feature: Manipulate strings
  Scenario: Uppercased strings
    Given I have the string "lettuce leaves"
    When I put it in upper case
    Then I see the string is "LETTUCE LEAVES"

The output will be:

user@machine:~/projects/myproj$ lettuce -v 1
...

1 feature (1 passed)
1 scenario (1 passed)
3 steps (3 passed)
level 2 - scenario names
user@machine:~/projects/myproj$ lettuce --verbosity=2

In this mode, lettuce will print each scenario name that is currently being ran, followed by OK, FAILED or ERROR depending of the status of the steps within that scenario.

For example, if you have a feature that looks like:

Feature: Manipulate strings
  Scenario: Uppercased strings
    Given I have the string "lettuce leaves"
    When I put it in upper case
    Then I see the string is "LETTUCE LEAVES"

  Scenario: basic math
    Given I sum 2 and 5
    Then I see the result is 9

The output will be:

user@machine:~/projects/myproj$ lettuce -v 2
Uppercased strings ... OK
basic math ... FAILED

1 feature (1 passed)
2 scenarios (2 passed)
5 steps (4 passed)
level 3 - full feature print, but colorless
user@machine:~/projects/myproj$ lettuce --verbosity=3

This mode is a lot more verbose than the later one. It prints every single feature, with really useful information like:

  • the relative path to the feature file being ran, and the current line in that file
  • the relative path to the step definition responsible for the step being ran, also followed by the current line
  • inline tracebacks when some feature fails
  • “ready-to-use” snippets for undefined steps

For example, let’s say you have the feature below, but only the step Given I have the string "lettuce leaves" is defined

Feature: Manipulate strings
  Scenario: Uppercased strings
    Given I have the string "lettuce leaves"
    When I put it in upper case
    Then I see the string is "LETTUCE LEAVES"

Your output will look like:

user@machine:~/projects/myproj$ lettuce -v 2

Feature: Manipulate strings                   # features/strings.feature:1

  Scenario: Uppercased strings                # features/strings.feature:2
    Given I have the string "lettuce leaves"  # features/step_definitions/example-steps.py:5
    When I put it in upper case               # features/strings.feature:4 (undefined)
    Then I see the string is "LETTUCE LEAVES" # features/strings.feature:5 (undefined)

1 feature (0 passed)
1 scenario (0 passed)
3 steps (2 undefined, 1 passed)

You can implement step definitions for undefined steps with these snippets:

# -*- coding: utf-8 -*-
from lettuce import step

@step(u'When I put it in upper case')
def when_i_put_it_in_upper_case(step):
    assert False, 'This step must be implemented'
@step(u'Then I see the string is "(.*)"')
def then_i_see_the_string_is_group1(step, group1):
    assert False, 'This step must be implemented'
level 4 - full feature print, but colorful

This mode is almost exactly the same of level 3, the difference is that it’s colorful.

_images/screenshot6.png

Note

If you are going to put lettuce running in a Continuous-Integration server, like Hudson. You may choose the levels 1, 2 or 3, so that the output won’t look messy.

integrating with continuous integration

Lettuce can use Subunit to output test results. Subunit is a stream format that can be multiplexed, viewed in real time or converted to many different formats (such as xUnit/jUnit XML format).

user@machine:~/projects/myproj$ lettuce --with-subunit > output.log
user@machine:~/projects/myproj$ subunit2junitxml < subunit.bin > lettucetests.xml

The –subunit-file flag can be used to specify a filename other than subunit.bin this is important if you’re combining test runs.

including coverage

You can also get test coverage information using the coverage package.

user@machine:~/projects/myproj$ coverage run lettuce --with-subunit
user@machine:~/projects/myproj$ coverage xml

getting help from shell

user@machine:~/projects/myproj$ lettuce -h

Shows all the options described here.

features, scenarios and steps reference

Features, scenarios and steps are python objects within lettuce’s feature engine.

Here you will find out very “hacky” details about those objects. If you stumbled here looking for a introduction, it might be a good idea to read the feature tutorial for a introduction.

In order to exemplify the usage of attributes and methods below, let’s consider that there is a feature in a file called some.feature

# language: en
# just a comment

# another one
Feature: some feature
  Here comes
  The feature
  Description

  Scenario: try out something
    Given I show lettuce running
    Then I should be happy

Feature

Feature.name

A string containing the name of the feature

feature.name == 'some feature'

Feature.scenarios

A list of scenario objects

The attribute scenarios could be used as follows

feature.scenarios[0].name == 'try out something'

Feature.described_at

A FeatureDescription object, has the file and line which the feature was described. Lettuce uses it to output those metadata.

The attribute described_at could be used as follows

# the line in which the feature started
feature.described_at.line == 5

# the filename path
'some.feature' in feature.described_at.file

# a tuple with the lines that contains the feature description
feature.described_at.description_at == (6, 7, 8)

Feature.max_length

A property that calculates the maximum length of all lines that built the feature.

Mostly used by shell output to find out where to print the feature description.

Example:

feature.max_length == 21

Feature.get_head

Does represent the feature with its first representation in current language followed by a colon and the feature name.

Example:

feature.get_head() == 'Feature: some feature'

But if the same feature would written in Brazilian Portuguese, for example:

# language: pt-br
# apenas um comentário

# e outro
Funcionalidade: alguma funcionalidade
  Aqui vem
  a descrição
  da funcionalidade

  Cenário: ...
    ...

Then, Feature.get_head() would give:

feature.get_head() == 'Funcionalidade: alguma funcionalidade'

TotalResult

TotalResult.features_ran

Integer, the total of features ran

TotalResult.features_passed

Integer, the total of features passed

TotalResult.scenarios_ran

Integer, the total of scenarios ran

TotalResult.scenarios_passed

Integer, the total of scenarios passed

TotalResult.steps

Integer, the number of steps that were supposed to run

TotalResult.proposed_definitions

A list of Step that have no step definition

Scenario

Scenario.steps

A list of scenario objects

The attribute scenarios could be used as follows

scenario.steps[0].sentence == 'try out something'

Step

Step.sentence

The string that represents the step

step.sentence == 'Given I show lettuce running'

Step.passed

Boolean, true if the step ran without an error.

Step.failed

Boolean, true if the step ran and an error occurred during execution.

step definition

A decorator that can be used on any python function, takes a regex string as parameter, so that the function can me matched against steps.

from lettuce import step

@step('I am (happy|sad)')
def show_lettuce_running_here(step, action):
    if action == 'happy':
        return # everything is fine!

    else:
        assert False, 'you should be happy, dude!'

the “terrain”

Terrain is a “pun” with lettuce and its “living place”, its about setup and teardown, and general hacking on your lettuce tests.

terrain.py

By convention lettuce tries do load a file called terrain.py located at the current directory.

Think at this file as a global setup place, there you can setup global hooks, and put things into lettuce “world”.

Note

You can also set a terrain.py file within the root of your Django project, when running the python manage.py harvest command, lettuce will load it. See more at the django command.

in practice

Try out this file layout:

/home/user/projects/some-project
       | features
            - the-name-of-my-test.feature
            - the-file-which-holds-step-definitions.py
            - terrain.py

Then add some setup at terrain.py and run lettuce

user@machine:~/projects/some-project$ lettuce

And notice terrain.py will be loaded before anything

world

For the sake of turning easier and funnier to write tests, lettuce “violates” some principles of good design in python, such as avoiding implicity and using global stuff.

The “world” concept of lettuce is mostly about “global stuff”.

in practice

Imagine a file located somewhere that will be imported by your application before lettuce start running tests:

from lettuce import world

world.some_variable = "yay!"

So that, within some step file you could use things previously set on world:

from lettuce import *

@step(r'exemplify "world" by seeing that some variable contains "(.*)"')
def exemplify_world(step, value):
    assert world.some_variable == value

And the feature could have something like:

Feature: test lettuce's world
  Scenario: check variable
    When I exemplify "world" by seeing that some variable contains "yay!"

world.absorb

It can be really useful to put functions and/or classes in lettuce.world

For example:

from lettuce import world

def my_project_wide_function():
    # do something

world.my_project_wide_function = my_project_wide_function

world.my_project_wide_function()

But as you can notice, as your project grows, there can be a lot of repetitive lines, not DRY at all :(

In order to avoid that, lettuce provides a “absorb” decorator that lives within “world”

Let’s see it in action:

from lettuce import world

@world.absorb
def my_project_wide_function():
    # do something

world.my_project_wide_function()

You can also use it with classes:

from lettuce import world

@world.absorb
class MyClass:
    pass

assert isinstance(world.MyClass(), MyClass)

And even with lambdas, but in this case you need to name it

from lettuce import world

world.absorb(lambda: "yeah", "optimist_function")

assert world.optimist_function() == 'yeah'

world.spew

Well, if you read the topic above, you may be guessing: “if I keep stashing things in lettuce.world, it may bloat it sometime, or confuse member names along my steps, or hooks.

For those cases after “absorbing” something, world can also “spew” it.

from lettuce import world

@world.absorb
def generic_function():
    # do something

assert hasattr(world, 'generic_function')

world.spew('generic_function')

assert not hasattr(world, 'generic_function')

hooks

Lettuce has hooks that are called sequentially before and after each action

Presented as python decorators, it can be used to take any actions you find useful.

For example, you can set a browser driver at world, and close the connection after all, populate database with test mass or anything you want, for example

Let’s see it from outside in

@before.all

This hook is ran before lettuce look for and load feature files

The decorated function takes NO parameters

from lettuce import *

@before.all
def say_hello():
    print "Hello there!"
    print "Lettuce will start to run tests right now..."

@after.all

This hook is ran after lettuce run all features, scenarios and steps

The decorated function takes a TotalResult as parameter, so that you can use the result statistics somehow

from lettuce import *

@after.all
def say_goodbye(total):
    print "Congratulations, %d of %d scenarios passed!" % (
        total.scenarios_ran,
        total.scenarios_passed
    )
    print "Goodbye!"

@before.each_feature

This hook is ran before lettuce run each feature

The decorated function takes a Feature as parameter, so that you can use it to fetch scenarios and steps inside.

from lettuce import *

@before.each_feature
def setup_some_feature(feature):
    print "Running the feature %r, at file %s" % (
        feature.name,
        feature.described_at.file
    )

@after.each_feature

This hooks behaves in the same way @before.each_feature does, except by the fact that its ran after lettuce run the feature.

from lettuce import *

@after.each_feature
def teardown_some_feature(feature):
    print "The feature %r just has just ran" % feature.name

@before.each_scenario

This hook is ran before lettuce run each scenario

The decorated function takes a Scenario as parameter, so that you can use it to fetch steps inside.

from lettuce import *
from fixtures import populate_test_database

@before.each_scenario
def setup_some_scenario(scenario):
    populate_test_database()

@after.each_scenario

This hooks behaves in the same way @before.each_scenario does, except by the fact that its ran after lettuce run the scenario.

from lettuce import *
from database import models
@after.each_scenario
def teardown_some_scenario(scenario):
    models.reset_all_data()

@before.each_step

This hook is ran before lettuce run each step

The decorated function takes a Step as parameter, so that you can use it to fetch tables and so.

from lettuce import *

@before.each_step
def setup_some_step(step):
    print "running step %r, defined at %s" % (
        step.sentence,
        step.defined_at.file
    )
@after.each_step

This hooks behaves in the same way @before.each_step does, except by the fact that its ran after lettuce run the step.

from lettuce import *

@after.each_step
def teardown_some_step(step):
    if not step.hashes:
       print "no tables in the step"

django-specific hooks

Since lettuce officially supports Django, there are a few specific hooks that help on setting up your test suite on it.

@before.harvest

This hook is ran before lettuce start harvesting your Django tests. It can be very useful for setting up browser drivers (such as selenium), before all tests start to run on Django.

The decorated function takes a dict with the local variables within the harvest management command.

from lettuce import *
from lettuce.django import django_url
from selenium import selenium

@before.harvest
def prepare_browser_driver(variables):
    if variables.get('run_server', False) is True:
        world.browser = selenium('localhost', 4444, '*firefox', django_url('/'))
        world.browser.start()

@after.harvest

This hook is ran right after lettuce finish harvesting your Django tests. It can be very useful for shutting down previously started browser drivers (see the example above).

The decorated function takes a list of TotalResult objects.

from lettuce import *

@after.harvest
def shutdown_browser_driver(results):
    world.browser.stop()

@before.each_app

This hook is ran before lettuce run each Django app.

The decorated function takes the python module that corresponds to the current app.

from lettuce import *

@before.each_app
def populate_blog_database(app):
    if app.__name__ == 'blog':
        from blog.models import Post
        Post.objects.create(title='Nice example', body='I like writting!')

@after.each_app

This hook is ran after lettuce run each Django app.

The decorated function takes two arguments:

  • the python module that corresponds to the current app.
  • a TotalResult as parameter, so that you can use the result statistics somehow
from lettuce import *

@after.each_app
def clear_blog_database_if_successful(app, result):
    if app.__name__ == 'blog':
        if result.scenarios_ran is result.scenarios_passed:
            from blog.models import Post, Comment
            Comment.objects.all()
            Post.objects.all()

@before.runserver and @after.runserver

These hooks are ran right before, and after lettuce starts up the built-in http server.

The decorated function takes a lettuce.django.server.ThreadedServer object.

from lettuce import *
from django.core.servers.basehttp import WSGIServer

@before.runserver
def prepare_database(server):
    assert isinstance(server, WSGIServer)
    import mydatabase
    mydatabase.prepare()

@after.runserver
def say_goodbye(server):
    assert isinstance(server, WSGIServer)
    print "goodbye, see you soon"

@before.handle_request and @after.handle_request

These hooks are ran right before, and after lettuce’s built-in HTTP server responds to a request.

Both decorated functions takes these two arguments:

  • a django.core.servers.basehttp.WSGIServer object.
  • a lettuce.django.server.ThreadedServer object.
from lettuce import *
from django.core.servers.basehttp import WSGIServer

@before.handle_request
def print_request(httpd, server):
    socket_object, (client_address, size) = httpd.get_request()
    print socket_object.dup().recv(size)

@after.handle_request
def say_goodbye(httpd, server):
    socket_object, (client_address, size) = httpd.get_request()
    print "I've just finished to respond to the client %s" % client_address

Warning

all the handle_request hooks are run within a python thread. If something went wrong within a calback, lettuce can get stuck.

language support

Lettuce currently supports 18 languages:

  • English
  • Portuguese (Português)
  • Polish (Polski)
  • Catalan (Català)
  • Spanish (Español)
  • Hungarian (Magyar)
  • French (Français)
  • German (Deutsch)
  • Japanese (日本語)
  • Turkish (Türkçe)
  • Simplified Chinese (简体中文)
  • Traditional Chinese (繁體中文)
  • Russian (Русский)
  • Ukrainian (Українська)
  • Italian (Italiano)
  • Norwegian (Norsk)
  • Swedish (Svenska)
  • Czech (Čeština)

Although it’s only about writing tests since the current version does output only in English.

writing features in a specific language

You can tell lettuce the language of a feature file through adding a comment in the first line of the file, using the following syntax:

# language: <code>

english example

# language: en
Feature: write features in english
   Scenario: simple scenario
      Given I write a file which starts with "# language: en"
      Then it must be parsed with proper english keywords

brazilian portuguese example

# language: pt-br
Funcionalidade: escrever funcionalidades em português
   Cenário: cenário simples
      Dado que eu crio um arquivo que começa com "# language: pt-br"
      Então ele deve ser interpretado com as devidas palavras-chave brasileiras

adding support to other languages

We love contribution, so if you want to bring lettuce to your native language there is a single and simple way.

fetch the code

First of all, you must have git control version installed in your machine.

Once you have it installed, grab the code with

user@machine:~$ git clone git://github.com/gabrielfalcao/lettuce.git

And edit the file located at:

lettuce/languages.py

And add a new dictionary entry for your native language.

Let’s see the Brazilian Portuguese translation to exemplify.

LANGUAGES = {
    'pt-br': {
        'examples': u'Exemplos|Cenários',
        'feature': u'Funcionalidade',
        'name': u'Portuguese',
        'native': u'Português',
        'scenario': u'Cenário|Cenario',
        'scenario_outline': u'Esquema do Cenário|Esquema do Cenario',
        'scenario_separator': u'(Esquema do Cenário|Esquema do Cenario|Cenario|Cenário)',
    },
}

The key of the dict will be used as identifier for the comment # language: identifier at feature files.

The value must be a dict, where the keys are canonical representation of keywords (string), and the values must be a pipe-separated string with translation possibilities.

It allows different translations for the same keyword in the current language, which offers many possibilities for different semantical cases.

For example, when using scenario outlines, it can be semantically nicer to write:

Scenarios:
   | name | age |
   | John | 22  |
   | Mary | 53  |

Instead of:

Examples:
   | name | age |
   | John | 22  |
   | Mary | 53  |

add your translation

Now you can add your own language to lettuce, save the languages.py file and commit in the source control with.

For example, let’s suppose that you’ve added Spanish support:

user@machine:~/lettuce$ git commit lettuce/languages.py -m 'adding translation for spanish'

Generate a patch:

user@machine:~/lettuce$ git format patch HEAD^1

And send to lettuce’s ticket tracker as a gist or something like it.

Built-in Django Steps

Lettuce features a number of built-in steps for Django to simplify the creation of fixtures.

creating fixture data

Lettuce can automatically introspect your available Django models to create fixture data, e.g.

Background:
    Given I have options in the database:
        | name    | value |
        | Lettuce | Rocks |

This will find a model whose verbose name is options. It will then create objects for that model with the parameters specified in the table (i.e. name=Lettuce, value=Rocks).

You can also specify relational information. Assuming a model Profile with foreign key user and field avatar:

Background:
    Given user with username "harvey" has profile in the database:
        | avatar  |
        | cat.jpg |

To create many-to-many relationships, assuming User has and belongs to many Group objects:

Background:
    Given user with username "harvey" is linked to groups in the database:
        | name |
        | Cats |

For many-to-many relationship to be created, both models must exist prior to linking.

Most common data can be parsed, i.e. true/false, digits, strings and dates in the form 2013-10-30.

registering your own model creators

For more complex models that have to process or parse data you can write your own creating steps using the creates_models decorator.

from lettuce.django.steps.models import (creates_models,
                                         reset_sequence,
                                         hashes_data)

@creates_models(Note)
def create_note(step):
    data = hashes_data(step)
    for row in data:
        # convert the author into a user object
        row['author'] = get_user(row['author'])
        Note.objects.create(**row)

    reset_sequence(Note)

testing models

Two steps exist to test models.

Then features should be present in the database:
    | name    | value |
    | Lettuce | Rocks |
And there should be 1 feature in the database

You can also test non-database model attributes by prefixing an @ to the attribute name. Non-database attributes are tested after the records are selected from the database.

Then features should be present in the database:
    | name    | value | @language |
    | Lettuce | Rocks | Python    |

registering your own model testers

For more complex tests that have to process or parse data you can write your own creating steps using the checks_existence decorator.

settings.py variables

LETTUCE_APPS – apps to test by default

LETTUCE_USE_TEST_DATABASE – use a test database instead of the live database. Equivalent of -T flag.

other considered variables

SOUTH_TESTS_MIGRATE – apply a south migration to the test database

Lettuce recipe: Using nose for pretty assertions

Lettuce uses python’s builtin exception AssertionError to mark tests as failed.

Although in order to describe the assertion with a custom string you would need to do something like:

from lettuce import step

@step('some step with "(.*)"'):
def some_step(step, from):
    assert from == 'expectation', \
        "Ooops, '%s' should be equal 'expectation', but isn't" % from

nose is a python module that provides a set of assert functions that already have a nice description, and fortunately it still uses AssertionError, which makes nose totally compliant with lettuce.

The example below shows how the step above could be written taking advantage of nose:

from lettuce import step
from nose.tools import assert_equals

@step('some step with "(.*)"'):
def some_step(step, from):
    assert_equals(from, 'expectation')

It rocks, huh?!

Web development fun with Lettuce and Django

Django is a awesome web framework, very mature, aims for simplicity and the best of all: it’s fun to use it.

To make it even more fun, lettuce has built-in support for Django.

Getting started

1. install the lettuce django app

Pick up any Django project, and add lettuce.django in its settings.py configuration file:

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.admin',

    # ... other apps here ...
    'my_app',
    'foobar',
    'another_app',
    'lettuce.django', # this guy will do the job :)
)

Considering the configuration above, let’s say we want to write tests for the my_app django application.

2. create the feature directories

Lettuce will look for a features folder inside every installed app:

/home/user/projects/djangoproject
     | settings.py
     | manage.py
     | urls.py
     | my_app
           | features
                - index.feature
                - index.py
     | foobar
           | features
                - carrots.feature
                - foobar-steps.py
     | another_app
           | features
                - first.feature
                - second.feature
                - many_steps.py

3. write your first feature

@index.feature:

Feature: Rocking with lettuce and django

    Scenario: Simple Hello World
        Given I access the url "/"
        Then I see the header "Hello World"

    Scenario: Hello + capitalized name
        Given I access the url "/some-name"
        Then I see the header "Hello Some Name"

@index-steps.py:

from lettuce import *
from lxml import html
from django.test.client import Client
from nose.tools import assert_equals

@before.all
def set_browser():
    world.browser = Client()

@step(r'I access the url "(.*)"')
def access_url(step, url):
    response = world.browser.get(url)
    world.dom = html.fromstring(response.content)

@step(r'I see the header "(.*)"')
def see_header(step, text):
    header = world.dom.cssselect('h1')[0]
    assert header.text == text

4. run the tests

Once you install the lettuce.django app, the command harvest will be available:

user@machine:~projects/djangoproject $ python manage.py harvest

The harvest command executes the django.test.utils.setup_test_environment function before it starts up the Django server. Typically, invoking this function would configure Django to use the locmem in-memory email backend. However, Lettuce uses a custom Django email backend to support retrieving email from Lettuce test scripts. See Checking email for more details.

5. specifying feature files

The harvest command accepts a path to feature files, in order to run only the features you want.

Example:

user@machine:~projects/djangoproject $ python manage.py harvest path/to/my-test.feature

6. grab actual example code

In order to assure that lettuce integrate well with Django, it have a set of integration tests, there are a actual Django project running with lettuce.

You can grab the code at the alfaces folder of lettuce git repository

Technical details

If you want to write acceptance tests that run with web browsers, you can user tools like twill, selenium, webdriver and windmill

red-tape-less builtin server

Lettuce cleverly runs an instance of the built-in Django HTTP server in the background. It tries to bind the HTTP server at localhost:8000 but if the port is busy, it keeps trying to run in higher ports: 8001, 8002 and so on until it reaches the maximum port number 65535.

Note

You can override the default starting port from “8000” to any other port you want.

To do so, refer to “running the HTTP server in other port than 8000” below.

So that you can use browser-based tools such as those listed above to access Django.

Warning

When running the http server, lettuce sets the environment variables SERVER_NAME and SERVER_PORT. It was brought for a GAE issue. If it can possibly bring any errors, be warned.

figure out django urls

As the Django HTTP server can be running in any port within the range 8000 - 65535, it could be hard to figure out the correct URL for your project, right?

Wrong!

Lettuce is here for you. Within your steps you can use the django_url utility function:

from lettuce import step, world
from lettuce.django import django_url

@step(r'Given I navigate to "(.*)"')
def navigate_to_url(step, url):
    full_url = django_url(url)
    world.browser.get(full_url)
what does django_url do ?!?

It prepends a Django-internal URL with the HTTP server address.

In other words, if lettuce binds the http server to localhost:9090 and you call django_url with "/admin/login":

from lettuce.django import django_url
django_url("/admin/login")

It returns:

"http://localhost:9090/admin/login"

terrain also available in django projects

At this point you probably know how terrain.py works, and it also works with Django projects.

You can setup environment and stuff like that within a terrain.py file located at the root of your Django project.

Taking the very first example of this documentation page, your Django project layout would like like this:

/home/user/projects/djangoproject
     | settings.py
     | manage.py
     | urls.py
     | terrain.py
     | my_app
           | features
                - index.feature
                - index.py
     | foobar
           | features
                - carrots.feature
                - foobar-steps.py
     | another_app
           | features
                - first.feature
                - second.feature
                - many_steps.py

Notice the terrain.py file at the project root, there you can populate the world and organize your features and steps with it :)

Checking email

When you run your Django server under lettuce, emails sent by your server do not get transmitted over the Internet. Instead, these emails are added to a multiprocessing.Queue object at lettuce.django.mail.queue.

Example:

from lettuce import step
from lettuce.django import mail
from nose.tools import assert_equals


@step(u'an email is sent to "([^"]*?)" with subject "([^"]*)"')
def email_sent(step, to, subject):
    message = mail.queue.get(True, timeout=5)
    assert_equals(message.subject, subject)
    assert_equals(message.recipients(), [to])

Running without HTTP server

Sometimes you may just do not want to run Django’s built-in HTTP server running in background, in those cases all you need to do is run the harvest command with the --no-server or -S option.

Example:

python manage.py harvest --no-server
python manage.py harvest -S

running the HTTP server in other port than 8000

If you face the problem of having lettuce running on port 8000, you can change that behaviour.

Before running the server, lettuce will try to read the setting LETTUCE_SERVER_PORT which must be a integer

Example:

LETTUCE_SERVER_PORT = 7000

This can be really useful if 7000 is your default development port, for example.

running the HTTP server with settings.DEBUG=True

In order to run tests against the nearest configuration of production, lettuce sets up settings.DEBUG=False

However, for debug purposes one can face a misleading HTTP 500 error without traceback in Django. For those cases lettuce provides the --debug-mode or -d option.

python manage.py harvest --debug-mode
python manage.py harvest -d

using the test database

If you want to use a test database by default, instead of a live database, with your test server you can specify the -T flag or set the following configuration variable in settings.py.

LETTUCE_USE_TEST_DATABASE = True

running only the specified scenarios

You can also specify the index of the scenarios you want to run through the command line, to do so, run with --scenarios or -s options followed by the scenario numbers separated by commas.

For example, let’s say you want to run the scenarios 4, 7, 8 and 10:

python manage.py harvest --scenarios=4,7,8,10
python manage.py harvest -s 4,7,8,10

to run or not to run? That is the question!

During your development workflow you may face two situations:

running tests from just certain apps

Lettuce takes a comma-separated list of app names to run tests against.

For example, the command below would run ONLY the tests within the apps myapp and foobar:

python manage.py harvest --apps=myapp,foobar

# or

python manage.py harvest --a  myapp,foobar

You can also specify it at settings.py so that you won’t need to type the same command-line parameters all the time:

LETTUCE_APPS = (
    'myapp',
    'foobar',
)
INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.admin',
    'my_app',
    'foobar',
    'another_app',
    'lettuce.django',
)
running tests from all apps, except by some

Lettuce takes a comma-separated list of app names which tests must NOT be ran.

For example, the command below would run ALL the tests BUT those within the apps another_app and foobar:

python manage.py harvest --avoid-apps=another_app,foobar

You can also specify it at settings.py so that you won’t need to type the same command-line parameters all the time:

LETTUCE_AVOID_APPS = (
    'another_app',
    'foobar',
)

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.admin',
    'my_app',
    'foobar',
    'another_app',
    'lettuce.django',
)

Development guidelines

Synopsis

  1. fork and clone the project: see lettuce repository at github.com [1]
  2. Install a development environment for lettuce
  3. run the tests: see Testing
  4. hack at will
  5. commit, push, etc...
  6. send a pull request

Table of contents

Install a development environment for lettuce

Here are guidelines to get a development environment for lettuce.

OS specific

Here are repcipes for specific operating systems. They should help you go fast or automate lettuce installation procedure:

Installation on Debian Squeeze

Recipe to get a development environment for lettuce in a fresh install of Debian Squeeze.

Variables

The following values are used below. You may customize them depending on your needs.

# Lettuce installation directory.
lettuce_dir=~/lettuce
# Virtualenv directory.
lettuce_env_dir=$lettuce_dir
# Git.
upstream_url="https://github.com/gabrielfalcao/lettuce.git"
fork_url=$upstream_url
# System's package manager.
system-install() { su -c "aptitude install ${*}" }
Install system dependencies

Execute the following commands:

system-install python-dev python-virtualenv git libxml2-dev libxslt-dev
Get sources
git clone $fork_url $lettuce_dir
# Configure upstream
cd $lettuce_dir
git remote add upstream $upstream_url
Create virtualenv
virtualenv --distribute --no-site-packages $lettuce_env_dir
source $lettuce_env_dir/bin/activate
cd $lettuce_dir
pip install -r requirements.txt
Install lettuce in develop mode
python setup.py develop
Check installation

You should be able to run lettuce and tests.

lettuce --help
Done!

Go back to Development guidelines and learn about Testing.

Generic guidelines
Dependencies

you will need to install these dependencies in order to hack lettuce :)

All of them are used within lettuce tests.

you could use a virtualenv
mkvirtualenv lettuce
workon lettuce
pip install -r requirements.txt
or just install manually
sudo pip install -r requirements.txt
or do it really from scratch
Installing lettuce itself
[sudo] python setup.py develop

Testing

How to run and write tests for lettuce.

Run tests
make unit functional integration doctest

Contributing to the documentation

This documentation uses Python-sphinx [1]. It uses reStructuredText [2] syntax.

Conventions
Language

The documentation is written in english.

Line length

Limit all lines to a maximum of 79 characters.

Headings

Use the following symbols to create headings: #  *  =  -

As an example:

##################
H1: document title
##################

*********
Sample H2
*********

Sample H3
=========

Sample H4
---------

And sample text.

If you need more than H4, then consider creating a new document.

Code blocks

Combine a “highlight” directive and a ”::” to specify the programming language. As an example:

.. highlight:: python

::

  import this

Warning

“code-block” and “sourcecode” directives are not natively supported by rst2html, which is used by Github’s repository browser, i.e. those lines won’t appear on Github. Whereas ”::” lines will.

Install Sphinx

Python-sphinx [1] installation is covered in Install a development environment for lettuce.

In other cases, please refer to Python-sphinx [1] documentation.

Export documentation to HTML
  • Install Python-sphinx [1].

  • Make sure sphinx-build is in your shell’s $PATH. If you are using virtualenv as told in Install a development environment for lettuce, then activate your virtual environment.

  • Go to lettuce folder and use the provided Makefile:

    make documentation
    
  • HTML documentation is exported to docs/_build/html/.

Use doctests!

This documentation uses the Sphinx’s doctest extension [3].

Write doctests

Here is a RST code sample to write doctests. You can find some doctests in the “terrain”.

.. highlight:: python

.. doctest::

   >>> print "Hello world!"
   Hello world!

See Sphinx’s doctest extension [3] and Python’s doctest [4] documentations for details.

Run doctests

Go to lettuce folder and use the provided Makefile:

make doctests

Keep in mind

your lack of tests if disturbing the force

that lettuce is a testing software, patches and pull requests must come with automated tests, and if suitable, with proper documentation.

Indices, glossary and tables