Lettuce documentation contents¶
Lettuce is a Behavior-Driven Development tool written by Gabriel Falcão G. de Moura
nutshell¶
install it
user@machine:~$ [sudo] pip install lettuce
describe your first feature
Feature: Manipulate strings
In order to have some fun
As a programming beginner
I want to manipulate strings
Scenario: Uppercased strings
Given I have the string "lettuce leaves"
When I put it in upper case
Then I see the string is "LETTUCE LEAVES"
define its steps
>>> from lettuce import *
>>> @step('I have the string "(.*)"')
... def have_the_string(step, string):
... world.string = string
...
>>> @step
... def i_put_it_in_upper_case(step):
... world.string = world.string.upper()
...
>>> @step
... def see_the_string_is(step, expected):
... '''I see the string is "(.*)"'''
... assert world.string == expected, \
... "Got %s" % world.string
watch it pass
user@machine:~/Projects/my-project$ lettuce features/
getting involved !¶
github project page
Fork it, propose features, explore the code
github.com/gabrielfalcao/lettuce.
feedback
discuss
hack
donate
hands on!¶
Is this your first experience with Lettuce ?!?
So, why don’t you go straight to the quick start tutorial ?!
walkthrough¶
integrate!¶
- Lettuce and Django, for the sake of web development fun
furthermore¶
reference and concepts
- the command line, how to run lettuce with different verbosity levels, and other cli options
- features, scenarios and steps, diving into lettuce’s core
- terrain, world and hooks, stuff about setting up a environment for lettuce
- language support
- django steps
recipes¶
make your own salad
Warning
Disclaimer on unstable features you should NOT rely on.
For instance use the features step.behave_as and the xunit XML output are unstable and untested.
Lettuce is a software for testing other software, but I’ve been receiving some contributions of new features that are coming without proper tests.
These contributions have to be rewritten with proper tests, so that they won’t break so often, but they are a current source of issues in the bug tracker.
So until someone contributes with unit and functional tests for those features, or I find some time to do it myself, you should avoid relying on them.
Sincerely, Gabriel Falcão
Overview¶
On BDD¶
Behaviour-driven development is a very good approach for maintaining the workflow plain, so you only spend time with what really matters: business value.
The common BDD approach basically consists in:
- writing some unit tests
- running these tests
- making these tests fail
- writing code
- making the code pass these tests (green status)
This is a very awesome practice, since you can build huge and reliable software without fearing the future. You don’t need to worry if those millions of lines of code won’t make sense in 10 years, as long as they keep passing the tests. Despite BDD, other kind of tests are very important and usually follow a similar workflow: functional, integration and acceptance.
Nevertheless, BDD brings new perspectives to you, one of them is the outside-in testing development. With this approach you can build your software starting with the most external layer, and go deeper until reach unitary tests.
Introducing Lettuce¶
Lettuce is a very simple BDD tool based on the Cucumber, which currently has many more features than Lettuce.
Lettuce aims the most common tasks on BDD and it focus specially on those that make BDD so fun :)
Lettuce pragma¶
Provide to the developers the ability of describing features in a natural language, by creating one or more scenarios
Each scenario has one possible behaviour of the feature you want to implement. To make the scenarios run python code, it is necessary to define steps.
Hands on!¶
This documentation will drive you through all the Lettuce features. When you feel a bit comfortable, go to the first part of the tutorial, or go further on the reference.
Installing Lettuce¶
Stable release¶
You can install the latest stable release with pip
user@machine:~$ [sudo] pip install lettuce
Using control version’s HEAD¶
Otherwise, if you’re a more adventurous developer, you can use the bleeding edge version of Lettuce by taking the git HEAD
If you want so, you have basically 2 options:
Build and install the egg from sources¶
Good for those that just want to use the latest features
user@machine:~/Downloads$ git clone git://github.com/gabrielfalcao/lettuce.git
user@machine:~/Downloads$ cd lettuce
user@machine:~/Downloads/lettuce$ sudo python setup.py install
Use the latest code to contribute with lettuce’s codebase¶
If it is your case, I strongly recommend a sandbox:
GNU/Linux:
- Fetch the code
user@machine:~/Projects$ git clone git://github.com/gabrielfalcao/lettuce.git
- Add to your PYTHONPATH
user@machine:~/Projects$ echo "export PYTHONPATH=$HOME/Projects/lettuce:$PYTHONPATH" >> $HOME/.bashrc
- Open a new terminal and enjoy!
What the f** eature ?¶
Unless you are used to Cucumber nomenclature, you may be wondering about the terms that surround Lettuce concepts.
If this is your case, this introduction will guide you through the very basic keywords that cover Lettuce.
Features¶
Since Lettuce is used to test the behavior of a project, the behavior is broken up in to features of the system.
After enumerating features, you need to create scenarios which will describe those feature. Thus, scenarios are components of a feature.
Let’s learn by example: suppose we want to create a system to manage a address book.
OK, one of the very basic features of a address book is adding contacts, which will include their name and phone numbers.
This is how Lettuce allows you to describe such feature:
Feature: Add people to address book
In order to organize phone numbers of friends
As a wise person
I want to add a people to my address book
Scenario: Add a person with name and phone number
Given I fill the field "name" with "John"
And fill the field "phone" with "2233-4455"
When I save the data
Then I see that my contact book has the persons:
| name | phone |
| John | 2233-4455 |
Scenario: Avoiding a invalid phone number
Given I fill the field "name" with "John"
And fill the field "phone" with "000"
When I save the data
Then I get the error: "000 is a invalid phone number"
In the feature above we can notice a few elements, for instance:
- The feature name:
Feature: Add people to contact book
- Feature headline:
In order to organize phone numbers of friends As a wise person I want to add a people to my address book
- Scenarios:
Scenario: Add a person with name and phone Given I fill the field "name" with "John" And fill the field "phone" with "2233-4455" When I save the data Then I see that my contact book has the persons: | name | phone | | John | 2233-4455 | Scenario: Avoiding a invalid phone number Given I fill the field "name" with "John" And fill the field "phone" with "000" When I save the data Then I get the error: "000 is a invalid phone number"
Scenarios¶
One or more scenarios compose a feature. There are two kinds of scenarios:
Simple¶
The simple scenarios are composed by steps, no matter if they are simple or tabulated steps.
The feature above is composed by two simple scenarios.
Outlined¶
Outlined scenarios are very handy because they help you to avoid repetition.
Suppose that we need to fill the same form many times, each time with a different data set. This is how it could be done using scenario outlines:
Let’s see how it could be done with scenario outlines:
Feature: Apply all my friends to attend a conference
In order to apply all my friends to the next PyCon_
As a lazy person
I want to fill the same form many times
Scenario Outline: Apply my friends
Go to the conference website
Access the link "I will attend"
Fill the field "name" with "<friend_name>"
Fill the field "email" with "<friend_email>"
Fill the field "birthday" with "<friend_birthdate>"
Click on "confirm attendance" button
Examples:
| friend_name | friend_email | friend_birthdate |
| Mary | mary@domain.com | 1988/02/10 |
| Lincoln | lincoln@provider.net | 1987/09/10 |
| Marcus | marcus@other.org | 1990/10/05 |
In a nutshell, the scenario above is equivalent to write the huge code bellow
Feature: Apply all my friends to attend a conference
In order to apply all my friends to the next PyCon_
As a lazy person
I want to fill the same form many times
Scenario: Apply Mary
Go to the conference website
Access the link "I will attend"
Fill the field "name" with "Mary"
Fill the field "email" with "mary@domain.com"
Fill the field "birthday" with "1988/02/10"
Click on "confirm attendance" button
Scenario: Apply Lincoln
Go to the conference website
Access the link "I will attend"
Fill the field "name" with "Lincoln"
Fill the field "email" with "lincoln@provider.net"
Fill the field "birthday" with "1987/09/10"
Click on "confirm attendance" button
Scenario: Apply Marcus
Go to the conference website
Access the link "I will attend"
Fill the field "name" with "Marcus"
Fill the field "email" with "marcus@other.org"
Fill the field "birthday" with "1990/10/05"
Click on "confirm attendance" button
As you can notice, scenario outlines are very useful and help you on avoiding text and code repetition
Steps and its definitions¶
Comparable with Scenarios, Steps comes in two kinds:
Simple steps¶
Simple steps are actually simple and they are related to the step definitions inside the scenarios.
Lettuce considers each line of a scenario as a simple step. The only exception is if the first non-blank character of the line is a pipe |. In this case, Lettuce will consider the step as a tabular step.
For instance, a simple step may look like this:
Given I go to the conference website
Tabular steps¶
Analog to Outlined Scenarios, the tabular steps are very useful, and avoid repetition of text.
Tabular steps are specially useful to set up some data set in a scenario, or to compare a set of data to the expected results in the end of the scenario.
However, feel free to use this whenever you find it useful.
Example:
Given I have the following contacts in my database
| name | phone |
| John | 2233-4455 |
| Smith | 9988-7766 |
introduction¶
Lettuce is an extremely useful and charming tool for BDD (Behavior Driven Development). It can execute plain-text functional descriptions as automated tests for Python projects, just as Cucumber does for Ruby.
Lettuce makes the development and testing process really easy, scalable, readable and - what is best - it allows someone who doesn’t program to describe the behavior of a certain system, without imagining those descriptions will automatically test the system during its development.
get lettuce¶
Make sure you’ve got Python installed and then run from the terminal:
user@machine:~$ [sudo] pip install lettuce
define a problem¶
Let’s choose a problem to lettuce: Given a number, what is its factorial?
Note
The factorial of a positive integer n, denoted by n!, is the product of all positive integers less than or equal to n. The factorial of 0 is 1
project structure¶
Build the directory tree bellow such as the files zero.feature and steps.py are empty.
/home/user/projects/mymath
| tests
| features
- zero.feature
- steps.py
lettuce it!¶
Lets begin to describe and solve our problem...
first round¶
[a] describe behaviour¶
Start describing the expected behaviour of factorial in zero.feature using English:
Feature: Compute factorial
In order to play with Lettuce
As beginners
We'll implement factorial
Scenario: Factorial of 0
Given I have the number 0
When I compute its factorial
Then I see the number 1
Note
zero.feature must be inside features directory and its extension must be .feature. However, you’re free to choose its name.
[b] define steps in python¶
Now let’s define the steps of the scenario, so Lettuce can understand the behaviour description. Create the steps.py file which will contain python code describing the steps.
Python:
from lettuce import *
@step('I have the number (\d+)')
def have_the_number(step, number):
world.number = int(number)
@step('I compute its factorial')
def compute_its_factorial(step):
world.number = factorial(world.number)
@step('I see the number (\d+)')
def check_number(step, expected):
expected = int(expected)
assert world.number == expected, \
"Got %d" % world.number
def factorial(number):
return -1
Note
steps.py must be inside features directory, but the names doesn’t need to be steps.py it can be any python file with a .py extension. Lettuce will look for python files recursively within features dir.
Ideally, factorial will be defined somewhere else. However, as this is just a first example, we’ll implement it inside steps.py, so you get the idea of how to use Lettuce.
Notice that, until now, we haven’t defined the factorial function (it’s returning -1).
[c] run and watch it fail¶
Go to the tests directory and run from the terminal:
user@machine:~/projects/mymath/tests$ lettuce
As you haven’t implemented factorial, it is no surprise the behavior won’t be reached:
Our only scenario failed :( Let’s solve it...
[d] write code to make it pass¶
Well, by definition, we know that the factorial of 0 is 1. As our only feature is this... we could force factorial to return 1.
from lettuce import *
@step('I have the number (\d+)')
def have_the_number(step, number):
world.number = int(number)
@step('I compute its factorial')
def compute_its_factorial(step):
world.number = factorial(world.number)
@step('I see the number (\d+)')
def check_number(step, expected):
expected = int(expected)
assert world.number == expected, \
"Got %d" % world.number
def factorial(number):
return 1
[e] run again and watch it pass¶
Again, run from the terminal:
user@machine:~/projects/mymath/tests$ lettuce
And you’ll be happy to see your factorial implementation passed all the behaviours expected:
Great! :)
However, one test is not enough for checking the quality of our solution... So let’s lettuce it again!
second round¶
Let’s provide more tests so our problem is better described, and so we provide a more accurate implementation of factorial:
[a] describe behaviour¶
Let’s provide two new scenarios, for numbers 1 and 2:
Feature: Compute factorial
In order to play with Lettuce
As beginners
We'll implement factorial
Scenario: Factorial of 0
Given I have the number 0
When I compute its factorial
Then I see the number 1
Scenario: Factorial of 1
Given I have the number 1
When I compute its factorial
Then I see the number 1
Scenario: Factorial of 2
Given I have the number 2
When I compute its factorial
Then I see the number 2
[b] define steps in python¶
As we haven’t changed the definition, no need to make changes on this step.
[c] run and watch it fail¶
user@machine:~/projects/mymath/tests$ lettuce
When running Lettuce we realize that our previous implementation of factorial works fine both for 0 and for 1, but not for 2 - it fails. :(
[d] write code to make it pass¶
Let’s provide a solution so we get the right factorial for all scenarios, specially for number 2:
from lettuce import *
@step('I have the number (\d+)')
def have_the_number(step, number):
world.number = int(number)
@step('I compute its factorial')
def compute_its_factorial(step):
world.number = factorial(world.number)
@step('I see the number (\d+)')
def check_number(step, expected):
expected = int(expected)
assert world.number == expected, \
"Got %d" % world.number
def factorial(number):
number = int(number)
if (number == 0) or (number == 1):
return 1
else:
return number
[e] run again and watch it pass¶
user@machine:~/projects/mymath/tests$ lettuce
Great! Three scenarios described and they are alright!
third round¶
Let’s provide more tests so our problem is better described and we get new errors so we’ll be able to solve them.
[a] describe behaviour¶
Feature: Compute factorial
In order to play with Lettuce
As beginners
We'll implement factorial
Scenario: Factorial of 0
Given I have the number 0
When I compute its factorial
Then I see the number 1
Scenario: Factorial of 1
Given I have the number 1
When I compute its factorial
Then I see the number 1
Scenario: Factorial of 2
Given I have the number 2
When I compute its factorial
Then I see the number 2
Scenario: Factorial of 3
Given I have the number 3
When I compute its factorial
Then I see the number 6
Scenario: Factorial of 4
Given I have the number 4
When I compute its factorial
Then I see the number 24
[b] define steps in python¶
As we haven’t changed the definition, no need to make changes on this step.
[c] run and watch it fail¶
user@machine:~/projects/mymath/tests$ lettuce
[d] write code to make it pass¶
from lettuce import *
@step('I have the number (\d+)')
def have_the_number(step, number):
world.number = int(number)
@step('I compute its factorial')
def compute_its_factorial(step):
world.number = factorial(world.number)
@step('I see the number (\d+)')
def check_number(step, expected):
expected = int(expected)
assert world.number == expected, \
"Got %d" % world.number
def factorial(number):
number = int(number)
if (number == 0) or (number == 1):
return 1
else:
return number*factorial(number-1)
[e] run again and watch it pass¶
user@machine:~/projects/mymath/tests$ lettuce
forth round¶
All steps should be repeated as long as you can keep doing them - the quality of your software depends on these.
Syntactic sugar¶
Available for versions > 0.2.19
Steps sentence can now be given by function name or doc.¶
To take a step sentence from function name or doc, just decorate it with “@step” without argument.
These two steps below, are identicals than the example above.
from lettuce import *
@step
def have_the_number(step, number):
'I have the number (\d+)'
world.number = int(number)
@step
def i_compute_its_factorial(step):
world.number = factorial(world.number)
Steps can be grouped in class decorated with “@steps”¶
from lettuce import world, steps
@steps
class FactorialSteps(object):
"""Methods in exclude or starting with _ will not be considered as step"""
exclude = ['set_number', 'get_number']
def __init__(self, environs):
self.environs = environs
def set_number(self, value):
self.environs.number = int(value)
def get_number(self):
return self.environs.number
def _assert_number_is(self, expected, msg="Got %d"):
number = self.get_number()
assert number == expected, msg % number
def have_the_number(self, step, number):
'''I have the number (\d+)'''
self.set_number(number)
def i_compute_its_factorial(self, step):
number = self.get_number()
self.set_number(factorial(number))
def check_number(self, step, expected):
'''I see the number (\d+)'''
self._assert_number_is(int(expected))
# Important!
# Steps are added only when you instanciate the "@steps" decorated class
# Internally decorator "@steps" build a closure with __init__
FactorialSteps(world)
def factorial(number):
number = int(number)
if (number == 0) or (number == 1):
return 1
else:
return number*factorial(number-1)
Have a nice lettuce...! ;)
handling data with tables¶
Let’s imagine writing a MVC application. While writing the tests you will stumble in to a situation where there is a few models that must be added to the database, maybe you will also need to check the new state of those models.
It means that as you write tests with lettuce, it can be very useful to handle data within steps.
Step tables are here for you
Feature: bill students alphabetically
In order to bill students properly
As a financial specialist
I want to bill those which name starts with some letter
Scenario: Bill students which name starts with "G"
Given I have the following students in my database:
| name | monthly_due | billed |
| Anton | $ 500 | no |
| Jack | $ 400 | no |
| Gabriel | $ 300 | no |
| Gloria | $ 442.65 | no |
| Ken | $ 907.86 | no |
| Leonard | $ 742.84 | no |
When I bill names starting with "G"
Then I see those billed students:
| name | monthly_due | billed |
| Gabriel | $ 300 | no |
| Gloria | $ 442.65 | no |
And those that weren't:
| name | monthly_due | billed |
| Anton | $ 500 | no |
| Jack | $ 400 | no |
| Ken | $ 907.86 | no |
| Leonard | $ 742.84 | no |
In the example above there are 4 steps, in which 3 contains tables.
Now let us imagine that we’re using Django and write a step definition that uses the table.
from lettuce import step
from school.models import Student
@step('I have the following students in my database:')
def students_in_database(step):
for student_dict in step.hashes:
person = Student(**student_dict)
person.save()
What about handy functions for getting the first or the last row of the tables ?!
from lettuce import step
from school.models import Student
@step('I have the following students in my database:')
def students_in_database(step):
person1 = Student(**step.hashes.first)
person2 = Student(**step.hashes.last)
person1.save()
person2.save()
Easy, huh?!
Every step has a attribute called hashes which is a list of dicts. Each dict has represents table headers as keys and each table row as value.
In other words, lettuce will translate the table written in the first step as this equivalent dict
@step('I have the following students in my database:')
def students_in_database(step):
assert step.hashes == [
{
'name': 'Anton',
'monthly_due': '$ 500',
'billed': 'no'
},
{
'name': 'Jack',
'monthly_due': '$ 400',
'billed': 'no'
},
{
'name': 'Gabriel',
'monthly_due': '$ 300',
'billed': 'no'
},
{
'name': 'Gloria',
'monthly_due': '$ 442.65',
'billed': 'no'
},
{
'name': 'Ken',
'monthly_due': '$ 907.86',
'billed': 'no'
},
{
'name': 'Leonard',
'monthly_due': '$ 742.84',
'billed': 'no'
},
]
multi-line strings¶
Now imagine you are writing an application which manipulates strings. When writing the tests, you may find yourself wanting to put multi-line strings in your steps.
Multi-line strings will do the trick
Feature: Split a string into multiple lines on spaces
In order to make strings more readable
As a user
I want to have words split into their own lines
Scenario: Split small-ish string
Given I have the string "one two three four five"
When I ask to have the string split into lines
Then I should see the following:
"""
one
two
three
four
five
"""
A line with nothing but three quotes (“””) is used to indicate the beginning and the end of a multi-line string.
Now, let’s define a step that knows how to use this.
from lettuce import step
@step('I should see the following:')
def i_should_see_the_following(step):
assert step.multiline == """one
two
three
four
five"""
Nice and straightforward.
Notice that leading spaces are stripped, and there’s not a newline at the beginning or end. This is due to the way that the parser strips blank lines and leading and trailing whitespace.
If you need blank lines leading or trailing whitespace, you can include lines which start and/or end with double quote, and they will be concatenated with the other multiline lines, with the quotes stripped off and their whitespace preserved.
For example
Feature: Split a string into multiple lines on spaces
In order to make strings more readable
As a user
I want to have words split into their own lines
Scenario: Split small-ish string
Given I have the string "one two three four five"
When I ask to have the string split into lines
Then I should see the following:
"""
" one
" two "
" three "
" four "
" five "
"
"""
Which we can verify like so:
from lettuce import step
@step('I should see the following:')
def i_should_see_the_following(step):
assert step.multiline == '\n'.join([
' one',
' two ',
' three ',
' four ',
' five ',
''])
Admittedly, this is a hack, but there’s no clean way to preserve whitespace in only one section of a feature definition in the current parser implementation.
Note that the first line doesn’t have any whitespace at the end, and thus doesn’t need to have a quote at the end of it.
Also note that if you want a double quote at the beginning of a line in your string, you’ll have to start your line with two double quotes, since the first one will be stripped off.
scenario outlines¶
On our first description file, zero.feature, all scenarios were similar. This made us repeat most of the text again and again.
Isn’t there a better way to deal with this - when several scenarios are almost equal and only some values change?
Yes, there is! :) You just need to use scenarios outlines.
An example is shown below:
Feature: Compute factorial
In order to play with Lettuce
As beginners
We'll implement factorial
Scenario Outline: Factorials [0-4]
Given I have the number <number>
When I compute its factorial
Then I see the number <result>
Examples:
| number | result |
| 0 | 1 |
| 1 | 1 |
| 2 | 2 |
| 3 | 6 |
| 4 | 24 |
This way, you will only need to provide the values that really change, reducing “copy & paste” work and making your tests more clear.
Note
If you overwrite zero.feature using the example above, and goto step [e], you’ll see your description expanding to the five previous scenarios:
calling steps from step definitions¶
Our tests should be as expressive as possible. However, we also want to re-use steps that we’ve seen before. With the tools we’ve used so far, you could end up with seriously long step definitions.
Scenario: Logged-in user does something cool.
Given I go to the home page
And I click the login button
And I fill in username:floppy password:banana
And I click "Login"
When I finally do something interesting
Then I'm already too bored to care.
In this case, we probably had a test case (maybe several) for which it was actually valuable to express how the user interacted with the login form. That’s where we got the step definitions for our login sequence. When the login form isn’t especially interesting any more, however, these steps are just noise. We’d really like to be able to define something like this without duplicating our step definitions.
Scenario: Logged-in user does something cool.
Given I am logged in
When I do something interesting
Then The world becomes a better place
Lettuce affords you the ability to write such a “step of steps” with a set of helpers matching each of the grammar terms Given, When and Then. You could accomplish the above like so.
@step('I am logged in')
def is_logged_in(step):
step.given('I go to the home page')
step.given('I click the login button')
# ... and so on.
running blocks of steps¶
It is sometimes even desirable to run blocks of steps, copy-and-pasted directly from Feature specifications. The Step.behave_as method lets you do this, and you can use str.format to fill in parameters dynamically. For example, we can write the above step definition like so:
@step('I am logged in')
def is_logged_in(step):
step.behave_as("""
Given I go to the home page
And I click the login button
And I fill in username:{user} password:{pass}
And I click "Login"
""".format(user='floppy', pass='banana'))
This can be combined with step argument capture for step definitions that are both expressive and DRY.
the command line¶
Lettuce is used as a command line utility, it means that currently the only way to use it is through a shell.
Once in a shell, you can use lettuce in 2 ways:
- in the usual way
Which means having the simple features/step_definitions folder somewhere in your project
- within a Django-powered project.
The difference between them is that within Django you have more options, but both ways have these common options:
running a specific feature file¶
user@machine:~/projects/myproj$ lettuce path/to/some/file.feature
With this option, your feature can even be out of the default features folder.
running only some scenarios of a specific feature file¶
user@machine:~/projects/myproj$ lettuce path/to/some/file.feature -s 3,5,9
This will run the scenarios 3, 5 and 9 from file path/to/some/file.feature
running only some scenarios all feature files¶
Maybe you can find it senseless, but it works like that, and does not hurt so far :)
user@machine:~/projects/myproj$ lettuce -s 3,5,9
Yeah, guess what?
This command will run the scenarios 3, 5 and 9 of all feature files living on myproj/features folder.
verbosity levels¶
level 1 - dots for each feature¶
user@machine:~/projects/myproj$ lettuce --verbosity=1
This is lettuce’s minimum verbosity level. It shows dots for each step run, regardless of what scenario or what feature is currently running.
For example, if you have a feature that looks like:
Feature: Manipulate strings
Scenario: Uppercased strings
Given I have the string "lettuce leaves"
When I put it in upper case
Then I see the string is "LETTUCE LEAVES"
The output will be:
user@machine:~/projects/myproj$ lettuce -v 1
...
1 feature (1 passed)
1 scenario (1 passed)
3 steps (3 passed)
level 2 - scenario names¶
user@machine:~/projects/myproj$ lettuce --verbosity=2
In this mode, lettuce will print each scenario name that is currently being ran, followed by OK, FAILED or ERROR depending of the status of the steps within that scenario.
For example, if you have a feature that looks like:
Feature: Manipulate strings
Scenario: Uppercased strings
Given I have the string "lettuce leaves"
When I put it in upper case
Then I see the string is "LETTUCE LEAVES"
Scenario: basic math
Given I sum 2 and 5
Then I see the result is 9
The output will be:
user@machine:~/projects/myproj$ lettuce -v 2
Uppercased strings ... OK
basic math ... FAILED
1 feature (1 passed)
2 scenarios (2 passed)
5 steps (4 passed)
level 3 - full feature print, but colorless¶
user@machine:~/projects/myproj$ lettuce --verbosity=3
This mode is a lot more verbose than the later one. It prints every single feature, with really useful information like:
- the relative path to the feature file being ran, and the current line in that file
- the relative path to the step definition responsible for the step being ran, also followed by the current line
- inline tracebacks when some feature fails
- “ready-to-use” snippets for undefined steps
For example, let’s say you have the feature below, but only the step Given I have the string "lettuce leaves" is defined
Feature: Manipulate strings
Scenario: Uppercased strings
Given I have the string "lettuce leaves"
When I put it in upper case
Then I see the string is "LETTUCE LEAVES"
Your output will look like:
user@machine:~/projects/myproj$ lettuce -v 2
Feature: Manipulate strings # features/strings.feature:1
Scenario: Uppercased strings # features/strings.feature:2
Given I have the string "lettuce leaves" # features/step_definitions/example-steps.py:5
When I put it in upper case # features/strings.feature:4 (undefined)
Then I see the string is "LETTUCE LEAVES" # features/strings.feature:5 (undefined)
1 feature (0 passed)
1 scenario (0 passed)
3 steps (2 undefined, 1 passed)
You can implement step definitions for undefined steps with these snippets:
# -*- coding: utf-8 -*-
from lettuce import step
@step(u'When I put it in upper case')
def when_i_put_it_in_upper_case(step):
assert False, 'This step must be implemented'
@step(u'Then I see the string is "(.*)"')
def then_i_see_the_string_is_group1(step, group1):
assert False, 'This step must be implemented'
level 4 - full feature print, but colorful¶
This mode is almost exactly the same of level 3, the difference is that it’s colorful.
Note
If you are going to put lettuce running in a Continuous-Integration server, like Hudson. You may choose the levels 1, 2 or 3, so that the output won’t look messy.
integrating with continuous integration¶
Lettuce can use Subunit to output test results. Subunit is a stream format that can be multiplexed, viewed in real time or converted to many different formats (such as xUnit/jUnit XML format).
user@machine:~/projects/myproj$ lettuce --with-subunit > output.log
user@machine:~/projects/myproj$ subunit2junitxml < subunit.bin > lettucetests.xml
The –subunit-file flag can be used to specify a filename other than subunit.bin this is important if you’re combining test runs.
including coverage¶
You can also get test coverage information using the coverage package.
user@machine:~/projects/myproj$ coverage run lettuce --with-subunit
user@machine:~/projects/myproj$ coverage xml
getting help from shell¶
user@machine:~/projects/myproj$ lettuce -h
Shows all the options described here.
features, scenarios and steps reference¶
Features, scenarios and steps are python objects within lettuce’s feature engine.
Here you will find out very “hacky” details about those objects. If you stumbled here looking for a introduction, it might be a good idea to read the feature tutorial for a introduction.
In order to exemplify the usage of attributes and methods below, let’s consider that there is a feature in a file called some.feature
# language: en
# just a comment
# another one
Feature: some feature
Here comes
The feature
Description
Scenario: try out something
Given I show lettuce running
Then I should be happy
Feature¶
Feature.scenarios¶
A list of scenario objects
The attribute scenarios could be used as follows
feature.scenarios[0].name == 'try out something'
Feature.described_at¶
A FeatureDescription object, has the file and line which the feature was described. Lettuce uses it to output those metadata.
The attribute described_at could be used as follows
# the line in which the feature started
feature.described_at.line == 5
# the filename path
'some.feature' in feature.described_at.file
# a tuple with the lines that contains the feature description
feature.described_at.description_at == (6, 7, 8)
Feature.max_length¶
A property that calculates the maximum length of all lines that built the feature.
Mostly used by shell output to find out where to print the feature description.
Example:
feature.max_length == 21
Feature.get_head¶
Does represent the feature with its first representation in current language followed by a colon and the feature name.
Example:
feature.get_head() == 'Feature: some feature'
But if the same feature would written in Brazilian Portuguese, for example:
# language: pt-br
# apenas um comentário
# e outro
Funcionalidade: alguma funcionalidade
Aqui vem
a descrição
da funcionalidade
Cenário: ...
...
Then, Feature.get_head() would give:
feature.get_head() == 'Funcionalidade: alguma funcionalidade'
TotalResult¶
TotalResult.features_ran¶
Integer, the total of features ran
TotalResult.features_passed¶
Integer, the total of features passed
TotalResult.scenarios_ran¶
Integer, the total of scenarios ran
TotalResult.scenarios_passed¶
Integer, the total of scenarios passed
TotalResult.steps¶
Integer, the number of steps that were supposed to run
TotalResult.proposed_definitions¶
A list of Step that have no step definition
Scenario¶
Scenario.steps¶
A list of scenario objects
The attribute scenarios could be used as follows
scenario.steps[0].sentence == 'try out something'
Step¶
Step.passed¶
Boolean, true if the step ran without an error.
Step.failed¶
Boolean, true if the step ran and an error occurred during execution.
step definition¶
A decorator that can be used on any python function, takes a regex string as parameter, so that the function can me matched against steps.
from lettuce import step
@step('I am (happy|sad)')
def show_lettuce_running_here(step, action):
if action == 'happy':
return # everything is fine!
else:
assert False, 'you should be happy, dude!'
the “terrain”¶
Terrain is a “pun” with lettuce and its “living place”, its about setup and teardown, and general hacking on your lettuce tests.
terrain.py¶
By convention lettuce tries do load a file called terrain.py located at the current directory.
Think at this file as a global setup place, there you can setup global hooks, and put things into lettuce “world”.
Note
You can also set a terrain.py file within the root of your Django project, when running the python manage.py harvest command, lettuce will load it. See more at the django command.
in practice¶
Try out this file layout:
/home/user/projects/some-project
| features
- the-name-of-my-test.feature
- the-file-which-holds-step-definitions.py
- terrain.py
Then add some setup at terrain.py and run lettuce
user@machine:~/projects/some-project$ lettuce
And notice terrain.py will be loaded before anything
world¶
For the sake of turning easier and funnier to write tests, lettuce “violates” some principles of good design in python, such as avoiding implicity and using global stuff.
The “world” concept of lettuce is mostly about “global stuff”.
in practice¶
Imagine a file located somewhere that will be imported by your application before lettuce start running tests:
from lettuce import world
world.some_variable = "yay!"
So that, within some step file you could use things previously set on world:
from lettuce import *
@step(r'exemplify "world" by seeing that some variable contains "(.*)"')
def exemplify_world(step, value):
assert world.some_variable == value
And the feature could have something like:
Feature: test lettuce's world
Scenario: check variable
When I exemplify "world" by seeing that some variable contains "yay!"
world.absorb¶
It can be really useful to put functions and/or classes in lettuce.world
For example:
from lettuce import world
def my_project_wide_function():
# do something
world.my_project_wide_function = my_project_wide_function
world.my_project_wide_function()
But as you can notice, as your project grows, there can be a lot of repetitive lines, not DRY at all :(
In order to avoid that, lettuce provides a “absorb” decorator that lives within “world”
Let’s see it in action:
from lettuce import world
@world.absorb
def my_project_wide_function():
# do something
world.my_project_wide_function()
You can also use it with classes:
from lettuce import world
@world.absorb
class MyClass:
pass
assert isinstance(world.MyClass(), MyClass)
And even with lambdas, but in this case you need to name it
from lettuce import world
world.absorb(lambda: "yeah", "optimist_function")
assert world.optimist_function() == 'yeah'
world.spew¶
Well, if you read the topic above, you may be guessing: “if I keep stashing things in lettuce.world, it may bloat it sometime, or confuse member names along my steps, or hooks.
For those cases after “absorbing” something, world can also “spew” it.
from lettuce import world
@world.absorb
def generic_function():
# do something
assert hasattr(world, 'generic_function')
world.spew('generic_function')
assert not hasattr(world, 'generic_function')
hooks¶
Lettuce has hooks that are called sequentially before and after each action
Presented as python decorators, it can be used to take any actions you find useful.
For example, you can set a browser driver at world, and close the connection after all, populate database with test mass or anything you want, for example
Let’s see it from outside in
@before.all¶
This hook is ran before lettuce look for and load feature files
The decorated function takes NO parameters
from lettuce import *
@before.all
def say_hello():
print "Hello there!"
print "Lettuce will start to run tests right now..."
@after.all¶
This hook is ran after lettuce run all features, scenarios and steps
The decorated function takes a TotalResult as parameter, so that you can use the result statistics somehow
from lettuce import *
@after.all
def say_goodbye(total):
print "Congratulations, %d of %d scenarios passed!" % (
total.scenarios_ran,
total.scenarios_passed
)
print "Goodbye!"
@before.each_feature¶
This hook is ran before lettuce run each feature
The decorated function takes a Feature as parameter, so that you can use it to fetch scenarios and steps inside.
from lettuce import *
@before.each_feature
def setup_some_feature(feature):
print "Running the feature %r, at file %s" % (
feature.name,
feature.described_at.file
)
@after.each_feature¶
This hooks behaves in the same way @before.each_feature does, except by the fact that its ran after lettuce run the feature.
from lettuce import *
@after.each_feature
def teardown_some_feature(feature):
print "The feature %r just has just ran" % feature.name
@before.each_scenario¶
This hook is ran before lettuce run each scenario
The decorated function takes a Scenario as parameter, so that you can use it to fetch steps inside.
from lettuce import *
from fixtures import populate_test_database
@before.each_scenario
def setup_some_scenario(scenario):
populate_test_database()
@after.each_scenario¶
This hooks behaves in the same way @before.each_scenario does, except by the fact that its ran after lettuce run the scenario.
from lettuce import *
from database import models
@after.each_scenario
def teardown_some_scenario(scenario):
models.reset_all_data()
@before.each_step¶
This hook is ran before lettuce run each step
The decorated function takes a Step as parameter, so that you can use it to fetch tables and so.
from lettuce import *
@before.each_step
def setup_some_step(step):
print "running step %r, defined at %s" % (
step.sentence,
step.defined_at.file
)
@after.each_step¶
This hooks behaves in the same way @before.each_step does, except by the fact that its ran after lettuce run the step.
from lettuce import *
@after.each_step
def teardown_some_step(step):
if not step.hashes:
print "no tables in the step"
django-specific hooks¶
Since lettuce officially supports Django, there are a few specific hooks that help on setting up your test suite on it.
@before.harvest¶
This hook is ran before lettuce start harvesting your Django tests. It can be very useful for setting up browser drivers (such as selenium), before all tests start to run on Django.
The decorated function takes a dict with the local variables within the harvest management command.
from lettuce import *
from lettuce.django import django_url
from selenium import selenium
@before.harvest
def prepare_browser_driver(variables):
if variables.get('run_server', False) is True:
world.browser = selenium('localhost', 4444, '*firefox', django_url('/'))
world.browser.start()
@after.harvest¶
This hook is ran right after lettuce finish harvesting your Django tests. It can be very useful for shutting down previously started browser drivers (see the example above).
The decorated function takes a list of TotalResult objects.
from lettuce import *
@after.harvest
def shutdown_browser_driver(results):
world.browser.stop()
@before.each_app¶
This hook is ran before lettuce run each Django app.
The decorated function takes the python module that corresponds to the current app.
from lettuce import *
@before.each_app
def populate_blog_database(app):
if app.__name__ == 'blog':
from blog.models import Post
Post.objects.create(title='Nice example', body='I like writting!')
@after.each_app¶
This hook is ran after lettuce run each Django app.
The decorated function takes two arguments:
- the python module that corresponds to the current app.
- a TotalResult as parameter, so that you can use the result statistics somehow
from lettuce import *
@after.each_app
def clear_blog_database_if_successful(app, result):
if app.__name__ == 'blog':
if result.scenarios_ran is result.scenarios_passed:
from blog.models import Post, Comment
Comment.objects.all()
Post.objects.all()
@before.runserver and @after.runserver¶
These hooks are ran right before, and after lettuce starts up the built-in http server.
The decorated function takes a lettuce.django.server.ThreadedServer object.
from lettuce import *
from django.core.servers.basehttp import WSGIServer
@before.runserver
def prepare_database(server):
assert isinstance(server, WSGIServer)
import mydatabase
mydatabase.prepare()
@after.runserver
def say_goodbye(server):
assert isinstance(server, WSGIServer)
print "goodbye, see you soon"
@before.handle_request and @after.handle_request¶
These hooks are ran right before, and after lettuce’s built-in HTTP server responds to a request.
Both decorated functions takes these two arguments:
- a django.core.servers.basehttp.WSGIServer object.
- a lettuce.django.server.ThreadedServer object.
from lettuce import *
from django.core.servers.basehttp import WSGIServer
@before.handle_request
def print_request(httpd, server):
socket_object, (client_address, size) = httpd.get_request()
print socket_object.dup().recv(size)
@after.handle_request
def say_goodbye(httpd, server):
socket_object, (client_address, size) = httpd.get_request()
print "I've just finished to respond to the client %s" % client_address
Warning
all the handle_request hooks are run within a python thread. If something went wrong within a calback, lettuce can get stuck.
language support¶
Lettuce currently supports 18 languages:
- English
- Portuguese (Português)
- Polish (Polski)
- Catalan (Català )
- Spanish (Español)
- Hungarian (Magyar)
- French (Français)
- German (Deutsch)
- Japanese (日本語)
- Turkish (Türkçe)
- Simplified Chinese (简体ä¸æ–‡)
- Traditional Chinese (ç¹é«”ä¸æ–‡)
- Russian (РуÑÑкий)
- Ukrainian (УкраїнÑька)
- Italian (Italiano)
- Norwegian (Norsk)
- Swedish (Svenska)
- Czech (Čeština)
Although it’s only about writing tests since the current version does output only in English.
writing features in a specific language¶
You can tell lettuce the language of a feature file through adding a comment in the first line of the file, using the following syntax:
# language: <code>
english example¶
# language: en
Feature: write features in english
Scenario: simple scenario
Given I write a file which starts with "# language: en"
Then it must be parsed with proper english keywords
brazilian portuguese example¶
# language: pt-br
Funcionalidade: escrever funcionalidades em português
Cenário: cenário simples
Dado que eu crio um arquivo que começa com "# language: pt-br"
Então ele deve ser interpretado com as devidas palavras-chave brasileiras
adding support to other languages¶
We love contribution, so if you want to bring lettuce to your native language there is a single and simple way.
fetch the code¶
First of all, you must have git control version installed in your machine.
Once you have it installed, grab the code with
user@machine:~$ git clone git://github.com/gabrielfalcao/lettuce.git
And edit the file located at:
lettuce/languages.py
And add a new dictionary entry for your native language.
Let’s see the Brazilian Portuguese translation to exemplify.
LANGUAGES = {
'pt-br': {
'examples': u'Exemplos|Cenários',
'feature': u'Funcionalidade',
'name': u'Portuguese',
'native': u'Português',
'scenario': u'Cenário|Cenario',
'scenario_outline': u'Esquema do Cenário|Esquema do Cenario',
'scenario_separator': u'(Esquema do Cenário|Esquema do Cenario|Cenario|Cenário)',
},
}
The key of the dict will be used as identifier for the comment # language: identifier at feature files.
The value must be a dict, where the keys are canonical representation of keywords (string), and the values must be a pipe-separated string with translation possibilities.
It allows different translations for the same keyword in the current language, which offers many possibilities for different semantical cases.
For example, when using scenario outlines, it can be semantically nicer to write:
Scenarios:
| name | age |
| John | 22 |
| Mary | 53 |
Instead of:
Examples:
| name | age |
| John | 22 |
| Mary | 53 |
add your translation¶
Now you can add your own language to lettuce, save the languages.py file and commit in the source control with.
For example, let’s suppose that you’ve added Spanish support:
user@machine:~/lettuce$ git commit lettuce/languages.py -m 'adding translation for spanish'
Generate a patch:
user@machine:~/lettuce$ git format patch HEAD^1
And send to lettuce’s ticket tracker as a gist or something like it.
Built-in Django Steps¶
Lettuce features a number of built-in steps for Django to simplify the creation of fixtures.
creating fixture data¶
Lettuce can automatically introspect your available Django models to create fixture data, e.g.
Background:
Given I have options in the database:
| name | value |
| Lettuce | Rocks |
This will find a model whose verbose name is options. It will then create objects for that model with the parameters specified in the table (i.e. name=Lettuce, value=Rocks).
You can also specify relational information. Assuming a model Profile with foreign key user and field avatar:
Background:
Given user with username "harvey" has profile in the database:
| avatar |
| cat.jpg |
To create many-to-many relationships, assuming User has and belongs to many Group objects:
Background:
Given user with username "harvey" is linked to groups in the database:
| name |
| Cats |
For many-to-many relationship to be created, both models must exist prior to linking.
Most common data can be parsed, i.e. true/false, digits, strings and dates in the form 2013-10-30.
registering your own model creators¶
For more complex models that have to process or parse data you can write your own creating steps using the creates_models decorator.
from lettuce.django.steps.models import (creates_models,
reset_sequence,
hashes_data)
@creates_models(Note)
def create_note(step):
data = hashes_data(step)
for row in data:
# convert the author into a user object
row['author'] = get_user(row['author'])
Note.objects.create(**row)
reset_sequence(Note)
testing models¶
Two steps exist to test models.
Then features should be present in the database:
| name | value |
| Lettuce | Rocks |
And there should be 1 feature in the database
You can also test non-database model attributes by prefixing an @ to the attribute name. Non-database attributes are tested after the records are selected from the database.
Then features should be present in the database:
| name | value | @language |
| Lettuce | Rocks | Python |
registering your own model testers¶
For more complex tests that have to process or parse data you can write your own creating steps using the checks_existence decorator.
Lettuce recipe: Using nose for pretty assertions¶
Lettuce uses python’s builtin exception AssertionError to mark tests as failed.
Although in order to describe the assertion with a custom string you would need to do something like:
from lettuce import step
@step('some step with "(.*)"'):
def some_step(step, from):
assert from == 'expectation', \
"Ooops, '%s' should be equal 'expectation', but isn't" % from
nose is a python module that provides a set of assert functions that already have a nice description, and fortunately it still uses AssertionError, which makes nose totally compliant with lettuce.
The example below shows how the step above could be written taking advantage of nose:
from lettuce import step
from nose.tools import assert_equals
@step('some step with "(.*)"'):
def some_step(step, from):
assert_equals(from, 'expectation')
It rocks, huh?!
Web development fun with Lettuce and Django¶
Django is a awesome web framework, very mature, aims for simplicity and the best of all: it’s fun to use it.
To make it even more fun, lettuce has built-in support for Django.
Getting started¶
1. install the lettuce django app¶
Pick up any Django project, and add lettuce.django in its settings.py configuration file:
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.admin',
# ... other apps here ...
'my_app',
'foobar',
'another_app',
'lettuce.django', # this guy will do the job :)
)
Considering the configuration above, let’s say we want to write tests for the my_app django application.
2. create the feature directories¶
Lettuce will look for a features folder inside every installed app:
/home/user/projects/djangoproject
| settings.py
| manage.py
| urls.py
| my_app
| features
- index.feature
- index.py
| foobar
| features
- carrots.feature
- foobar-steps.py
| another_app
| features
- first.feature
- second.feature
- many_steps.py
3. write your first feature¶
@index.feature:
Feature: Rocking with lettuce and django
Scenario: Simple Hello World
Given I access the url "/"
Then I see the header "Hello World"
Scenario: Hello + capitalized name
Given I access the url "/some-name"
Then I see the header "Hello Some Name"
@index-steps.py:
from lettuce import *
from lxml import html
from django.test.client import Client
from nose.tools import assert_equals
@before.all
def set_browser():
world.browser = Client()
@step(r'I access the url "(.*)"')
def access_url(step, url):
response = world.browser.get(url)
world.dom = html.fromstring(response.content)
@step(r'I see the header "(.*)"')
def see_header(step, text):
header = world.dom.cssselect('h1')[0]
assert header.text == text
4. run the tests¶
Once you install the lettuce.django app, the command harvest will be available:
user@machine:~projects/djangoproject $ python manage.py harvest
The harvest command executes the django.test.utils.setup_test_environment function before it starts up the Django server. Typically, invoking this function would configure Django to use the locmem in-memory email backend. However, Lettuce uses a custom Django email backend to support retrieving email from Lettuce test scripts. See Checking email for more details.
5. specifying feature files¶
The harvest command accepts a path to feature files, in order to run only the features you want.
Example:
user@machine:~projects/djangoproject $ python manage.py harvest path/to/my-test.feature
Technical details¶
If you want to write acceptance tests that run with web browsers, you can user tools like twill, selenium, webdriver and windmill
red-tape-less builtin server¶
Lettuce cleverly runs an instance of the built-in Django HTTP server in the background. It tries to bind the HTTP server at localhost:8000 but if the port is busy, it keeps trying to run in higher ports: 8001, 8002 and so on until it reaches the maximum port number 65535.
Note
You can override the default starting port from “8000” to any other port you want.
To do so, refer to “running the HTTP server in other port than 8000” below.
So that you can use browser-based tools such as those listed above to access Django.
Warning
When running the http server, lettuce sets the environment variables SERVER_NAME and SERVER_PORT. It was brought for a GAE issue. If it can possibly bring any errors, be warned.
figure out django urls¶
As the Django HTTP server can be running in any port within the range 8000 - 65535, it could be hard to figure out the correct URL for your project, right?
Wrong!
Lettuce is here for you. Within your steps you can use the django_url utility function:
from lettuce import step, world
from lettuce.django import django_url
@step(r'Given I navigate to "(.*)"')
def navigate_to_url(step, url):
full_url = django_url(url)
world.browser.get(full_url)
what does django_url do ?!?¶
It prepends a Django-internal URL with the HTTP server address.
In other words, if lettuce binds the http server to localhost:9090 and you call django_url with "/admin/login":
from lettuce.django import django_url
django_url("/admin/login")
It returns:
"http://localhost:9090/admin/login"
terrain also available in django projects¶
At this point you probably know how terrain.py works, and it also works with Django projects.
You can setup environment and stuff like that within a terrain.py file located at the root of your Django project.
Taking the very first example of this documentation page, your Django project layout would like like this:
/home/user/projects/djangoproject
| settings.py
| manage.py
| urls.py
| terrain.py
| my_app
| features
- index.feature
- index.py
| foobar
| features
- carrots.feature
- foobar-steps.py
| another_app
| features
- first.feature
- second.feature
- many_steps.py
Notice the terrain.py file at the project root, there you can populate the world and organize your features and steps with it :)
Checking email¶
When you run your Django server under lettuce, emails sent by your server do not get transmitted over the Internet. Instead, these emails are added to a multiprocessing.Queue object at lettuce.django.mail.queue.
Example:
from lettuce import step
from lettuce.django import mail
from nose.tools import assert_equals
@step(u'an email is sent to "([^"]*?)" with subject "([^"]*)"')
def email_sent(step, to, subject):
message = mail.queue.get(True, timeout=5)
assert_equals(message.subject, subject)
assert_equals(message.recipients(), [to])
Running without HTTP server¶
Sometimes you may just do not want to run Django’s built-in HTTP server running in background, in those cases all you need to do is run the harvest command with the --no-server or -S option.
Example:
python manage.py harvest --no-server
python manage.py harvest -S
running the HTTP server in other port than 8000¶
If you face the problem of having lettuce running on port 8000, you can change that behaviour.
Before running the server, lettuce will try to read the setting LETTUCE_SERVER_PORT which must be a integer
Example:
LETTUCE_SERVER_PORT = 7000
This can be really useful if 7000 is your default development port, for example.
running the HTTP server with settings.DEBUG=True¶
In order to run tests against the nearest configuration of production, lettuce sets up settings.DEBUG=False
However, for debug purposes one can face a misleading HTTP 500 error without traceback in Django. For those cases lettuce provides the --debug-mode or -d option.
python manage.py harvest --debug-mode
python manage.py harvest -d
using the test database¶
If you want to use a test database by default, instead of a live database, with your test server you can specify the -T flag or set the following configuration variable in settings.py.
LETTUCE_USE_TEST_DATABASE = True
running only the specified scenarios¶
You can also specify the index of the scenarios you want to run through the command line, to do so, run with --scenarios or -s options followed by the scenario numbers separated by commas.
For example, let’s say you want to run the scenarios 4, 7, 8 and 10:
python manage.py harvest --scenarios=4,7,8,10
python manage.py harvest -s 4,7,8,10
to run or not to run? That is the question!¶
During your development workflow you may face two situations:
running tests from just certain apps¶
Lettuce takes a comma-separated list of app names to run tests against.
For example, the command below would run ONLY the tests within the apps myapp and foobar:
python manage.py harvest --apps=myapp,foobar
# or
python manage.py harvest --a myapp,foobar
You can also specify it at settings.py so that you won’t need to type the same command-line parameters all the time:
LETTUCE_APPS = (
'myapp',
'foobar',
)
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.admin',
'my_app',
'foobar',
'another_app',
'lettuce.django',
)
running tests from all apps, except by some¶
Lettuce takes a comma-separated list of app names which tests must NOT be ran.
For example, the command below would run ALL the tests BUT those within the apps another_app and foobar:
python manage.py harvest --avoid-apps=another_app,foobar
You can also specify it at settings.py so that you won’t need to type the same command-line parameters all the time:
LETTUCE_AVOID_APPS = (
'another_app',
'foobar',
)
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.admin',
'my_app',
'foobar',
'another_app',
'lettuce.django',
)
Development guidelines¶
Synopsis¶
- fork and clone the project: see lettuce repository at github.com [1]
- Install a development environment for lettuce
- run the tests: see Testing
- hack at will
- commit, push, etc...
- send a pull request
Table of contents¶
Install a development environment for lettuce¶
Here are guidelines to get a development environment for lettuce.
OS specific¶
Here are repcipes for specific operating systems. They should help you go fast or automate lettuce installation procedure:
Installation on Debian Squeeze¶
Recipe to get a development environment for lettuce in a fresh install of Debian Squeeze.
The following values are used below. You may customize them depending on your needs.
# Lettuce installation directory.
lettuce_dir=~/lettuce
# Virtualenv directory.
lettuce_env_dir=$lettuce_dir
# Git.
upstream_url="https://github.com/gabrielfalcao/lettuce.git"
fork_url=$upstream_url
# System's package manager.
system-install() { su -c "aptitude install ${*}" }
Execute the following commands:
system-install python-dev python-virtualenv git libxml2-dev libxslt-dev
git clone $fork_url $lettuce_dir
# Configure upstream
cd $lettuce_dir
git remote add upstream $upstream_url
virtualenv --distribute --no-site-packages $lettuce_env_dir
source $lettuce_env_dir/bin/activate
cd $lettuce_dir
pip install -r requirements.txt
python setup.py develop
Go back to Development guidelines and learn about Testing.
Generic guidelines¶
Dependencies¶
you will need to install these dependencies in order to hack lettuce :)
All of them are used within lettuce tests.
mkvirtualenv lettuce
workon lettuce
pip install -r requirements.txt
sudo pip install -r requirements.txt
- [nose](http://code.google.com/p/python-nose/)
> [sudo] pip install nose
- [mox](http://code.google.com/p/pymox/)
> [sudo] pip install mox
- [sphinx](http://sphinx.pocoo.org/)
> [sudo] pip install sphinx
- [lxml](http://codespeak.net/lxml/)
> [sudo] pip install lxml
- [tornado](http://tornadoweb.org/)
> [sudo] pip install tornado
- [django](http://djangoproject.com/)
> [sudo] pip install django
Installing lettuce itself¶
[sudo] python setup.py develop
Testing¶
How to run and write tests for lettuce.
Run tests¶
make unit functional integration doctest
Contributing to the documentation¶
This documentation uses Python-sphinx [1]. It uses reStructuredText [2] syntax.
Conventions¶
Language¶
The documentation is written in english.
Line length¶
Limit all lines to a maximum of 79 characters.
Headings¶
Use the following symbols to create headings: # * = -
As an example:
##################
H1: document title
##################
*********
Sample H2
*********
Sample H3
=========
Sample H4
---------
And sample text.
If you need more than H4, then consider creating a new document.
Code blocks¶
Combine a “highlight” directive and a ”::” to specify the programming language. As an example:
.. highlight:: python
::
import this
Warning
“code-block” and “sourcecode” directives are not natively supported by rst2html, which is used by Github’s repository browser, i.e. those lines won’t appear on Github. Whereas ”::” lines will.
Links and references¶
On pages which are quite long, use links and references footnotes with the “target-notes” directive. As an example:
#############
Some document
#############
Some text which includes links to `Example website`_ and many other links.
`Example website`_ can be referenced multiple times.
(... document content...)
And at the end of the document...
**********
References
**********
.. target-notes::
.. _`Example website`: http://www.example.com/
This Contributing to the documentation page uses this syntax.
Install Sphinx¶
Python-sphinx [1] installation is covered in Install a development environment for lettuce.
In other cases, please refer to Python-sphinx [1] documentation.
Export documentation to HTML¶
Install Python-sphinx [1].
Make sure sphinx-build is in your shell’s $PATH. If you are using virtualenv as told in Install a development environment for lettuce, then activate your virtual environment.
Go to lettuce folder and use the provided Makefile:
make documentation
HTML documentation is exported to docs/_build/html/.
Use doctests!¶
This documentation uses the Sphinx’s doctest extension [3].
Write doctests¶
Here is a RST code sample to write doctests. You can find some doctests in the “terrain”.
.. highlight:: python
.. doctest::
>>> print "Hello world!"
Hello world!
See Sphinx’s doctest extension [3] and Python’s doctest [4] documentations for details.
Keep in mind¶
that lettuce is a testing software, patches and pull requests must come with automated tests, and if suitable, with proper documentation.