16.1. Test Doctest

  • Tests are always the most up-to-date code documentation

  • Tests cannot get out of sync from code

  • Checks if function output is exactly as expected

  • Useful for regex modifications

  • Can add text (i.e. explanations) between tests

Use Cases:

16.1.1. Docstring

  • Docstring is a first multiline comment in: File/Module, Class, Method/Function

  • Used for generating help() documentation

  • It is accessible in __doc__ property of an object

  • Used for doctest

  • PEP 257 -- Docstring Conventions: For multiline str always use three double quote (""") characters

  • More information in Function Doctest

Docstring used for documentation:

>>> def say_hello():
...     """This is the say_hello function"""
...     print('Hello')
>>>
>>>
>>> 
... help(say_hello)
Help on function say_hello in module __main__:

say_hello()
    This is the say_hello function
>>>
>>> print(say_hello.__doc__)
This is the say_hello function

Docstring used for documentation:

>>> def say_hello():
...     """
...     This is the say_hello function
...     And the description is longer then one line
...     """
...     print('Hello')
>>>
>>>
>>> help(say_hello)  
Help on function say_hello in module __main__:

say_hello()
    This is the say_hello function
    And the description is longer then one line
>>>
>>> print(say_hello.__doc__)

    This is the say_hello function
    And the description is longer then one line

16.1.2. Syntax

  • Docstring is a first multiline comment in: File/Module, Class, Method/Function

  • Used for generating help() documentation

  • It is accessible in __doc__ property of an object

  • Used for doctest

  • PEP 257 -- Docstring Conventions: For multiline str always use three double quote (""") characters

>>> def add(a, b):
...     """
...     >>> add(1, 2)
...     3
...     >>> add(-1, 1)
...     0
...     >>> add(0, 0)
...     0
...     """
...     return a + b

>>> 
... """
... >>> add(1, 2)
... 3
... >>> add(-1, 1)
... 0
... >>> add(0, 0)
... 0
... """
>>>
>>> def add(a, b):
...     return a + b

16.1.3. Running Tests

Running tests in Pycharm IDE (either option):

  • Right click on source code with doctests -> Run 'Doctest for ...'

  • View menu -> Run... -> Doctest in myfunction

  • Note, that doctests are not discovered in scratch files in PyCharm

Running Tests from Python Code:

>>> if __name__ == "__main__":
...     from doctest import testmod
...     testmod()

Running tests from command line (displays errors only):

$ python -m doctest myfile.py

Add -v to display more verbose output.

$ python -m doctest -v myfile.py

16.1.4. Test Int, Float

int values:

>>> 
... """
... >>> add(1, 2)
... 3
... >>> add(-1, 1)
... 0
... >>> add(0, 0)
... 0
... """
>>>
>>> def add(a, b):
...     return a + b

float values:

>>> 
... """
... >>> add(1.0, 2.0)
... 3.0
...
... >>> add(0.1, 0.2)
... 0.30000000000000004
...
... >>> add(0.1, 0.2)   
... 0.3000...
... """
>>>
>>> def add(a, b):
...     return a + b

This is due to the floating point arithmetic in IEEE 754 standard:

>>> 0.1 + 0.2
0.30000000000000004

>>> 0.1 + 0.2  
0.3000...

>>> round(0.1+0.2, 16)
0.3

>>> round(0.1+0.2, 17)
0.30000000000000004

More information in Math Precision

16.1.5. Test Bool

>>> 
... """
... Function checks if person is adult.
... Adult person is over 18 years old.
...
... >>> is_adult(18)
... True
...
... >>> is_adult(17.9)
... False
... """
>>>
>>> AGE_ADULT = 18
>>>
>>> def is_adult(age):
...     if age >= AGE_ADULT:
...         return True
...     else:
...         return False

16.1.6. Test Str

  • Python will change to single quotes in most cases

  • Python will change to double quotes to avoid escapes

  • print() function output, don't have quotes

Returning str. Python will change to single quotes in most cases:

>>> 
... """
... >>> echo('hello')
... 'hello'
...
... # Python will change to single quotes in most cases
... >>> echo("hello")
... 'hello'
...
... Following test will fail
... >>> echo('hello')
... "hello"
...
... Python will change to double quotes to avoid escapes
... >>> echo('It\\'s Twardowski\\'s Moon')
... "It's Twardowski's Moon"
... """
>>>
>>> def echo(data):
...     return data

There are no quotes in print() function output:

>>> 
... """
... >>> echo('hello')
... hello
... """
>>>
>>> def echo(data):
...     print(data)

Testing print(str) with newlines:

>>> 
... """
... >>> echo('hello')
... hello
... hello
... hello
... <BLANKLINE>
... """
>>>
>>> def echo(data):
...     print(f'{data}\n' * 3)

16.1.7. Test Ordered Sequence

>>> 
... """
... >>> echo([1,2,3])
... [1, 2, 3]
...
... >>> echo((1,2,3))
... (1, 2, 3)
... """
>>>
>>> def echo(data):
...     return data

>>> 
... """
... >>> echo([1,2,3])
... [1, 2, 3]
...
... >>> echo((1,2,3))
... [1, 2, 3]
... """
>>>
>>> def echo(data):
...     return [x for x in data]

>>> 
... """
... >>> echo([1,2,3])
... [274.15, 275.15, 276.15]
...
... >>> echo((1,2,3))
... (274.15, 275.15, 276.15)
... """
>>>
>>> def echo(data):
...     cls = type(data)
...     return cls(x+273.15 for x in data)

16.1.8. Test Unordered Sequence

Hash from numbers are constant:

>>> 
... """
... >>> echo({1})
... {1}
... >>> echo({1,2})
... {1, 2}
... """
>>>
>>> def echo(data):
...     return data

However hash from str elements changes at every run:

>>> 
... """
... >>> echo({'a', 'b'})
... {'b', 'a'}
... """
>>>
>>> def echo(data):
...     return data

Therefore you should test if element is in the result, rather than comparing output:

>>> 
... """
... >>> result = echo({'a', 'b'})
... >>> 'a' in result
... True
... >>> 'b' in result
... True
... """
>>>
>>> def echo(data):
...     return data

16.1.9. Test Mapping

>>> 
... """
... >>> result = echo({'a': 1, 'b': 2})
... >>> result
... {'a': 1, 'b': 2}
... >>> 'a' in result.keys()
... True
... >>> 1 in result.values()
... True
... >>> ('a', 1) in result.items()
... True
... >>> result['a']
... 1
... """
>>>
>>> def echo(data):
...     return data

16.1.10. Test Nested

>>> 
... """
... >>> DATA = [
... ...     ('Sepal length', 'Sepal width', 'Petal length', 'Petal width', 'Species'),
... ...     (5.8, 2.7, 5.1, 1.9, 'virginica'),
... ...     (5.1, 3.5, 1.4, 0.2, 'setosa'),
... ...     (5.7, 2.8, 4.1, 1.3, 'versicolor'),
... ...     (6.3, 2.9, 5.6, 1.8, 'virginica'),
... ...     (6.4, 3.2, 4.5, 1.5, 'versicolor'),
... ...     (4.7, 3.2, 1.3, 0.2, 'setosa')]
...
... >>> echo(DATA)
... [('Sepal length', 'Sepal width', 'Petal length', 'Petal width', 'Species'), (5.8, 2.7, 5.1, 1.9, 'virginica'), (5.1, 3.5, 1.4, 0.2, 'setosa'), (5.7, 2.8, 4.1, 1.3, 'versicolor'), (6.3, 2.9, 5.6, 1.8, 'virginica'), (6.4, 3.2, 4.5, 1.5, 'versicolor'), (4.7, 3.2, 1.3, 0.2, 'setosa')]
...
... >>> echo(DATA)  
... [('Sepal length', 'Sepal width', 'Petal length', 'Petal width', 'Species'),
...  (5.8, 2.7, 5.1, 1.9, 'virginica'),
...  (5.1, 3.5, 1.4, 0.2, 'setosa'),
...  (5.7, 2.8, 4.1, 1.3, 'versicolor'),
...  (6.3, 2.9, 5.6, 1.8, 'virginica'),
...  (6.4, 3.2, 4.5, 1.5, 'versicolor'),
...  (4.7, 3.2, 1.3, 0.2, 'setosa')]
... """
>>>
>>> def echo(data):
...     return data

16.1.11. Test Exceptions

>>> 
... """
... >>> echo()
... Traceback (most recent call last):
... NotImplementedError
... """
>>>
>>> def echo():
...     raise NotImplementedError

>>> 
... """
... >>> echo()
... Traceback (most recent call last):
... NotImplementedError: This will work in future
... """
>>>
>>> def echo():
...     raise NotImplementedError('This will work in future')

16.1.12. Test Type

>>> 
... """
... >>> result = echo(1)
... >>> type(result)
... <class 'int'>
...
... >>> result = echo(1.1)
... >>> type(result)
... <class 'float'>
...
... >>> result = echo(True)
... >>> type(result)
... <class 'bool'>
...
... >>> result = echo([1, 2])
... >>> type(result)
... <class 'list'>
...
... >>> result = echo([1, 2])
... >>> any(type(x) is int
... ...     for x in result)
... True
... """
>>>
>>> def echo(data):
...     return data

The following doctest will fail:

>>> 
... """
... >>> add_numbers('one', 'two')
... Traceback (most recent call last):
... TypeError: not a number
...
... >>> add_numbers(True, 1)
... Traceback (most recent call last):
... ValueError: not a number
... """
>>>
>>> def add_numbers(a, b):
...     if not isinstance(a, (int, float)):
...         raise ValueError('c')
...     if not isinstance(b, (int, float)):
...         raise ValueError('not a number')
...     return a + b

Expected exception, got 2.0:

Expected:

Traceback (most recent call last): ValueError: not a number

Got:

2.0

This test will pass:

>>> 
... """
... >>> add_numbers('one', 'two')
... Traceback (most recent call last):
... TypeError: not a number
...
... >>> add_numbers(True, 1)
... Traceback (most recent call last):
... ValueError: not a number
... """
>>>
>>> def add_numbers(a, b):
...     if type(a) not in (int, float):
...         raise ValueError('not a number')
...     if type(b) not in (int, float):
...         raise ValueError('not a number')
...     return a + b

16.1.13. Test Python Expressions

Using python statements in doctest:

>>> def echo(text):
...     """
...     >>> name = 'Mark Watney'
...     >>> print(name)
...     Mark Watney
...     """
...     return text

>>> def when(date):
...     """
...     >>> from datetime import datetime, timezone
...     >>> moon = datetime(1969, 7, 21, 17, 54, tzinfo=timezone.utc)
...     >>> when(moon)
...     1969-07-21 17:54 UTC
...     """
...     print(f'{date:%Y-%m-%d %H:%M %Z}')

16.1.14. Flags

  • DONT_ACCEPT_TRUE_FOR_1

  • DONT_ACCEPT_BLANKLINE

  • NORMALIZE_WHITESPACE

  • ELLIPSIS

  • IGNORE_EXCEPTION_DETAIL

  • SKIP

  • COMPARISON_FLAGS

  • REPORT_UDIFF

  • REPORT_CDIFF

  • REPORT_NDIFF

  • REPORT_ONLY_FIRST_FAILURE

  • FAIL_FAST

  • REPORTING_FLAGS

16.1.15. Case Studies

Docstring used for doctest:

>>> def apollo_dsky(noun, verb):
...     """
...     This is the Apollo Display Keyboard
...     It takes noun and verb
...
...     >>> apollo_dsky(6, 61)
...     Program selected. Noun: 06, verb: 61
...
...     >>> apollo_dsky(16, 68)
...     Program selected. Noun: 16, verb: 68
...     """
...     print(f'Program selected. Noun: {noun:02}, verb: {verb:02}')

Celsius to Kelvin conversion:

>>> def celsius_to_kelvin(data):
...     """
...     >>> celsius_to_kelvin([1,2,3])
...     [274.15, 275.15, 276.15]
...
...     >>> celsius_to_kelvin((1,2,3))
...     [274.15, 275.15, 276.15]
...     """
...     return [x+273.15 for x in data]

>>> def celsius_to_kelvin(data):
...     """
...     >>> celsius_to_kelvin([1,2,3])
...     [274.15, 275.15, 276.15]
...
...     >>> celsius_to_kelvin((1,2,3))
...     (274.15, 275.15, 276.15)
...     """
...     cls = type(data)
...     return cls(x+273.15 for x in data)

Adding two numbers:

>>> def add_numbers(a, b):
...     """
...     >>> add_numbers(1, 2)
...     3.0
...     >>> add_numbers(-1, 1)
...     0.0
...     >>> add_numbers(0.1, 0.2)  
...     0.3000...
...     >>> add_numbers(1.5, 2.5)
...     4.0
...     >>> add_numbers(1, 1.5)
...     2.5
...     >>> add_numbers([1, 2], 3)
...     Traceback (most recent call last):
...     ValueError: not a number
...
...     >>> add_numbers(0, [1, 2])
...     Traceback (most recent call last):
...     ValueError: not a number
...
...     >>> add_numbers('one', 'two')
...     Traceback (most recent call last):
...     ValueError: not a number
...
...     >>> add_numbers(True, 1)
...     Traceback (most recent call last):
...     ValueError: not a number
...     """
...     if type(a) not in (int, float):
...         raise ValueError('not a number')
...
...     if type(b) not in (int, float):
...         raise ValueError('not a number')
...
...     return float(a + b)

Celsius to Kelvin temperature conversion:

>>> def celsius_to_kelvin(celsius):
...     """
...     >>> celsius_to_kelvin(0)
...     273.15
...
...     >>> celsius_to_kelvin(1)
...     274.15
...
...     >>> celsius_to_kelvin(-1)
...     272.15
...
...     >>> celsius_to_kelvin(-273.15)
...     0.0
...
...     >>> celsius_to_kelvin(-273.16)
...     Traceback (most recent call last):
...     ValueError: Negative Kelvin
...
...     >>> celsius_to_kelvin(-300)
...     Traceback (most recent call last):
...     ValueError: Negative Kelvin
...
...     >>> celsius_to_kelvin(True)
...     Traceback (most recent call last):
...     TypeError: Argument must be: int, float or Sequence[int, float]
...
...     >>> celsius_to_kelvin([0, 1, 2, 3])
...     [273.15, 274.15, 275.15, 276.15]
...
...     >>> celsius_to_kelvin({0, 1, 2, 3})
...     {273.15, 274.15, 275.15, 276.15}
...
...     >>> celsius_to_kelvin([0, 1, 2, -300])
...     Traceback (most recent call last):
...     ValueError: Negative Kelvin
...
...     >>> celsius_to_kelvin([0, 1, [2, 3], 3])
...     [273.15, 274.15, [275.15, 276.15], 276.15]
...     """
...     datatype = type(celsius)
...
...     if type(celsius) in {list, tuple, set, frozenset}:
...         return datatype(celsius_to_kelvin(x) for x in celsius)
...
...     if datatype not in {int, float}:
...         raise TypeError('Argument must be: int, float or Sequence[int, float]')
...
...     kelvin = celsius + 273.15
...
...     if kelvin < 0.0:
...         raise ValueError('Negative Kelvin')
...
...     return float(kelvin)

16.1.16. Assignments

Code 16.1. Solution
"""
* Assignment: Test Doctest Distance
* Required: no
* Complexity: easy
* Lines of code: 21 lines
* Time: 13 min

English:
    1. Write doctests to a functions which convert distance given in kilometers to meters
    2. Valid arguments:
        a. `int`
        b. `float`
    3. Invalid argumentm, raise exception `TypeError`:
        a. `str`
        b. `list[int]`
        c. `list[float]`
        d. `bool`
        e. any other type
    4. Returned distance must be float
    5. Returned distance cannot be negative
    6. Run doctests - all must succeed

Polish:
    1. Napisz doctesty do funkcji, która przeliczy dystans podany w kilometrach na metry
    2. Poprawne argumenty:
        a. `int`
        b. `float`
    3. Niepoprawne argumenty, podnieś wyjątek `TypeError`:
        a. `str`
        b. `list[int]`
        c. `list[float]`
        d. `bool`
        e. any other type
    4. Zwracany dystans musi być float
    5. Zwracany dystans nie może być ujemny
    6. Uruchom doctesty - wszystkie muszą się powieść

Hint:
    * 1 km = 1000 m

Tests:
    >>> import sys; sys.tracebacklimit = 0
"""

def km_to_meters(kilometers):
    if type(kilometers) not in {int, float}:
        raise TypeError('Invalid argument type')

    if kilometers < 0:
        raise ValueError('Argument must be not negative')

    return float(kilometers * 1000)
Code 16.2. Solution
"""
* Assignment: Test Doctest Temperature
* Required: no
* Complexity: easy
* Lines of code: 5 lines
* Time: 13 min

English:
    1. Write doctests to `celsius_to_kelvin` function
    2. Parameter `degrees` can be:
        a. int
        b. float
        c. list[int|float]
        d. tuple[int|float,...]
        e. set[int|float]
        f. In other case raise an exception: TypeError with message: "Invalid type"
    3. Run doctests - all must succeed

Polish:
    1. Napisz doctesty do funkcji `celsius_to_kelvin`
    2. Parametr `degrees` może być:
        a. int
        b. float
        c. list[int|float]
        d. tuple[int|float,...]
        e. set[int|float]
        f. W przeciwnym wypadku podnieś wyjątek: TypeError z komunikatem: "Invalid type"
    3. Uruchom doctesty - wszystkie muszą się powieść

Tests:
    >>> import sys; sys.tracebacklimit = 0
    >>> from inspect import isfunction
"""

def celsius_to_kelvin(degrees):
    if type(degrees) in (int, float):
        return 273.15 + degrees

    if type(degrees) in (list, tuple, set):
        cls = type(degrees)
        return cls(x+273.15 for x in degrees)

    raise TypeError('Invalid argument')