Testing your software in Python

We’ll follow the concise Software Carpentry Testing Tutorial authored by Katy Huff.

A simple, testable mean

Let’s checkpoint here with a directory that should have:

• mymean.py:

def mean(num_list):
    first = num_list[0]
    if isinstance(first, complex):
        return NotImplemented
    return sum(num_list)/len(num_list)

• tests.py:

from mymean import mean

def test_mean():
    assert mean([1]) == 1

def test_ints():
    num_list = [1, 2, 3, 4, 5]
    obs = mean(num_list)
    exp = 3
    assert obs == exp

def test_zero():
    num_list=[0,2,4,6]
    obs = mean(num_list)
    exp = 3
    assert obs == exp

def test_double():
    # This one will fail in Python 2
    num_list=[1,2,3,4]
    obs = mean(num_list)
    exp = 2.5
    assert obs == exp

def test_long():
    big = 100000000
    obs = mean(range(1,big))
    exp = big/2.0
    assert obs == exp

def test_complex():
    # given that complex numbers are an unordered field
    # the arithmetic mean of complex numbers is meaningless
    num_list = [2 + 3j, 3 + 4j, -32 - 2j]
    obs = mean(num_list)
    exp = NotImplemented
    assert obs == exp

• And let’s also put in there a simple .gitignore:

.DS_Store
*~
__pycache__
*.py[co]
.coverage
.ipynb_checkpoints
Untitled*.ipynb
_configtest.o.d

Next, let’s make this into a public repo on our personal accounts. Here’s mine (note that one has a few more things that we’ll add later).

Continuous Integration with Travis

Let’s setup Travis CI for Github, using the free plan for Open Source:

You’ll need to accept the Github authorization for Travis to access your account. More info here on the Travis docs.

Once that’s set up and your repo is up on github, go to your Travis Profile page and turn on the green checkmark to authorize this repo for Travis builds:

Note: if you don’t see your repo listed, click on the green “Sync Account” button in the top right.

Running your tests on Travis

Now, let’s have Travis run the tests for us on each commit, automatically!

Add the following, absolutely minimal .travis.yml file to your repo:

language: python
python:
  - "3.6"
install:
  - pip install pytest
script:
  - pytest tests.py

Once you make a git push to your repo, you should see a build start, e.g:

and if all goes well, you should then see something like this in the logs:

Exercise

• Add a test of your mean function that uses numpy:
• Generate 10,000 uniformly distributed random numbers with numpy.
• Test that your mean is a proper numerical match to the numpy one.
• Once this test passes locally, add it to your Travis run.

Travis + Conda

If you want to build your travis run with Conda, you can create a proper environment as part of your Travis setup, here’s a simple example .travis.yml that uses Conda:

language: python
python:
  # We don't actually use the Travis Python, but this keeps it organized.
  - "3.6"
install:
  - sudo apt-get update
  - wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh;
  - bash miniconda.sh -b -p $HOME/miniconda
  - export PATH="$HOME/miniconda/bin:$PATH"
  - hash -r
  - conda config --set always_yes yes --set changeps1 no
  - conda update -q conda
  # Useful for debugging any issues with conda
  - conda info -a

  # List your dependencies here to create the necessary environment
  - conda create -q -n test-environment python=$TRAVIS_PYTHON_VERSION pytest
  - source activate test-environment

script:
  - pytest tests.py

Exercise: conda with an environment.yml file for Travis

Explore whether it’s possible to modify the above Travis setup to use a proper environment.yml file.

Exercise: add a README and status badge to your repo

If you add a README file, you can display in it the build status of your repo thanks to Travis’ status badges. It will look like this:

and that badge is actually a live link to your Travis build page for the repo.