cloud-init has both unit tests and integration tests. Unit tests can
be found at
tests/unittests. Integration tests can be found at
tests/integration_tests. Documentation specifically for integration
tests can be found on the Integration Testing page, but
the guidelines specified below apply to both types of tests.
cloud-init uses pytest to run its tests, and has tests written both
unittest.TestCase sub-classes and as un-subclassed pytest tests.
The following guidelines should be followed.
For ease of organisation and greater accessibility for developers not familiar with pytest, all cloud-init unit tests must be contained within test classes
Put another way, module-level test functions should not be used
As all tests are contained within classes, it is acceptable to mix
TestCasetest classes and pytest test classes within the same test file
Unit tests and integration tests are located under cloud-init/tests
For consistency, unit test files should have a matching name and directory location under tests/unittests
For example: the expected test file for code in cloudinit/path/to/file.py is tests/unittests/path/to/test_file.py
pytest Version Gotchas
As we still support Ubuntu 18.04 (Bionic Beaver), we can only use pytest features that are available in v3.3.2. This is an inexhaustive list of ways in which this may catch you out:
Only the following built-in fixtures are available 1:
Mocking and Assertions
Variables/parameter names for
MagicMockinstances should start with
m_to clearly distinguish them from non-mock variables
m_readurl(which would be a mock for
assert_*methods that are available on
MagicMockobjects should be avoided, as typos in these method names may not raise
AttributeError(and so can cause tests to silently pass). An important exception: if a
Mockis autospecced then misspelled assertion methods will raise an
AttributeError, so these assertion methods may be used on autospecced
Mocks, these substitutions can be used (
mis assumed to be a
assert mock.call(*args, **kwargs) in m.call_args_list
assert 0 != m.call_count
assert 1 == m.call_count
assert [mock.call(*args, **kwargs)] == m.call_args_list
assert mock.call(*args, **kwargs) == m.call_args_list[-1]
for call in call_list: assert call in m.call_args_list
m.assert_has_calls(..., any_order=False)are not easily replicated in a single statement, so their use when appropriate is acceptable.
assert 0 == m.call_count
When there are multiple patch calls in a test file for the module it is testing, it may be desirable to capture the shared string prefix for these patch calls in a module-level variable. If used, such variables should be named
M_PATHor, for datasource tests,
Test Argument Ordering
Test arguments should be ordered as follows:
mock.patcharguments. When used as a decorator,
mock.patchpartially applies its generated
Mockobject as the first argument, so these arguments must go first.
pytest.mark.parametrizearguments, in the order specified to the
parametrizedecorator. These arguments are also provided by a decorator, so it’s natural that they sit next to the
Fixture arguments, alphabetically. These are not provided by a decorator, so they are last, and their order has no defined meaning, so we default to alphabetical.
It follows from this ordering of test arguments (so that we retain the property that arguments left-to-right correspond to decorators bottom-to-top) that test decorators should be ordered as follows:
This list of fixtures (with markup) can be reproduced by running:
python3 -m pytest --fixtures -q | grep "^[^ -]" | grep -v 'no tests ran in' | sort | sed 's/ \[session scope\]//g;s/.*/* ``\0``/g'
in an ubuntu lxd container with python3-pytest installed.