Reproducing DevStack-based OSC Gate Jobs Locally

The python-openstackclient project provides two types of tests: unit tests with mocked server responses and functional tests that require a real server deployed. The latter expect a specific DevStack-based deployment and attempts to run these tests against a “standard” DevStack deployment with minimal configuration or a deployment created by another tool with result in a lot of test failures.

==============================
Failed 48 tests - output below:
==============================

setUpClass (openstackclient.tests.functional.network.v2.test_network_meter_rule.TestMeterRule)
----------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/home/fedora/python-openstackclient/openstackclient/tests/functional/network/v2/test_network_meter_rule.py", line 34, in setUpClass
    json_output = json.loads(cls.openstack(

      File "/home/fedora/python-openstackclient/openstackclient/tests/functional/base.py", line 74, in openstack
    return execute(

      File "/home/fedora/python-openstackclient/openstackclient/tests/functional/base.py", line 41, in execute
    raise exceptions.CommandFailed(

    tempest.lib.exceptions.CommandFailed: Command 'openstack --os-cloud=devstack-admin network meter create -f json 82c6e512857e40d78e86b9db54c66e73' returned non-zero exit status 1.
stdout:

stderr:
b'ResourceNotFound: 404: Client Error for url: http://140.211.166.52:9696/v2.0/metering/metering-labels, The resource could not be found.\n'


openstackclient.tests.functional.network.v2.test_network_qos_rule.NetworkQosRuleTestsBandwidthLimit.test_qos_rule_create_delete
-------------------------------------------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/home/fedora/python-openstackclient/openstackclient/tests/functional/network/v2/test_network_qos_rule.py", line 161, in setUp
    self.openstack(

      File "/home/fedora/python-openstackclient/openstackclient/tests/functional/base.py", line 74, in openstack
    return execute(

      File "/home/fedora/python-openstackclient/openstackclient/tests/functional/base.py", line 41, in execute
    raise exceptions.CommandFailed(

    tempest.lib.exceptions.CommandFailed: Command 'openstack --os-cloud=devstack-admin network qos policy create qos_policy_178299ca49d544bb9d5c145ee4c71d80' returned non-zero exit status 1.
stdout:

stderr:
b'ResourceNotFound: 404: Client Error for url: http://140.211.166.52:9696/v2.0/qos/policies, The resource could not be found.\n'


openstackclient.tests.functional.network.v2.test_network_qos_rule.NetworkQosRuleTestsBandwidthLimit.test_qos_rule_list
----------------------------------------------------------------------------------------------------------------------

...

But how does one actually configure their system appropriately. Docs are non-existent but we know that the CI works so clearly that’s a good starting place. As you’ll likely know if you’re reading this, the OpenStack community uses Zuul for testing. You can see the zuul job configuration for projects in either a .zuul.yaml file or in various YAML files in a .zuul.d directory, both of which can be found in the root directory of the project. python-openstackclient uses the former, which you can see here. Looking at that, we can see the definition for the osc-functional- devstack job, which inherits from the osc-functional-devstack-base job, which in turns inherits from the devstack-tox-functional job and so on…

Looking at the osc-functional-devstack job, we can see that it enables the neutron DevStack plugin and sets a couple of neutron-related feature flags and some other misc attributes in DevStack via the devstack_plugins devstack_services, and devstack_localrc job variables, respectively. You can see the documentation for these variables in the DevStack docs and you can use the codesearch.o.o service to figure out what enabling each of these services or specifying those extra config variables will do, like this one. If you want to see the definition of the Ansible role that these variables are being passed to, you can look at that too.

If you read through all the docs, you’ll eventually come to the conclusion that to get this to work locally, you’re going to need to add the following to your local.conf when deploying DevStack:

# Plugin configuration
enable_plugin neutron https://opendev.org/openstack/neutron

# Service configuration
## Disable OVN services
disable_service br-ex-tcpdump
disable_service br-int-flows
disable_service ovn-controller
disable_service ovn-northd
disable_service ovs-vswitchd
disable_service ovsdb-server
disable_service q-ovn-metadata-agent

## Neutron services
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service neutron-network-segment-range
enable_service neutron-segments
enable_service q-metering
enable_service q-qos
enable_service neutron-tag-ports-during-bulk-creation
enable_service neutron-conntrack-helper

# Misc configuration
Q_AGENT=openvswitch
Q_ML2_TENANT_NETWORK_TYPE=vxlan
Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch

Add this to your otherwise uncomplicated local.conf and in theory things should just work (TM). There might be more to it but hopefully there’s enough here for folks to figure out how to debug things if so.