Closing the OpenStackClient (and OpenStackSDK) Gaps

The unified OpenStack command-line client project, python-openstackclient (OSC), has been around for over a decade. As noted in the README, “[t]he primary goal is to provide a unified shell command structure and a common language to describe operations in OpenStack.” In essence, this means OSC should replace the shell command implementations of the various project-specific clients, such as python-novaclient and python-glanceclient. It does not aim to replace the client library implementations (Python binding) of these libraries: that task is handled by another project, the unified OpenStack client library, openstacksdk (SDK).

For most high-level “simpler” OpenStack-related activities, OSC has long been “good enough” as it provided the ability to do CRUD operations on the essential resources in a cloud including instances, volumes and images. However, as soon as you started trying to use more advanced or less common features, you’d run into a swathe of feature gaps. These would generally mean having to revert to using the legacy project-specific clients like python-novaclient (e.g. nova foo) or python-glanceclient (glance foo). There were exceptions to this, most notably neutronclient, but ultimately the result of this partial switchover was a poor user experience that was more confusing than it should have been.

Over the course of the past number of cycles (starting with Train), there has been a concerted effort to close these gaps and achieve feature parity between the shells of the various legacy clients and OSC. As of Yoga, we have effectively achieved this for nova and are well on our way to doing so for cinder.

I’ve put together a little FAQ below that should be helpful in helping people understand what has gone into this effort and how you can identify features gaps to to be closed.

How do we determine where these gaps lie?

Unfortunately, pretty manually. OSC only aims to implement support for the six “core” services: nova, neutron, keystone, cinder, glance, and swift. We provide CSV files mapping commands found in the legacy clients with their OSC equivalents. The first step is looking for obvious gaps in this document, by which I mean missing entries for an OSC equivalent to a given legacy client command.

The next step is looking for command that should exist in those documents but don’t. This is pretty easy: just run e.g. nova help, capture the output, and reformat it so that it resembles the CSV file (this would actually be one of the few good places where scripting could help!). It’s worth noting that many of the legacy clients change their help output depending on the API version requested so you should run this a couple of times with different API versions. For example:

$ OS_COMPUTE_API_VERSION=2.88 nova help

The final and most involved step required is examining individual commands. There are two things we’re concerned with: missing options and missing microversion support. Assessing the first can be achieved similarly to identifying missing commands: just run e.g. nova help boot, capture the output and compare the options supported with those supported by the equivalent OSC command (openstack server create in this case). The second is a little trickier and usually involves manual inspection of the corresponding shell and client library code in the legacy code. You could also look at the API reference documentation for the particular service but projects like Cinder have large documentation gaps that means the client is a better source of truth.

How do I fix them?

Just push a patch. It probably makes sense to push a highly WIP patch first since OSC has some design conventions, noted in the contributor guide, which can take some time to get a feel for. There are also plenty of existing examples to look for: search for the topics cinder-gaps and nova-gaps to get an idea.

What about non-“core” projects?

OSC is built on top of osc-lib, which is in turn built on top of cliff. cliff is highly extensible and provides a number of hook points that allow users to build plugins which can extend an existing CLI tool. Projects like manila, ironic and cyborg all provide their own OSC plugins. This is also the mechanism that neutron uses to provide more advanced commands. Many of these projects never had a legacy client to speak of, so there are fewer (if any) gaps to worry about.

What about OpenStackSDK?

We’re working on that too and the work has been mostly going on in parallel. There has been debate on whether newly added commands should use SDK or the legacy client under the hood. I tend to favour using the latter first and switching to SDK later since it makes it easier to compare the OSC and legacy client shell implementations during development and review. In addition, an OSC command can sometimes be the first real-world user of the SDK implementation which can highlight bugs. This is particularly an issue for the quirkier, less utilized APIs. It would be a shame to get stuck implementing an OSC command due to an SDK bug. The sole exception to this is where library clients have removed support for an API, such as Cinder dropping support for API v2 in cinderclient 9.0.0. In this case, we’ve no choice but to use SDK.

Where can I get more info? / How can I get involved?

Just ask! You can find us on IRC at #openstack-sdks (OFTC) or via email at [email protected].

comments powered by Disqus