An Update on OpenAPI in Openstack

I previously presented on our work to bring OpenAPI to OpenStack as part of the 2024 OpenInfra Summit Asia, the slides for which you can find here. Since that talk, another release cycle has come and gone and our work has continued to progress. Below is a summary of the current “state of play” for OpenAPI support in OpenStack and a reminder of our long-term goals in the area.

Overview

The work to add OpenAPI schemas can be broken into two parts: adding schemas to services and writing tooling to consume these schemas and generate both documentation and clients.

Services

The tl;dr: of this section is that we are relying on the fact that OpenAPI 3.1 is a superset of JSON Schema Draft 2020-12 and are adding JSON Schema schemas to as many services as possible. When put this way, it sounds pretty simple but as is often the case, the devil is in the detail. OpenStack is made up of multiple independent services maintained by different groups of people, and while the Oslo project has helped ensure some level of consistency around things like configuration (oslo.conf), database connectivity (oslo.db) and messaging (oslo.messaging), there is huge variance in the API frameworks used:

Nova, Cinder, Glance, and Manila use a combination of Routes (for routing), WebOb (for request/response models), Paste (for application dispatch) and PasteDeploy (for middleware).
Neutron uses the same Routes + WebOb + Paste + PasteDeploy combo used by Nova, Cinder, Glance, and Manila, but with the addition of Pecan in place of the “homegrown” WSGI frameworks used in those projects.
Keystone uses Flask and Flask-Restful.
Swift uses PasteDeploy (but I admittedly know very little about Swift).
Ironic uses Pecan (which in turn uses WebOb, so you’ll see references to this too)
etc etc…

Fortunately, despite the wide differences in frameworks, they pretty much all have the same building blocks:

Controllers (or Applications) which are responsible for a given resource and handles calls for same. These are typically classes with methods for each HTTP verb that the API supports.
Middleware that inspects and/or modifies requests and responses.
Routers that map a HTTP request to a controller. This is typically implemented as a special type of middleware.

In this model, both parsing of requests and generation of responses happens in the controller methods, which makes this a natural place to add validation for same. And there are a few things we want to validate:

API version (for services that uses API microversions)
Request path and query string parameters
Request and response bodies

By wrapping the controller methods with decorators, we we are able to inspect both the request or response objects, comparing them against schemas for same.

Tooling

Having schemas in place for all services is of little help if we don’t do anything with them. To this end, we want tooling that can inspect services and extract their JSON Schema schemas, combining them to produce OpenAPI schemas.

Once we have these OpenAPI schemas, we can start generating (self-validating) documentation and clients/libraries. For the former, OpenAPI will replace the os-api-ref Sphinx extension currently used across OpenStack. os-api-ref allows us to describe our APIs in reStructuredText and it is a tool that has worked relatively well for us, but the lack of machine readability means it’s hard to validate against the code. For the latter, we hope to lessen the burden of maintaining libraries and clients in multiple languages, as this has proven very challenging to do given the very large number of OpenStack APIs in multiple. Once again, OpenAPI is better suited to this challenge than os-api-ref or anything else we have.

Current status and future plans

Nova

Nova started from the best position, since it was already using JSON Schema for request validation. However, there were a couple of issues to overcome:

The version of JSON Schema used for the schemas, Draft 04, is over 12 years old, meaning the schemas needed migrating to Draft 2020-12.
There were two different mechanisms for API versioning: if-else checks inside the controller methods, plus a decorator that relied on the descriptor protocol to allow us to define version-specific controller methods. The latter made inspection of the API router more difficult than necessary and had to be replaced.
While many APIs had request query string and request body schemas, not all of them did.
Most importantly, there were no schemas for response bodies.

Most of these have now been resolved. We have bumped our schemas to Draft 2020-12, we have added all the missing request schemas, and we have added response body schemas for a number of resources. The outstanding changes to address the API versioning issues and add the remaining response body schemas are all ready and just waiting for review, so with any luck we will be able to close this out in the 2025.2 (Flamingo) release. Once this work has merged, the final step will be to start generating api-ref documentation from the OpenAPI schemas instead of using the os-api-ref tool we currently use. The patches for Nova can be found here.

Cinder

Cinder was in a very similar position to Nova thanks to their shared lineage. This means it had the same advantage - pre-existing use of JSON Schema for request validation - and the same issues. However, it also had the added issue of having a mountain of tech debt mainly related to the support for multiple API versions that Cinder offered until relatively recently. This has necessitated a lot of additional cleanup patches to do things like remove the cinder.api.v2 module and consolidate everything under cinder.api.v3.

Unfortunately, while the bulk of the changes have been written, none of them have merged. I’m hoping we can double down on this in 2025.2 (Flamingo) release and start making some progress. The patches for Cinder can be found here.

Keystone

Once again, Keystone had the good fortune of having existing use of JSON Schema for request query string and request body validation, but this was done inconsistently and didn’t cover response body schemas. Most of the work here has focused on adding these missing schemas and making the router easier to inspect by moving validation from inside the controller methods to decorators and splitting methods that previously handling multiple endpoints (e.g. GET /foo and GET /foo/123) into multiple single-purpose methods.

Many resources have been fully specced by now but some remain. In addition, there are some decisions to be made regarding support for undocumented API options that have been (re)discovered during this work. None of it should be insurmountable, however. The patches for Keystone can be found here.

Manila

Yet another service with JSON Schema already in place. Once again, the work here consists of adding missing schemas and making the router easier to inspect. A small number of patches have merged here and more are in-flight. The patches for Manila can be found here.

Ironic

Ironic hasn’t historically used JSON Schema for validation, instead using its own homegrown validation framework and taking advantage of some Pecan functionality to maintain API versioning. As a result, there’s quite a bit of work needed to get schemas into Ironic:

Move API versioning to decorators
Rework request path, query string and body parameter validation to use JSON Schema schemas (via decorators) instead of the homegrown framework
Add request body parameter validation

This work is in very early stages, but there are people working on it and there appears to be broad buy-in from the Ironic team, so I hope to see significant progress over the course of the 2025.2 (Flamingo) cycle. The patches for Ironic can be found here.

Other services (Neutron, Swift, Glance, Octavia, …)

To the best of my knowledge, no work has been started in other projects. Of the “core” projects, I expect the Glance effort to be relatively small since they already have formal schemas (exposed via the API!) for most of their resources. Conversely, I expect the Neutron effort to be both large and complicated, given the number and highly dynamic nature of Neutron’s API, driven by it’s extension-based “versioning” system. I have somehow never worked on Swift and haven’t a clue about that.

Tooling

There are two tooling-related efforts ongoing:

The codegenerator project provides a collection of utilities for generating OpenAPI schemas and a (Rust-based 🦀) client from these schemas. This tool has continued to evolve over the course of the 2025.1 (Epoxy) cycle, and I expect to see more progress as schemas continue to get added to the Nova project.
The openstacksdk project provides the primary OpenStack client library, while the python-openstackclient project is the primary OpenStack CLI. Both have seen significant work towards typing and removing “weirdness”, in order to prepare them for a future when they can both be at least partially automatically generated by another tool. This effort has included work on the dependencies of the projects: keystoneauth for openstacksdk, and cliff and osc-lib for python-openstackclient. The end goal is to have a fully-typed code base that includes an openstacksdk Resource class that no longer munges request and response parameters and proxy layers that actually expose the parameters each method supports rather than abusing *args and **kwargs. It’s a huge goal but, again, we are making steady progress here.

Questions?

Come discuss these topics in the OpenStack Virtual PTG, running from 07-11 April 2025. More information of the event and sessions can be found at ptg.opendev.org.