OpenStack Nova allows the specification of image metadata to provide per-instance metadata. This is similar to the flavor extra specs. The CPU pinning feature makes use of this to allow for configuration of CPU policies and CPU thread policies via such metdata.
Someone suggested to me that CPU policies many not be being applied correctly when stored as image metadata rather than flavor properties. I decided to investigate this claim.
Commit 8bafc9 of Nova was used for this testing. Much of
this feature has been documented upstreamed since conducting
this testing.
Steps
Create a custom image with metadata
The first step is to create a new image and save metadata against this image. To do this, we’ll duplicate an existing image and modify this duplicate. Begin by duplicating the image:
$ openstack image list
+--------------------------------------+---------------------------------+--------+
| ID | Name | Status |
+--------------------------------------+---------------------------------+--------+
| a3450a21-f012-4807-a514-0838750d430e | cirros-0.3.4-x86_64-uec | active |
| 742bc914-e60c-4eab-9e5b-0a1c19ec1a84 | cirros-0.3.4-x86_64-uec-ramdisk | active |
| 3e507ab7-080d-4eea-845a-69950ec139b8 | cirros-0.3.4-x86_64-uec-kernel | active |
+--------------------------------------+---------------------------------+--------+
$ openstack image save --file cirros.img a3450a21-f012-4807-a514-0838750d430e
$ openstack image create --file cirros.img cirros-0.3.4-x86_64-pinned
+------------------+------------------------------------------------------+
| Field | Value |
+------------------+------------------------------------------------------+
| checksum | eb9139e4942121f22bbc2afc0400b2a4 |
| container_format | bare |
| created_at | 2016-02-16T09:40:12Z |
| disk_format | raw |
| file | /v2/images/e3e7d5a5-b044-493a-b3e1-fa9ba6225ed6/file |
| id | e3e7d5a5-b044-493a-b3e1-fa9ba6225ed6 |
| min_disk | 0 |
| min_ram | 0 |
| name | cirros-0.3.4-x86_64-pinned |
| owner | c1e8c7671b0f4016a9250a8787de6930 |
| protected | False |
| schema | /v2/schemas/image |
| size | 25165824 |
| status | active |
| tags | |
| updated_at | 2016-02-16T09:40:12Z |
| virtual_size | None |
| visibility | private |
+------------------+------------------------------------------------------+
Next up, add the metadata to the image. We’ll only worry about CPU pinning, by way of CPU policies, ignoring CPU thread pinning for now.
$ openstack image set cirros-0.3.4-x86_64-pinned --property hw_cpu_policy=dedicated
$ openstack image list
+--------------------------------------+---------------------------------+--------+
| ID | Name | Status |
+--------------------------------------+---------------------------------+--------+
| e3e7d5a5-b044-493a-b3e1-fa9ba6225ed6 | cirros-0.3.4-x86_64-pinned | active |
| a3450a21-f012-4807-a514-0838750d430e | cirros-0.3.4-x86_64-uec | active |
| 742bc914-e60c-4eab-9e5b-0a1c19ec1a84 | cirros-0.3.4-x86_64-uec-ramdisk | active |
| 3e507ab7-080d-4eea-845a-69950ec139b8 | cirros-0.3.4-x86_64-uec-kernel | active |
+--------------------------------------+---------------------------------+--------+
$ openstack image show cirros-0.3.4-x86_64-pinned
+------------------+------------------------------------------------------+
| Field | Value |
+------------------+------------------------------------------------------+
| checksum | eb9139e4942121f22bbc2afc0400b2a4 |
| container_format | bare |
| created_at | 2016-02-16T09:40:12Z |
| disk_format | raw |
| file | /v2/images/e3e7d5a5-b044-493a-b3e1-fa9ba6225ed6/file |
| id | e3e7d5a5-b044-493a-b3e1-fa9ba6225ed6 |
| min_disk | 0 |
| min_ram | 0 |
| name | cirros-0.3.4-x86_64-pinned |
| owner | c1e8c7671b0f4016a9250a8787de6930 |
| properties | hw_cpu_policy='dedicated' |
| protected | False |
| schema | /v2/schemas/image |
| size | 25165824 |
| status | active |
| tags | |
| updated_at | 2016-02-16T09:41:42Z |
| virtual_size | None |
| visibility | private |
+------------------+------------------------------------------------------+
Create a new instance
Now create the flavor. There’s no need to specify any metadata on the flavors: it’s already specified in the image:
$ openstack flavor list
+----+-----------+-------+------+-----------+-------+-----------+
| ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+----+-----------+-------+------+-----------+-------+-----------+
| 1 | m1.tiny | 512 | 1 | 0 | 1 | True |
| 2 | m1.small | 2048 | 20 | 0 | 1 | True |
| 3 | m1.medium | 4096 | 40 | 0 | 2 | True |
| 4 | m1.large | 8192 | 80 | 0 | 4 | True |
| 42 | m1.nano | 64 | 0 | 0 | 1 | True |
| 5 | m1.xlarge | 16384 | 160 | 0 | 8 | True |
| 84 | m1.micro | 128 | 0 | 0 | 1 | True |
+----+-----------+-------+------+-----------+-------+-----------+
$ openstack server create --flavor=m1.small --image=cirros-0.3.4-x86_64-pinned --wait test1
Validate that the instance has been created
Finally, ensure that things behave as expected. Instances should define an NUMA topology and CPU pinning in the libvirt XML file:
$ openstack server list
+--------------------------------------+-------+--------+--------------------------------------------------------+
| ID | Name | Status | Networks |
+--------------------------------------+-------+--------+--------------------------------------------------------+
| 22c1afeb-06d8-4f6d-a8d8-7ea40ea9ff47 | test1 | ACTIVE | private=fd4d:adb9:5ebe:0:f816:3eff:fecd:a674, 10.0.0.3 |
+--------------------------------------+-------+--------+--------------------------------------------------------+
$ sudo virsh list
Id Name State
----------------------------------------------------
4 instance-00000002 running
$ sudo virsh dumpxml 4
<domain type='kvm' id='4'>
<name>instance-00000002</name>
<vcpu placement='static'>1</vcpu>
...
<cputune>
<shares>1024</shares>
<vcpupin vcpu='0' cpuset='1'/>
<emulatorpin cpuset='1'/>
</cputune>
<numatune>
<memory mode='strict' nodeset='0'/>
<memnode cellid='0' mode='strict' nodeset='0'/>
</numatune>
...
</domain>
Result
The image metadata is being loaded as expected. No issues here.