openstack/performance-docs
Code Issues Proposed changes

New scenario for QoS testing

Browse Source 
All other tenant networking scenarios are extended with new
parameters: jitter and packet loss.

Change-Id: I3ee35671f4474187daf7ff345e81f9e22fb05024
This commit is contained in:
Ilya Shakhat 2016-06-14 14:49:49 +03:00
parent 2c6ab19e43
commit bb99fab148
2 changed files with 113 additions and 54 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

145
doc/source/test_plans/tenant_networking/plan.rst
View File

@ -120,14 +120,16 @@ done during 60 seconds, average values are calculated.
List of performance metrics List of performance metrics
^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
======== =============== ================= ====================================== ======== ================ ================= ===========================================
Priority Value Measurement Units Description Priority Value Measurement Units Description
======== =============== ================= ====================================== ======== ================ ================= ===========================================
1 Latency ms The network latency 1 Latency ms The network latency
1 TCP bandwidth Mbits/s TCP network bandwidth 1 TCP bandwidth Mbits/s TCP network bandwidth
2 UDP bandwidth packets per sec Number of UDP packets of 32 bytes size 1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload
2 TCP retransmits packets per sec Number of retransmitted TCP packets 2 UDP delay jitter ms Packet delay variation
======== =============== ================= ====================================== 2 UDP packet loss % Percentage of lost UDP packets
2 TCP retransmits packets per sec Number of retransmitted TCP packets
======== ================ ================= ===========================================
.. _openstack_tenant_networking_test_plan_l2_concurrent: .. _openstack_tenant_networking_test_plan_l2_concurrent:
@ -152,14 +154,16 @@ Test case result is series of following measurements done at different
levels of concurrency. The output may be shown in table form and/or as chart levels of concurrency. The output may be shown in table form and/or as chart
showing dependency of parameters from concurrency. showing dependency of parameters from concurrency.
======== =============== ================= ====================================== ======== ================ ================= ===========================================
Priority Value Measurement Units Description Priority Value Measurement Units Description
======== =============== ================= ====================================== ======== ================ ================= ===========================================
1 Latency ms The network latency 1 Latency ms The network latency
1 TCP bandwidth Mbits/s TCP network bandwidth 1 TCP bandwidth Mbits/s TCP network bandwidth
2 UDP bandwidth packets per sec Number of UDP packets of 32 bytes size 1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload
2 TCP retransmits packets per sec Number of retransmitted TCP packets 2 UDP delay jitter ms Packet delay variation
======== =============== ================= ====================================== 2 UDP packet loss % Percentage of lost UDP packets
2 TCP retransmits packets per sec Number of retransmitted TCP packets
======== ================ ================= ===========================================
.. _openstack_tenant_networking_test_plan_l3_east_west: .. _openstack_tenant_networking_test_plan_l3_east_west:
@ -178,14 +182,16 @@ average values are calculated.
List of performance metrics List of performance metrics
^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
======== =============== ================= ====================================== ======== ================ ================= ===========================================
Priority Value Measurement Units Description Priority Value Measurement Units Description
======== =============== ================= ====================================== ======== ================ ================= ===========================================
1 Latency ms The network latency 1 Latency ms The network latency
1 TCP bandwidth Mbits/s TCP network bandwidth 1 TCP bandwidth Mbits/s TCP network bandwidth
2 UDP bandwidth packets per sec Number of UDP packets of 32 bytes size 1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload
2 TCP retransmits packets per sec Number of retransmitted TCP packets 2 UDP delay jitter ms Packet delay variation
======== =============== ================= ====================================== 2 UDP packet loss % Percentage of lost UDP packets
2 TCP retransmits packets per sec Number of retransmitted TCP packets
======== ================ ================= ===========================================
.. _openstack_tenant_networking_test_plan_l3_east_west_concurrent: .. _openstack_tenant_networking_test_plan_l3_east_west_concurrent:
@ -212,14 +218,16 @@ Test case result is series of following measurements done at different
levels of concurrency. The output may be shown in table form and/or as chart levels of concurrency. The output may be shown in table form and/or as chart
showing dependency of parameters from concurrency. showing dependency of parameters from concurrency.
======== =============== ================= ====================================== ======== ================ ================= ===========================================
Priority Value Measurement Units Description Priority Value Measurement Units Description
======== =============== ================= ====================================== ======== ================ ================= ===========================================
1 Latency ms The network latency 1 Latency ms The network latency
1 TCP bandwidth Mbits/s TCP network bandwidth 1 TCP bandwidth Mbits/s TCP network bandwidth
2 UDP bandwidth packets per sec Number of UDP packets of 32 bytes size 1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload
2 TCP retransmits packets per sec Number of retransmitted TCP packets 2 UDP delay jitter ms Packet delay variation
======== =============== ================= ====================================== 2 UDP packet loss % Percentage of lost UDP packets
2 TCP retransmits packets per sec Number of retransmitted TCP packets
======== ================ ================= ===========================================
.. _openstack_tenant_networking_test_plan_l3_north_south: .. _openstack_tenant_networking_test_plan_l3_north_south:
@ -239,14 +247,16 @@ The measurement is done during 60 seconds, average values are calculated.
List of performance metrics List of performance metrics
^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
======== =============== ================= ====================================== ======== ================ ================= ===========================================
Priority Value Measurement Units Description Priority Value Measurement Units Description
======== =============== ================= ====================================== ======== ================ ================= ===========================================
1 Latency ms The network latency 1 Latency ms The network latency
1 TCP bandwidth Mbits/s TCP network bandwidth 1 TCP bandwidth Mbits/s TCP network bandwidth
2 UDP bandwidth packets per sec Number of UDP packets of 32 bytes size 1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload
2 TCP retransmits packets per sec Number of retransmitted TCP packets 2 UDP delay jitter ms Packet delay variation
======== =============== ================= ====================================== 2 UDP packet loss % Percentage of lost UDP packets
2 TCP retransmits packets per sec Number of retransmitted TCP packets
======== ================ ================= ===========================================
.. _openstack_tenant_networking_test_plan_l3_north_south_concurrent: .. _openstack_tenant_networking_test_plan_l3_north_south_concurrent:
@ -274,14 +284,53 @@ Test case result is series of following measurements done at different
levels of concurrency. The output may be shown in table form and/or as chart levels of concurrency. The output may be shown in table form and/or as chart
showing dependency of parameters from concurrency. showing dependency of parameters from concurrency.
======== =============== ================= ====================================== ======== ================ ================= ===========================================
Priority Value Measurement Units Description Priority Value Measurement Units Description
======== =============== ================= ====================================== ======== ================ ================= ===========================================
1 Latency ms The network latency 1 Latency ms The network latency
1 TCP bandwidth Mbits/s TCP network bandwidth 1 TCP bandwidth Mbits/s TCP network bandwidth
2 UDP bandwidth packets per sec Number of UDP packets of 32 bytes size 1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload
2 TCP retransmits packets per sec Number of retransmitted TCP packets 2 UDP delay jitter ms Packet delay variation
======== =============== ================= ====================================== 2 UDP packet loss % Percentage of lost UDP packets
2 TCP retransmits packets per sec Number of retransmitted TCP packets
======== ================ ================= ===========================================
.. _openstack_tenant_networking_test_plan_qos:
Test Case 7: Neutron QoS testing
--------------------------------
Description
^^^^^^^^^^^
This test case is used to verify Neutron QoS feature as scale. The feature
allows to limit the traffic bandwidth in a particular network. To simplify
testing instances are deployed in the same network (L2 domain). The target
bandwidth is specified in kBits/s.
.. note::
In order to work Neutron QoS extension must be installed and properly
configured.
List of performance metrics
^^^^^^^^^^^^^^^^^^^^^^^^^^^
======== ================ ================= ===========================================
Priority Value Measurement Units Description
======== ================ ================= ===========================================
1 Latency ms The network latency
1 TCP bandwidth Mbits/s TCP network bandwidth
1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload
2 UDP delay jitter ms Packet delay variation
2 UDP packet loss % Percentage of lost UDP packets
2 TCP retransmits packets per sec Number of retransmitted TCP packets
======== ================ ================= ===========================================
It's expected that achieved TCP bandwidth is not higher that the one set in
Neutron QoS policy.
Tools Tools

22
doc/source/test_plans/tenant_networking/shaker.rst
View File

@ -53,7 +53,7 @@ Test Case 1: L2 instance-to-instance performance
**Execution**:: **Execution**::
shaker --server-endpoint <host:port> --scenario networking/perf_l2 --book <report folder> shaker --server-endpoint <host:port> --scenario openstack/perf_l2 --book <report folder>
Test Case 2: L2 concurrent performance Test Case 2: L2 concurrent performance
@ -64,7 +64,7 @@ Test Case 2: L2 concurrent performance
**Execution**:: **Execution**::
shaker --server-endpoint <host:port> --scenario networking/full_l2 --book <report folder> shaker --server-endpoint <host:port> --scenario openstack/full_l2 --book <report folder>
Test Case 3: L3 east-west instance-to-instance performance Test Case 3: L3 east-west instance-to-instance performance
@ -74,7 +74,7 @@ Test Case 3: L3 east-west instance-to-instance performance
**Execution**:: **Execution**::
shaker --server-endpoint <host:port> --scenario networking/perf_l3_east_west --book <report folder> shaker --server-endpoint <host:port> --scenario openstack/perf_l3_east_west --book <report folder>
Test Case 4: L3 east-west concurrent performance Test Case 4: L3 east-west concurrent performance
@ -84,7 +84,7 @@ Test Case 4: L3 east-west concurrent performance
**Execution**:: **Execution**::
shaker --server-endpoint <host:port> --scenario networking/full_l3_east_west --book <report folder> shaker --server-endpoint <host:port> --scenario openstack/full_l3_east_west --book <report folder>
Test Case 5: L3 north-south instance-to-instance performance Test Case 5: L3 north-south instance-to-instance performance
@ -94,7 +94,7 @@ Test Case 5: L3 north-south instance-to-instance performance
**Execution**:: **Execution**::
shaker --server-endpoint <host:port> --scenario networking/perf_l3_north_south --book <report folder> shaker --server-endpoint <host:port> --scenario openstack/perf_l3_north_south --book <report folder>
Test Case 6: L3 north_south concurrent performance Test Case 6: L3 north_south concurrent performance
@ -104,7 +104,17 @@ Test Case 6: L3 north_south concurrent performance
**Execution**:: **Execution**::
shaker --server-endpoint <host:port> --scenario networking/full_l3_north_south --book <report folder> shaker --server-endpoint <host:port> --scenario openstack/full_l3_north_south --book <report folder>
Test Case 7: Neutron QoS testing
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Test case specification**: :ref:`openstack_tenant_networking_test_plan_qos`
**Execution**::
shaker --server-endpoint <host:port> --scenario openstack/qos/perf_l2 --book <report folder>
.. references: .. references: