openstack/ironic
Code Issues Proposed changes

CI: increase cleaning timeout and tie it to PXE boot timeout

Browse Source 
We're seeing cases where cleaning barely manages to finish after
a 2nd PXE retry, failing a job.

Also make the PXE retry timeout consistent between the CI and
local devstack installations.

Change-Id: I6dc7a91d1a482008cf4ec855a60a95ec0a1abe28
This commit is contained in:
Dmitry Tantsur 2020-10-27 08:45:20 +01:00
parent 5e68f79102
commit 87e634dee1
2 changed files with 8 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
devstack/lib/ironic
View File

@ -97,7 +97,7 @@ IRONIC_CALLBACK_TIMEOUT=${IRONIC_CALLBACK_TIMEOUT:-}
# Timeout before retrying PXE boot. Set low to help the CI. # Timeout before retrying PXE boot. Set low to help the CI.
if [[ "$IRONIC_IS_HARDWARE" == False ]]; then if [[ "$IRONIC_IS_HARDWARE" == False ]]; then
IRONIC_PXE_BOOT_RETRY_TIMEOUT=${IRONIC_PXE_BOOT_RETRY_TIMEOUT:-600} IRONIC_PXE_BOOT_RETRY_TIMEOUT=${IRONIC_PXE_BOOT_RETRY_TIMEOUT:-900}
else else
IRONIC_PXE_BOOT_RETRY_TIMEOUT=${IRONIC_PXE_BOOT_RETRY_TIMEOUT:-} IRONIC_PXE_BOOT_RETRY_TIMEOUT=${IRONIC_PXE_BOOT_RETRY_TIMEOUT:-}
fi fi
@ -665,9 +665,13 @@ fi
# Timeout for "manage" action. 2 minutes is more than enough. # Timeout for "manage" action. 2 minutes is more than enough.
IRONIC_MANAGE_TIMEOUT=${IRONIC_MANAGE_TIMEOUT:-120} IRONIC_MANAGE_TIMEOUT=${IRONIC_MANAGE_TIMEOUT:-120}
# Timeout for "provide" action. This involves cleaning. Generally, 15 minutes # Timeout for "provide" action. This involves cleaning.
# should be enough, but real hardware may need more. if [[ -n "$IRONIC_PXE_BOOT_RETRY_TIMEOUT" ]]; then
IRONIC_CLEANING_TIMEOUT=${IRONIC_CLEANING_TIMEOUT:-1200} IRONIC_DEFAULT_CLEANING_TIMEOUT=$(( $IRONIC_PXE_BOOT_RETRY_TIMEOUT * 2 ))
else
IRONIC_DEFAULT_CLEANING_TIMEOUT=1800
fi
IRONIC_CLEANING_TIMEOUT=${IRONIC_CLEANING_TIMEOUT:-$IRONIC_DEFAULT_CLEANING_TIMEOUT}
IRONIC_CLEANING_DELAY=10 IRONIC_CLEANING_DELAY=10
IRONIC_CLEANING_ATTEMPTS=$(( $IRONIC_CLEANING_TIMEOUT / $IRONIC_CLEANING_DELAY )) IRONIC_CLEANING_ATTEMPTS=$(( $IRONIC_CLEANING_TIMEOUT / $IRONIC_CLEANING_DELAY ))

1
zuul.d/ironic-jobs.yaml
View File

@ -39,7 +39,6 @@
IRONIC_BAREMETAL_BASIC_OPS: True IRONIC_BAREMETAL_BASIC_OPS: True
IRONIC_BUILD_DEPLOY_RAMDISK: False IRONIC_BUILD_DEPLOY_RAMDISK: False
IRONIC_CALLBACK_TIMEOUT: 1800 IRONIC_CALLBACK_TIMEOUT: 1800
IRONIC_PXE_BOOT_RETRY_TIMEOUT: 900
IRONIC_DEPLOY_DRIVER: ipmi IRONIC_DEPLOY_DRIVER: ipmi
IRONIC_INSPECTOR_BUILD_RAMDISK: False IRONIC_INSPECTOR_BUILD_RAMDISK: False
IRONIC_INSPECTOR_TEMPEST_INTROSPECTION_TIMEOUT: 1200 IRONIC_INSPECTOR_TEMPEST_INTROSPECTION_TIMEOUT: 1200