75b881bd31
Prior to this fix, we have been unable to run the Metal3 CI job with SQLAlchemy's internal autocommit setting enabled. However that setting is deprecated and needs to be removed. Investigating our DB queries and request patterns, we were able to identify some queries which generally resulted in the underlying task and lock being held longer because the output was not actually returned, which is something we've generally had to fix in some places previously. Doing some of these changes did drastically reduce the number of errors encountered with the Metal3 CI job, however it did not eliminate them entirely. Further investigation, we were able to determine that the underlying issue we were encountering was when we had an external semi-random reader, such as Metal3 polling endpoints, we could reach a situation where we would be blocked from updating the database as to open a write lock, we need the active readers not to be interacting with the database, and with a random reader of sorts, the only realistic option we have is to enable the Write Ahead Log[0]. We didn't have to do this with SQLAlchemy previously because autocommit behavior hid the complexities from us, but in order to move to SQLAlchemy 2.0, we do need to remove autocommit. Additionally, adds two unit tests for get_node_with_token rpc method, which apparently we missed or lost somewhere along the way. Also, adds notes to two Database interactions to suggest we look at them in the future as they may not be the most efficient path forward. [0]: https://www.sqlite.org/wal.html Change-Id: Iebcc15fe202910b942b58fc004d077740ec61912
38 lines
1.6 KiB
Python
38 lines
1.6 KiB
Python
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
|
# not use this file except in compliance with the License. You may obtain
|
|
# a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
|
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
|
# License for the specific language governing permissions and limitations
|
|
# under the License.
|
|
|
|
from oslo_config import cfg
|
|
from oslo_db.sqlalchemy import enginefacade
|
|
from sqlalchemy.engine import Engine
|
|
from sqlalchemy import event
|
|
|
|
CONF = cfg.CONF
|
|
|
|
# FIXME(stephenfin): we need to remove reliance on autocommit semantics ASAP
|
|
# since it's not compatible with SQLAlchemy 2.0
|
|
# NOTE(dtantsur): we want sqlite as close to a real database as possible.
|
|
enginefacade.configure(sqlite_fk=True, __autocommit=True)
|
|
|
|
|
|
# NOTE(TheJulia): Setup a listener to trigger the sqlite write-ahead
|
|
# log to be utilized to permit concurrent access, which is needed
|
|
# as we can get read requests while we are writing via the API
|
|
# surface *when* we're using sqlite as the database backend.
|
|
@event.listens_for(Engine, "connect")
|
|
def _setup_journal_mode(dbapi_connection, connection_record):
|
|
# NOTE(TheJulia): The string may not be loaded in some unit
|
|
# tests so handle whatever the output is as a string so we
|
|
# can lower/compare it and send the appropriate command to
|
|
# the database.
|
|
if 'sqlite' in str(CONF.database.connection).lower():
|
|
dbapi_connection.execute("PRAGMA journal_mode=WAL")
|