ce1fd47287
Situation:
The function get_part_nodes is being called by Openstack-Swift's proxy/controllers/base.py:
https://github.com/openstack/swift/blob/1.7.4/swift/proxy/controllers/base.py#L410
https://github.com/openstack/swift/blob/1.7.6/swift/proxy/controllers/base.py#L447
As this was not implemented in the current GlusterFS version of ring.py, it was calling swift's
original get_part_nodes, which would often return the incorrect node, resulting in the incorrect
drive being associated with a request.
There is another function that the original ring.py implements -- get_other_nodes, which has to
do with replication. Since GlusterFS is handling replication, this function should never be
called. However, in the interest of completeness, that function is also being replaced.
Code changes:
The two functions, get_part_nodes, and get_other_nodes have been implemented to override the
default functions, and get_nodes has been updated to store information in self vars, about the
account being operated on, as the two new functions are not called with that info, and get_nodes
appears to always be called first.
The code has be refactored to all call _get_part_nodes, much like swift has refactored their code.
Reason for implementation this way:
I didn't see a better way to do it, but am open to suggestions.
Test cases:
A mock ring is created with two different devices, test and iops
test_first_device: Ensure that the first device, test, is returned for both get_nodes, and
get_part_node, and get_more_nodes returns volume_not_in_ring.
test_invalid_device: Ensure that a request for a non-existant device returns volume_not_in_ring.
test_second_device: Same as test_first_device, but for the second device, iops instead of test.
test_second_device_with_reseller_prefix: Test that calling with the reseller prefix, AUTH_ will
still return the correct data.
Change-Id: I2f3d526934a47b01e9c065d0edf0fbf06f300369
BUG: 924792
Signed-off-by: Alex Wheeler <wheelear@gmail.com>
Reviewed-on: http://review.gluster.org/4748
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
112 lines
4.3 KiB
Python
112 lines
4.3 KiB
Python
# Copyright (c) 2013 Red Hat, Inc.
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
# implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
|
|
from ConfigParser import ConfigParser
|
|
from swift.common.ring import ring
|
|
from swift.common.utils import search_tree
|
|
from gluster.swift.common.Glusterfs import SWIFT_DIR
|
|
|
|
reseller_prefix = "AUTH_"
|
|
conf_files = search_tree(SWIFT_DIR, "proxy-server*", 'conf')
|
|
if conf_files:
|
|
conf_file = conf_files[0]
|
|
|
|
_conf = ConfigParser()
|
|
if conf_files and _conf.read(conf_file):
|
|
if _conf.defaults().get("reseller_prefix", None):
|
|
reseller_prefix = _conf.defaults().get("reseller_prefix")
|
|
else:
|
|
for key, value in _conf._sections.items():
|
|
if value.get("reseller_prefix", None):
|
|
reseller_prefix = value["reseller_prefix"]
|
|
break
|
|
|
|
if not reseller_prefix.endswith('_'):
|
|
reseller_prefix = reseller_prefix + '_'
|
|
|
|
class Ring(ring.Ring):
|
|
def _get_part_nodes(self, part):
|
|
seen_ids = set()
|
|
nodes = [dev for dev in self._devs \
|
|
if dev['device'] == self.acc_name \
|
|
and not (dev['id'] in seen_ids \
|
|
or seen_ids.add(dev['id']))]
|
|
if not nodes:
|
|
nodes = [self.false_node]
|
|
return nodes
|
|
|
|
def get_part_nodes(self, part):
|
|
"""
|
|
Get the nodes that are responsible for the partition. If one
|
|
node is responsible for more than one replica of the same
|
|
partition, it will only appear in the output once.
|
|
|
|
:param part: partition to get nodes for
|
|
:returns: list of node dicts
|
|
|
|
See :func:`get_nodes` for a description of the node dicts.
|
|
"""
|
|
return self._get_part_nodes(part)
|
|
|
|
def get_nodes(self, account, container=None, obj=None):
|
|
"""
|
|
Get the partition and nodes for an account/container/object.
|
|
If a node is responsible for more than one replica, it will
|
|
only appear in the output once.
|
|
:param account: account name
|
|
:param container: container name
|
|
:param obj: object name
|
|
:returns: a tuple of (partition, list of node dicts)
|
|
|
|
Each node dict will have at least the following keys:
|
|
====== ===============================================================
|
|
id unique integer identifier amongst devices
|
|
weight a float of the relative weight of this device as compared to
|
|
others; this indicates how many partitions the builder will try
|
|
to assign to this device
|
|
zone integer indicating which zone the device is in; a given
|
|
partition will not be assigned to multiple devices within the
|
|
same zone
|
|
ip the ip address of the device
|
|
port the tcp port of the device
|
|
device the device's name on disk (sdb1, for example)
|
|
meta general use 'extra' field; for example: the online date, the
|
|
hardware description
|
|
====== ===============================================================
|
|
"""
|
|
self.false_node = {'zone': 1, 'weight': 100.0, 'ip': '127.0.0.1', 'id': 0, \
|
|
'meta': '', 'device': 'volume_not_in_ring', \
|
|
'port': 6012}
|
|
if account.startswith(reseller_prefix):
|
|
self.acc_name = account.replace(reseller_prefix, '', 1)
|
|
else:
|
|
self.acc_name = account
|
|
|
|
part = 0
|
|
return part, self._get_part_nodes(part)
|
|
|
|
|
|
def get_more_nodes(self, part):
|
|
"""
|
|
Generator to get extra nodes for a partition for hinted handoff.
|
|
|
|
:param part: partition to get handoff nodes for
|
|
:returns: generator of node dicts
|
|
|
|
See :func:`get_nodes` for a description of the node dicts.
|
|
Should never be called in the swift UFO environment, so yield nothing
|
|
"""
|
|
yield self.false_node
|