From 5d5d48db44cb49d7fd0d6544aee2cbab97114b29 Mon Sep 17 00:00:00 2001 From: Donagh McCabe Date: Fri, 27 Sep 2013 13:56:16 +0100 Subject: [PATCH] Clarify UTF-8 and URL-encoding of names MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes two issues: o Where UTF-8 and URL-encoding are mentioned, often the URL-encoding is mentioned first. This might imply that URL-encoding is key -- it's not. The clasic mistake is to simply URL-encode a string. This can result in illegal UTF-8. Example: ÿ (y-umlaut) is URL-encoded as %FF. Whereas, the correct UTF-8, then URL-encoding is %C3%BF. o Where container and object names appear in headers (such as X-Versions-Location), the UTF-8 string must be URL-encoded. This fix clarifies the header descriptions of such headers. Change-Id: I82dea0e789aab6eb9bf27a1e70b01de427d50b80 --- .../src/docbkx/object-api-general.xml | 11 ++-- .../object-api-storage-container-svc.xml | 20 +++---- .../docbkx/object-api-storage-object-svc.xml | 58 ++++++++----------- .../docbkx/object-api-storage-services.xml | 3 +- .../src/docbkx/os-objectstorage-devguide.xml | 10 ++++ 5 files changed, 51 insertions(+), 51 deletions(-) diff --git a/openstack-object-storage-dev/src/docbkx/object-api-general.xml b/openstack-object-storage-dev/src/docbkx/object-api-general.xml index 209b9c1..efc3727 100644 --- a/openstack-object-storage-dev/src/docbkx/object-api-general.xml +++ b/openstack-object-storage-dev/src/docbkx/object-api-general.xml @@ -152,12 +152,11 @@ format="SVG" scale="60"/> Maximum length of object name: 1024 bytes - Container and object names should be properly - URL-encoded prior to interacting with the ReST interface - (the language APIs handle URL encoding/decoding) and the - container and object names must be UTF-8 encoded. The - length restrictions should be checked against the - URL-encoded string. + Container and object names must be UTF-8 encoded and then should be properly + URL-encoded prior to interacting with the ReST interface. You may be using an API + binding that performs the URL-encoding on your behalf. If so, do not URL-encode before + calling the API binding otherwise you will double-encode container and object names. The + length restrictions should be checked against the URL-encoded string. Each ReST request against the OpenStack Object Storage system requires the inclusion of a specific authorization token HTTP header diff --git a/openstack-object-storage-dev/src/docbkx/object-api-storage-container-svc.xml b/openstack-object-storage-dev/src/docbkx/object-api-storage-container-svc.xml index c8a6405..7c74f98 100644 --- a/openstack-object-storage-dev/src/docbkx/object-api-storage-container-svc.xml +++ b/openstack-object-storage-dev/src/docbkx/object-api-storage-container-svc.xml @@ -367,17 +367,15 @@ photos/animals/dogs/terrier.jpg Create Container HTTP Response - Using custom container metadata, you can create - information in the header to effectively "tag" a container - with metadata. The container metadata restrictions are the - same as object metadata: you can have 4096 bytes maximum - overall metadata, 90 distinct metadata items at the most. - Each may have a 128 character name length with a 256 max - value length each. Any valid UTF-8 http header value is - allowed for metadata, however we recommend that you - URL-encode any non-ASCII values using a "%" symbol, - followed by the two-digit hexadecimal representation of - the ISO-Latin code for the character. + Using custom container metadata, you can create information in the header to + effectively tag a container with metadata. The container metadata restrictions are the + same as object metadata: you can have 4096 bytes maximum overall metadata, 90 distinct + metadata items at the most. Each may have a 128 character name length with a maximum of + 256 bytes in the value. Any valid UTF-8 encoded string value is allowed for metadata. In + addition for custom metadata, we recommend that you + URL-encode the header value. However, for non-custom metadata (such as + X-Versions-Location), you must UTF-8 encode and then + URL-encode values that contain a container or object name. Container Create Container with Metadata HTTP Request diff --git a/openstack-object-storage-dev/src/docbkx/object-api-storage-object-svc.xml b/openstack-object-storage-dev/src/docbkx/object-api-storage-object-svc.xml index 99f0ad5..68d9b82 100644 --- a/openstack-object-storage-dev/src/docbkx/object-api-storage-object-svc.xml +++ b/openstack-object-storage-dev/src/docbkx/object-api-storage-object-svc.xml @@ -404,17 +404,14 @@ format="SVG" scale="60"/> manifest object streams all the segments concatenated. There is no limit to the number of segments that can be a part of a single large object. - To ensure the download works correctly, you must - upload all the object segments to the same container, - ensure each object name has a common prefix where - their names sort in the order they should be - concatenated. You also create and upload a manifest - file. The manifest file is simply a zero-byte file - with the extra X-Object-Manifest: - <container>/<prefix> header, where - <container> is the container the object segments - are in and <prefix> is the common prefix for all - the segments. + To ensure the download works correctly, you must upload all the object segments to + the same container and ensure that each object name is prefixed in such a way that + it sorts in the order in which it should be concatenated. You also create and upload + a manifest file. The manifest file is a zero-byte file with the extra + X-Object-Manifest: <container>/<prefix> header, where <container> + is the container the object segments are in and <prefix> is the common prefix + for all the segments. The container and common prefix must be UTF-8 encoded and + URL-encoded in the X-Object-Manifest header. It is best to upload all the segments first and then create or update the manifest. With this method, the full object will not be available for downloading @@ -723,19 +720,16 @@ format="SVG" scale="60"/> "non-current version" container. Each of the non-current versions of an object has a time stamp appended to it, so you know when it was created. - To enable Object Versioning, your cloud provider has to - set allow_versions to TRUE in - their container config. Then, create a container where - your non-current versions will be written. Next, set the - metadata X-Versions-Location header on the - container that holds the current versions of your objects. - Set the metadata header to point to the new non-current - version container you created. This is where your - non-current versions will be stored. Once this is done, - each object in your current-version container will have - Object Versioning enabled; changes to the objects - automatically create non-current versions in the separate - container. + To enable Object Versioning, your cloud provider has to set + allow_versions to TRUE in their container config. Then, + create a container where your non-current versions will be written. Next, set the + metadata X-Versions-Location header on the container that holds the current + versions of your objects. Set the metadata header to point to the new non-current + version container you created. The name of the container must be UTF-8 encoded and then + URL-encoded before putting into the X-Versions-Location header. This is + where your non-current versions will be stored. Once this is done, each object in your + current-version container will have Object Versioning enabled; changes to the objects + automatically create non-current versions in the separate container. Nothing is written to the non-current version container when you initially &PUT; an object into the current-version container. Only when you make edits to the @@ -835,15 +829,13 @@ format="SVG" scale="60"/> you can save the step of re-uploading the content and thus also save the associated bandwidth charges, if any were to apply. - There are two ways to copy an existing object to another - object in OpenStack Object Storage. One way is to do a - &PUT; to the new object (the target) location, but add the - “X-Copy-From” header to designate the - source of the data. The header value should be the - container and object name of the source object in the form - of “/container/object”. Also, the X-Copy-From - &PUT; requests require a Content-Length header, even if it - is zero (0). + There are two ways to copy an existing object to another object in OpenStack Object + Storage. One way is to do a &PUT; to the new object (the target) location, but add the + “X-Copy-From” header to designate the source of the data. The header + value should be the container and object name of the source object in the form of + “/container/object”. The container and object name must be UTF-8 encoded and then + URL-encoded. Also, the X-Copy-From &PUT; requests require a Content-Length + header, even if it is zero (0). Object Copy Method 1 diff --git a/openstack-object-storage-dev/src/docbkx/object-api-storage-services.xml b/openstack-object-storage-dev/src/docbkx/object-api-storage-services.xml index dec86f3..2555127 100644 --- a/openstack-object-storage-dev/src/docbkx/object-api-storage-services.xml +++ b/openstack-object-storage-dev/src/docbkx/object-api-storage-services.xml @@ -49,7 +49,8 @@ format="SVG" scale="60"/> character restrictions. - Object and container names must be URL-encoded and UTF-8-encoded. + Object and container names must be UTF-8 encoded and then URL-encoded to interact with + the ReST API. The diff --git a/openstack-object-storage-dev/src/docbkx/os-objectstorage-devguide.xml b/openstack-object-storage-dev/src/docbkx/os-objectstorage-devguide.xml index 7766a17..e5c5244 100644 --- a/openstack-object-storage-dev/src/docbkx/os-objectstorage-devguide.xml +++ b/openstack-object-storage-dev/src/docbkx/os-objectstorage-devguide.xml @@ -66,6 +66,16 @@ Application Programming Interface (API) v1. + + 2013-09-28 + + + + Clarify UTF-8 and URL-encoding of container and object names in headers. + + + + 2013-06-10