openstack-attic/object-api
Code Issues Proposed changes

Clarify UTF-8 and URL-encoding of names

Browse Source 
Fixes two issues:

o Where UTF-8 and URL-encoding are mentioned, often the URL-encoding is
  mentioned first. This might imply that URL-encoding is key -- it's not.
  The clasic mistake is to simply URL-encode a string. This can result in
  illegal UTF-8. Example: ÿ (y-umlaut) is URL-encoded as %FF. Whereas,
  the correct UTF-8, then URL-encoding is %C3%BF.

o Where container and object names appear in headers (such as
  X-Versions-Location), the UTF-8 string must be URL-encoded. This fix
  clarifies the header descriptions of such headers.

Change-Id: I82dea0e789aab6eb9bf27a1e70b01de427d50b80
This commit is contained in:
Donagh McCabe 2013-09-27 13:56:16 +01:00
parent dd9cbfbb05
commit 5d5d48db44
5 changed files with 51 additions and 51 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
openstack-object-storage-dev/src/docbkx/object-api-general.xml
View File

@ -152,12 +152,11 @@ format="SVG" scale="60"/>
<para>Maximum length of object name: 1024 bytes</para>
</listitem>
</itemizedlist>
<para>Container and object names should be properly
URL-encoded prior to interacting with the ReST interface
(the language APIs handle URL encoding/decoding) and the
container and object names must be UTF-8 encoded. The
length restrictions should be checked against the
URL-encoded string.</para>
<para>Container and object names must be UTF-8 encoded and then should be properly
URL-encoded prior to interacting with the ReST interface. You may be using an API
binding that performs the URL-encoding on your behalf. If so, do not URL-encode before
calling the API binding otherwise you will double-encode container and object names. The
length restrictions should be checked against the URL-encoded string.</para>
<para>Each ReST request against the OpenStack Object Storage
system requires the inclusion of a specific
<firstterm>authorization token</firstterm> HTTP header

20
openstack-object-storage-dev/src/docbkx/object-api-storage-container-svc.xml
View File

@ -367,17 +367,15 @@ photos/animals/dogs/terrier.jpg</literallayout>
<title>Create Container HTTP Response</title>
<literallayout class="monospaced"><xi:include href="samples/container-create-resp.txt" parse="text"/></literallayout>
</example>
<para>Using custom container metadata, you can create
information in the header to effectively "tag" a container
with metadata. The container metadata restrictions are the
same as object metadata: you can have 4096 bytes maximum
overall metadata, 90 distinct metadata items at the most.
Each may have a 128 character name length with a 256 max
value length each. Any valid UTF-8 http header value is
allowed for metadata, however we recommend that you
URL-encode any non-ASCII values using a "%" symbol,
followed by the two-digit hexadecimal representation of
the ISO-Latin code for the character.</para>
<para>Using custom container metadata, you can create information in the header to
effectively tag a container with metadata. The container metadata restrictions are the
same as object metadata: you can have 4096 bytes maximum overall metadata, 90 distinct
metadata items at the most. Each may have a 128 character name length with a maximum of
256 bytes in the value. Any valid UTF-8 encoded string value is allowed for metadata. In
addition for custom metadata, we <emphasis role="italic">recommend</emphasis> that you
URL-encode the header value. However, for non-custom metadata (such as
X-Versions-Location), you <emphasis role="italic">must</emphasis> UTF-8 encode and then
URL-encode values that contain a container or object name.</para>
<example>
<title>Container Create Container with Metadata HTTP
Request</title>

58
openstack-object-storage-dev/src/docbkx/object-api-storage-object-svc.xml
View File

@ -404,17 +404,14 @@ format="SVG" scale="60"/>
manifest object streams all the segments concatenated.
There is no limit to the number of segments that can
be a part of a single large object.</para>
<para>To ensure the download works correctly, you must
upload all the object segments to the same container,
ensure each object name has a common prefix where
their names sort in the order they should be
concatenated. You also create and upload a manifest
file. The manifest file is simply a zero-byte file
with the extra X-Object-Manifest:
&lt;container&gt;/&lt;prefix&gt; header, where
&lt;container&gt; is the container the object segments
are in and &lt;prefix&gt; is the common prefix for all
the segments. </para>
<para>To ensure the download works correctly, you must upload all the object segments to
the same container and ensure that each object name is prefixed in such a way that
it sorts in the order in which it should be concatenated. You also create and upload
a manifest file. The manifest file is a zero-byte file with the extra
X-Object-Manifest: &lt;container&gt;/&lt;prefix&gt; header, where &lt;container&gt;
is the container the object segments are in and &lt;prefix&gt; is the common prefix
for all the segments. The container and common prefix must be UTF-8 encoded and
URL-encoded in the X-Object-Manifest header.</para>
<para>It is best to upload all the segments first and then
create or update the manifest. With this method, the
full object will not be available for downloading
@ -723,19 +720,16 @@ format="SVG" scale="60"/>
"non-current version" container. Each of the non-current
versions of an object has a time stamp appended to it, so
you know when it was created. </para>
<para>To enable Object Versioning, your cloud provider has to
set <code>allow_versions</code> to <code>TRUE</code> in
their container config. Then, create a container where
your non-current versions will be written. Next, set the
metadata <code>X-Versions-Location</code> header on the
container that holds the current versions of your objects.
Set the metadata header to point to the new non-current
version container you created. This is where your
non-current versions will be stored. Once this is done,
each object in your current-version container will have
Object Versioning enabled; changes to the objects
automatically create non-current versions in the separate
container. </para>
<para>To enable Object Versioning, your cloud provider has to set
<code>allow_versions</code> to <code>TRUE</code> in their container config. Then,
create a container where your non-current versions will be written. Next, set the
metadata <code>X-Versions-Location</code> header on the container that holds the current
versions of your objects. Set the metadata header to point to the new non-current
version container you created. The name of the container must be UTF-8 encoded and then
URL-encoded before putting into the <code>X-Versions-Location</code> header. This is
where your non-current versions will be stored. Once this is done, each object in your
current-version container will have Object Versioning enabled; changes to the objects
automatically create non-current versions in the separate container. </para>
<para>Nothing is written to the non-current version container
when you initially &PUT; an object into the
current-version container. Only when you make edits to the
@ -835,15 +829,13 @@ format="SVG" scale="60"/>
you can save the step of re-uploading the content and thus
also save the associated bandwidth charges, if any were to
apply.</para>
<para>There are two ways to copy an existing object to another
object in OpenStack Object Storage. One way is to do a
&PUT; to the new object (the target) location, but add the
<code>“X-Copy-From”</code> header to designate the
source of the data. The header value should be the
container and object name of the source object in the form
of “/container/object”. Also, the <code>X-Copy-From</code>
&PUT; requests require a Content-Length header, even if it
is zero (0).</para>
<para>There are two ways to copy an existing object to another object in OpenStack Object
Storage. One way is to do a &PUT; to the new object (the target) location, but add the
<code>“X-Copy-From”</code> header to designate the source of the data. The header
value should be the container and object name of the source object in the form of
“/container/object”. The container and object name must be UTF-8 encoded and then
URL-encoded. Also, the <code>X-Copy-From</code> &PUT; requests require a Content-Length
header, even if it is zero (0).</para>
<example>
<title>Object Copy Method 1</title>
<literallayout class="monospaced"><xi:include href="samples/object-copy-1-req.txt" parse="text"/></literallayout>

3
openstack-object-storage-dev/src/docbkx/object-api-storage-services.xml
View File

@ -49,7 +49,8 @@ format="SVG" scale="60"/>
character restrictions.</para>
</listitem>
<listitem>
<para>Object and container names must be URL-encoded and UTF-8-encoded.</para>
<para>Object and container names must be UTF-8 encoded and then URL-encoded to interact with
the ReST API.</para>
</listitem>
</itemizedlist>
<para><!--<info>Added paragraph to forecast info structure -\- dsh -2012-03-09 </info>-->The

10
openstack-object-storage-dev/src/docbkx/os-objectstorage-devguide.xml
View File

@ -66,6 +66,16 @@
Application Programming Interface (API) v1. </para>
</abstract>
<revhistory>
<revision>
<date>2013-09-28</date>
<revdescription>
<itemizedlist spacing="compact">
<listitem>
<para>Clarify UTF-8 and URL-encoding of container and object names in headers. </para>
</listitem>
</itemizedlist>
</revdescription>
</revision>
<revision>
<date>2013-06-10</date>
<revdescription>