blob: d6f981954fea1eaf994df2b47fe5828f2916c8c6 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
|
============
Introduction
============
Where to Start?
~~~~~~~~~~~~~~~
The ``python-swiftclient`` project comprises a command line tool and two
separate APIs for accessing swift programmatically. Choosing the most
appropriate method for a given use case is the first problem a user needs to
solve.
Use Cases
---------
Alongside the command line tool, the ``python-swiftclient`` includes two
levels of API:
* A low level client API that provides simple Python wrappers around the
various authentication mechanisms and the individual HTTP requests.
* A high level service API that provides methods for performing common
operations in parallel on a thread pool.
Example use cases:
* Uploading and retrieving data
Use the command line tool if you are simply uploading and downloading
files and directories to and from your filesystem. The command line tool
can be integrated into a shell script to automate tasks.
* Integrating into an automated Python workflow
Use the ``SwiftService`` API to perform operations offered by the CLI
if your use case requires integration with a Python-based workflow.
This method offers greater control and flexibility over individual object
operations, such as the metadata set on each object. The ``SwiftService``
class provides methods to perform multiple sets of operations against a
swift object store using a configurable shared thread pool. A single
instance of the ``SwiftService`` class can be shared between multiple
threads in your own code.
* Developing an application in Python to access a swift object store
Use the ``SwiftService`` API to develop Python applications that use
swift to store and retrieve objects. A ``SwiftService`` instance provides
a configurable thread pool for performing all operations supported by the
CLI.
* Fine-grained control over threading or the requests being performed
Use the ``Connection`` API if your use case requires fine grained control
over advanced features or you wish to use your own existing threading
model. Examples of advanced features requiring the use of the
``Connection`` API include creating an SLO manifest that references
already existing objects, or fine grained control over the query strings
supplied with each HTTP request.
Important considerations
~~~~~~~~~~~~~~~~~~~~~~~~
This section covers some important considerations, helpful hints, and things to
avoid when integrating an object store into your workflow.
An object store is not a filesystem
-----------------------------------
It cannot be stressed enough that your usage of the object store should reflect
the proper use case, and not treat the storage like a traditional filesystem.
There are two main restrictions to bear in mind when designing an application
that uses an object store:
* You cannot rename objects. Due to fact that the name of an object is one
of the factors that determines where the object and its replicas are stored,
renaming would require multiple copies of the data to be moved between
physical storage devices. If you want to rename an object you must upload
to the new location, or make a server side copy request to the new location,
and then delete the original.
* You cannot modify objects. Objects are stored in multiple locations and
are checked for integrity based on the MD5 sum calculated during
upload. In order to modify the contents of an object, the entire desired
contents must be re-uploaded. In certain special cases it is possible to
work around this restriction using large objects, but no general
file-like access is available to modify a stored object.
Objects cannot be locked
------------------------
There is no mechanism to perform a combination of reading the
data/metadata from an object and writing an update to that data/metadata in an
atomic way. Any user with access to a container could update the contents or
metadata associated with an object at any time.
Workflows that assume that no updates have been made since the last read of an
object should be discouraged. Enabling a workflow of this type requires an
external object locking mechanism and/or cooperation between all clients
accessing the data.
|
