1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
|
<html>
<!-- © Copyright Beman Dawes 2012 -->
<!-- Distributed under the Boost Software License, Version 1.0. -->
<!-- See http://www.boost.org/LICENSE_1_0.txt -->
<head>
<meta name="GENERATOR" content="Microsoft FrontPage 5.0">
<meta name="ProgId" content="FrontPage.Editor.Document">
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<title>Filesystem Proposal</title>
</head>
<body>
$id frontmatter=
<table border="0">
<tr>
<td>Document number: </td>
<td><span style="background-color: #FFFF00">D????=12-????</span></td>
</tr>
<tr>
<td>Date:</td>
<td>
<!--webbot bot="Timestamp" S-Type="EDITED" S-Format="%Y-%m-%d" startspan -->2012-06-02<!--webbot bot="Timestamp" endspan i-checksum="12362" --></td>
</tr>
<tr>
<td>Project:</td>
<td>Programming Language C++</td>
</tr>
<tr>
<td valign="top">Reply-to:</td>
<td>Beman Dawes <bdawes at acm dot org></td>
</tr>
</table>
<h1><span style="background-color: #FFFF00">Draft</span><br>
Filesystem Library Proposal (Revision 3)</h1>
<p>This paper proposes a filesystem library component suitable for a <i>C++
Standard Library Technical Specification</i> or for the <i>C++ Standard Library</i>.
The proposed library is based on Version 3 of the Boost Filesystem
Library (see <a href="http://www.boost.org/libs/filesystem">
www.boost.org/libs/filesystem</a>). Preliminary wording is provided. A
<a href="#TODO">TODO</a> list identifies remaining work to be done.</p>
<p><span style="font-style: italic; background-color: #FFFF00">Material in this
document high-lighted in yellow is known to be incomplete, incorrect, or
otherwise require further refinement.</span></p>
<h2>Revision history</h2>
<p><span style="background-color: #FFFF00">D????=12-???? </span>, Filesystem
Library Proposal (Revision 3). Changes include:</p>
<ul>
<li>Add path::operator+= and concat functions to tack on things like suffixes
or numbers. Suggested by Ed Smith-Rowland and others.</li>
<li>Add section tags and section numbers, per LWG
discussion in Kona.</li>
<li>Bring use of trademarked names into ISO compliance, and minimize use of
such names by moving into a single examples section. Thanks to Stefanus Du
Toit for advice on how to do this.</li>
<li>Introduce a definition for "operating system dependent", and then use it
to replace most "implementation defined behavior".</li>
<li>Replace uses of const string return types with non-const string
return types, per LWG discussion in Kona.</li>
<li>Remove permission for implementations to return const string& in certain
cases, per LWG discussion in Kona.</li>
<li>Remove path inserter and extractor dependency on Boost quoted manip (Issue
7).</li>
</ul>
<p><a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3365.html">N3365=12-0055</a>, Filesystem Library Proposal (Revision
2). Changes
include:</p>
<ul>
<li>Namespaces changed to <code>std::tbd::filesystem</code> per
<a href="#Issues-List">issue
1</a> discussion in Kona.</li>
<li>Removed references to Boost within the proposed wording.</li>
<li>Minor fixes to the proposed wording.</li>
<li>Added Issue 7: How should class path inserter and extractor be specified?</li>
<li>Marked up all issues to reflect Kona LWG discussions.</li>
</ul>
<p><a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3335.html">
N3335=12-0025</a>, Filesystem Library for C++11/TR2 (Revision 1). Changes
include:</p>
<ul>
<li>Regenerated the proposed wording from the Boost Filesystem library
reference documentation, using an automated process. This process reduces
the likelihood of inadvertent discrepancies between descriptions.</li>
<li>An <a href="#Issues-List">Issues list</a> was added, seeded with issues
raised by the LWG review of N3239 at the Bloomington meeting, and private
communications from LWG members.</li>
<li>Namespace changed to <code>files</code> as an experiment. Made this
issue number 1 so the LWG can pass judgment.</li>
<li>New functions were added, suggested by David Svoboda, to generate
canonical paths and manage permissions.</li>
<li>More C++11 functionality was applied. This process is still incomplete,
however.</li>
<li>Added proposed changes to header <fstream>. The previous paper had
inadvertently supplied the wrong wording.</li>
<li>Continued the general cleanup of wording.</li>
</ul>
<p><a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3239.html">
N3239 = 11-0009</a>, Filesystem Library Update for TR2 (Preliminary), reflected
changes made to the Boost library version 3 since the previously accepted
committee paper:</p>
<ul>
<li>A single class <code>path</code> handles all aspects of
internationalization, replacing the previous template and its <code>path</code>
and <code>wpath</code> instantiations. Character types <code>char</code>,
<code>wchar_t</code>, <code>char16_t</code>, and <code>char32_t</code> are
supported. This is a major simplification of the path abstraction,
particularly for functions that take path arguments. This change was based
on a suggestion by Peter Dimov.</li>
<li>Several operational functions have been added, including much better
symlink support, the ability to resize a file, and the ability to generate a
unique path.</li>
<li>Support for error reporting via <code>error_code</code> is now uniform
throughout the operations functions.</li>
<li>Several functions have been renamed, based on feedback from users.</li>
</ul>
<p><a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n1975.html">
N1975 = 06-0045</a>, Filesystem Library Proposal for TR2 (Revision 3), was
adopted by the committee in April, 2006, at the Berlin meeting. Shortly
afterward the Library Working Group set aside work on TR2 to concentrate on
C++0x.</p>
<h2>Motivation and Scope</h2>
<p>The motivation and scope for a filesystem library were described in <a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n1975.html">
N1975</a>, and are not repeated here. A minor scope reduction is that an
addition to the current C++ runtime library is no longer needed.</p>
<p>Boost Filesystem Version 3 introduced a single path type that interoperates well with both <code>
basic_string</code> and user defined string types. Thus the following Design
alternatives paragraph is no long applicable:</p>
<blockquote>
<p><strike><i>Single path type which can at runtime accept narrow or wide character
pathnames.</i> Although certainly interesting, and possibly superior, such a
design would not interoperate well with the current Standard Library's
compile-time typed <code>basic_string</code>. A new runtime polymorphic string
class would be the best place to experiment with this concept, not a path class.</strike></p>
</blockquote>
<h2><a name="TODO">TODO</a></h2>
<ul>
<li>Apply more C++0X features. Boost.Filesystem needs to implement these to verify their
application is correct.</li>
<li>Ion Todirel suggests "Would be nice if path::append was variadic, to
improve usability".</li>
<li>Boost.Filesystem needs to implement <code>char16_t</code> and <code>char32_t</code> support to verify the
specification for these is correct.</li>
<li>The Boost implementation has more class path non-member relational
functions that shown in the docs, and the specific set of relational
functions varies between Windows and POSIX. Figure out what's happening and
document it.</li>
<li><code><a href="#Source">Source</a></code> is not specified as actually
implemented. Expose <code>path_traits</code>?</li>
<li><i>Effects</i> for <code>copy</code> and <code>copy_directory</code>
need to be reviewed, revised, tested, peer reviewed.</li>
<li>Dinkumware/Microsoft report slightly different results for Decomposition
table. Rerun table. Check discrepancies.</li>
<li>Apply issue resolutions from Kona. Complete except some remaining
codecvt arguments</li>
<li>Review Nick Stoughton's email for suggestions, action items.</li>
<li>Change <code>time_t</code> to <code>chrono system_clock::time_point</code>,
per LWG discussion in Kona. </li>
<li>Ed Smith-Rowland:<br>
> I found it less confusing to switch to positive logic for
recursive_directory_iterator:<br>
> bool no_push_pending() const noexcept;<br>
> to say<br>
> bool is_push_pending() const noexcept;<br>
><br>
> and<br>
> void no_push(bool value=true);<br>
> to say<br>
> void push_pending(bool value=false);<br>
> or<br>
> void set_push_pending(bool value=false);<br>
> I found it less confusing and I think other parts of the library strive
for this.</li>
<li>recursive_directory_iterator is specified in terms of private data
members. Remove these (including the comment!) and specify in terms of
behavior only.</li>
<li>Ion Todirel: Would be nice to have a make_relative. Need to review
similar suggestions.</li>
</ul>
$endid
$id wording_prefix=
<h1>Proposed Wording</h1>
<!-- generate-section-numbers=true -->
<p><span style="font-style: italic; background-color: rgb(224, 224, 224);">
Gray-shaded italic text is commentary on the proposal. It is not to be added to
the working paper.</span></p>
<h1>Filesystem Library [filesystem]</h1>
<p>This clause describes components that perform operations on file systems and
their components, such as paths, regular files, and directories.</p>
<p>ISO/IEC 9945 contains provisions which, through reference in this
text, constitute provisions of this clause. At the time of
publication, the editions indicated were valid. All standards are subject to
revision, and parties to agreements based on this clause are
encouraged to investigate the possibility of applying the most recent editions
of the standard indicated below. Members of IEC and ISO maintain registers of
currently valid International Standards.</p>
<ul>
<li>ISO/IEC 9945:2003, <i>Portable Operating System Interface (POSIX</i>®<i>),
part 1 (Base Definitions) and part 2 (System Interfaces)</i>, both as
corrected by their respective 2004 Correction 1 documents.<p><i>
<span style="background-color: #E0E0E0">ISO/</span><span style="background-color: #E0E0E0">IEC</span><span style="background-color: #E0E0E0"> 9945:2003 is also IEEE Std 1003.1-2001, and The Open Group Base
Specifications, Issue 6, and is also known as The Single Unix®</span></i><font face="Times New Roman"><i><b><span style="background-color: #E0E0E0">
</span>
</b></i><span style="font-style: italic; background-color: #E0E0E0">Specification, Version 3. It is available from each of those
organizations, and may be read online or downloaded from
<a href="http://www.unix.org/single_unix_specification/">
www.unix.org/single_unix_specification/</a></span></font></li>
</ul>
$endid
$id wording_suffix=
<p><span style="font-style: italic; background-color: #E0E0E0">End of new
Clause.</span></p>
<hr>
<h2><a name="Issues-List">Issues List</a></h2>
<hr>
<h3>Issue 1: What is the appropriate namespace? Status:
Tentatively resolved</h3>
<h4>Discussion</h4>
<p>The N3335 proposal places the library in namespace <code>std::tr2::files</code>.
Rationale for a sub-namespace is that the library uses several names that don't
seem appropriate for namespace <code>tr2</code> since full standardization would
then put the names into <code>std</code>. The function names <code>remove</code>
and <code>rename</code> are of particular concern because these functions differ
in behavior from current standard library functions with those names. It also
doesn't seem desirable to preempt names like <code>equivalent</code> and <code>
status</code>.</p>
<h4>Resolution</h4>
<p><i>Kona: Strong support for <code>filesystem</code> as the library's
namespace. Strong support for a technical specification namespace that alerts uses that
contents are likely to change if and when they later get moved into the
standard. </i></p>
<p><i>No decision yet on a TR namespace; <code>tbs</code> being used as
a placeholder. Thus the full namespace is changed to <code>
std::tbs::filesystem.</code></i></p>
<hr>
<h3>Issue 2: Excessive use of <code>const codecvt_type&</code> arguments
Status: Open</h3>
<h4>Discussion</h4>
<p>Users sometimes need to do path conversions that use something other than the
imbued codecvt facet. The need is particularly acute in multi-threaded
applications where changing the imbued facet would introduce a data race. That
said, providing an optional <code>const codecvt_type&</code> argument for every
function where the need might possibly arise is excessive because its use is so
rare and it adds considerable interface clutter.</p>
<h4>Proposed resolution</h4>
<p dir="ltr"><i>Kona:</i></p>
<p dir="ltr"><i>Remove all existing class path <code>const codecvt_type&</code>
arguments. </i></p>
<p dir="ltr"><i>Beman to pursue separate encoding conversion functionality, per
Thursday N3336 "Adapting standard library strings and IO to a Unicode World"
discussion. See Kona wiki.</i></p>
<p dir="ltr"><i>If a separate encoding conversion facility isn't possible, then
some form of minimal class path encoding conversion facility will be needed.
But..., the LWG would like to see use cases and clearer explanation of
motivation before deciding on specifics, and be very sure that there is no way
to achieve with existing std library functionality, albeit with some loss of
efficiency.</i></p>
<hr>
<h3>Issue 3: Possible "implicit cast to native type"?
Status: Open</h3>
<h4>Discussion</h4>
<p>In Bloomington there was discussion of "implicit cast to implicit cast to
native OS type to inter operate with existing iostream library and native
functions instead of modifying fstream".</p>
<h4>Proposed resolution</h4>
<p><i>Kona: The Dinkumware/Microsoft implementation has added an implicit
conversion to the string type, eliminating any need to modify fstream.
(Discussion: might be safer to make the implicit conversion protected, then make
fstream classes friends).</i></p>
<p><i>Action: Beman to investigate, test, as avoiding modification of existing
header and classes is much desired.</i></p>
<hr>
<h3>Issue 4: Given move semantics, it is best not to return const strings.
Status: Resolved</h3>
<h4>Discussion</h4>
<p>The issue title pretty much says it all.</p>
<h4>Proposed resolution</h4>
<p><i>Kona: Return plain strings. Remove the permission for implementations to
return const refs, since uses are not in performace critical code and subtle
portability bugs may occur.</i></p>
<p><i>Action: Beman to apply to proposed wording.</i></p>
<p><i>Version 3: Resolution applied.</i></p>
<hr>
<h3>Issue 5: Is there a way to handle characters that are illegal for particular
OS? Status: NAD</h3>
<h4>Discussion</h4>
<p>Question raised by Pablo in Bloomington.</p>
<h4>Proposed resolution</h4>
<p>Beman suggests NAD, Future. I've done some work on this, including looking at
systems like OpenVMS that have an escape mechanism to handle otherwise
unrepresentable characters. There was a comment to that effect in N3239. I
believe it should be deferred to some future release since (1) it is complex
enough that I'd like to see actual implementation and use experience (presumably
via Boost), and (2) I can't recall a user ever requesting such a feature.</p>
<p><i>Kona: No consensus for change, since there doesn't appear to be any
compelling use case.</i></p>
<hr>
<h3>Issue 6: Could allocator support be added to class path?
Status: Open</h3>
<h4>Discussion</h4>
<p>Question raised by a committee member in private email.</p>
<p><i>Kona: Much discussion.</i></p>
<p><i>Action: Alisdair to write paper, based on a real implementation, so that
it is possible to assess the impact on interface, performance, etc.</i></p>
<p><i>Post-Kona: Alisdair will defer paper until after TR. </i></p>
<hr>
<h3>Issue 7: How should class path inserter and extractor be specified?
Status: Resolved</h3>
<h4>Discussion</h4>
<p>Class <code>path</code> currently is specified to used the Boost detail <code>
quoted</code> stream manipulator. A path object is inserted delimited by quotes,
and extracted accordingly, ensuring that paths containing spaces are
round-tripped correctly. Quotes in the path itself are escaped, and io
state is saved and restored.</p>
<p>See
<a href="http://www.boost.org/doc/libs/1_48_0/libs/io/doc/quoted_manip.html">
http://www.boost.org/doc/libs/1_48_0/libs/io/doc/quoted_manip.html</a></p>
<p>Since the standard can't specify behavior in terms of something in Boost, we
have to change the specification. Since the <code>quoted</code> stream
manipulator is a handy little component, the issue is raised as to whether it
should go in TR2.</p>
<h4>Resolution</h4>
<p><i>Kona: Specify the path inserter and extractor to have the desired
behavior without reference to how it is achieved. (If someone wants to propose a
quoted manipulator, that's a separate proposal for a different TR.)</i></p>
<p><i>Action: Beman to apply to proposed wording.</i></p>
<p><i>Version 3: Resolution applied.</i></p>
<hr>
<h3>Issue 8: Rename <code>rename</code>
Status: New</h3>
<h4>Discussion</h4>
<p>There are minor problems with the name of the <code>rename()</code> function:</p>
<ul>
<li>There is already a "rename" function, albeit in namespace std, and it has
different semantics as regards error handling. Thus giving the function a
different name might reduce the chance of user error.</li>
<li>The name "move" would better reflect the actual semantics,
particularly for moves between directories., and this is the name used by some
API's, console commands, and GUI file managers for that functionality.</li>
</ul>
<h4>Proposed resolution</h4>
<p>Change the name of the <code>rename()</code> function to <code>move()</code>.</p>
<hr>
<p>$endid
$id backmatter=
$endid </p>
</body>
</html>
|
