diff options
author | Ramon Fernandez <ramon@mongodb.com> | 2016-01-29 14:21:53 -0500 |
---|---|---|
committer | Ramon Fernandez <ramon@mongodb.com> | 2016-01-29 14:22:24 -0500 |
commit | 5d6532f3d5227ff76f62c4810c98a4ef4d0c8c56 (patch) | |
tree | 67f6b9581dce0fe7408639c3386e3694d9a8893b /src/third_party/wiredtiger/src/docs/tools | |
parent | 569041f7df6f62c1b026cc5fefe70c3998092db3 (diff) | |
download | mongo-5d6532f3d5227ff76f62c4810c98a4ef4d0c8c56.tar.gz |
Import wiredtiger-wiredtiger-2.7.0-559-g07966a4.tar.gz from wiredtiger branch mongodb-3.2
ref: 3c2ad56..07966a4
WT-1517 schema format edge cases
WT-1801 Add a directory sync after rollback of a WT_SESSION::rename operation
WT-2060 Simplify aggregation of statistics
WT-2073 metadata cleanups
WT-2099 Seeing memory underflow messages
WT-2113 truncate01 sometimes fails
WT-2142 Connection cleanup in Python tests
WT-2177 Add an optional per-thread seed to random number generator
WT-2198 bulk load and column store appends
WT-2216 simplify row-store search loop slightly
WT-2225 New split code performance impact
WT-2231 pinned page cursor searches could check parent keys
WT-2235 wt printlog option without unicode
WT-2242 WiredTiger treats dead trees the same as other trees in eviction
WT-2244 Trigger in-memory splits sooner
WT-2245 WTPERF Truncate has no ability to catch up when it falls behind
WT-2246 column-store append searches the leaf page; the maximum record number fails CRUD operations
WT-2247 variable-length column-store in-memory page splits
WT-2256 WTPERFs throttle option fires in bursts
WT-2257 wtperf doesn't handle overriding workload config
WT-2258 WiredTiger preloads pages even when direct-IO is configured.
WT-2259 __wt_evict_file_exclusive_on() should clear WT_BTREE_NO_EVICTION on error
WT-2260 Workloads evict internal pages unexpectedly
WT-2262 Random sampling is skewed by tree shape
WT-2265 Wiredtiger related change in ppc64le specific code block in gcc.h
WT-2266 Add wtperf config to set if perf thresholds are fatal
WT-2267 Improve wtperf throttling implementation to provide steady load
WT-2269 wtperf should dump its config everytime it runs
WT-2272 Stress test assertion in the sweep server
WT-2275 broken DB after application crash
WT-2276 tool to decode checkpoint addr
WT-2277 Remove WT check against big-endian systems
WT-2279 Define WT_PAUSE(), WT_FULL_BARRIER(), etc when s390x is defined
WT-2281 wtperf smoke.sh fails on ppc64le
WT-2282 error in wt_txn_update_oldest verbose message test
WT-2283 retry in txn_update_oldest results in a hang
WT-2284 Repeated macro definition
WT-2285 configure should set BUFFER_ALIGNMENT_DEFAULT to 4kb on linux
WT-2287 WT_SESSION.rebalance
WT-2289 failure in fast key check
WT-2290 WT_SESSION.compact could be more effective.
WT-2291 Random cursor walk inefficient in skip list only trees
WT-2295 WT_SESSION.create does a full-scan of the main table
WT-2296 New log algorithm needs improving for sync/flush settings
WT-2297 Fix off-by-one error in Huffman config file parsing
WT-2299 upper-level WiredTiger code is reaching into the block manager
WT-2301 Add reading a range to wtperf
WT-2303 Build warning in wtperf
WT-2304 wtperf crash dumping config
WT-2305 Fix coverity scan issues on 23/12/2015
WT-2307 Internal page splits can corrupt cursor iteration
WT-2308 custom extractor for ref_cursors in join cursor
WT-2311 Support Sparc
WT-2312 re-creating a deleted column-store page can corrupt the in-memory tree
WT-2313 sweep-server: conn_dhandle.c, 610: dhandle != conn->cache->evict_file_next
WT-2314 page-swap error handling is inconsistent
WT-2316 stress test failure: WT_CURSOR.prev out-of-order returns
WT-2320 Only check copyright when cutting releases
WT-2321 WT-2321: race between eviction and worker threads on the eviction queue
WT-2326 Change WTPERF to use new memory allocation functions instead of the standard
WT-2328 schema drop does direct unlink, it should use a block manager interface.
WT-2331 Checking of search() result for reference cursors before join()
WT-2332 Bug in logging write-no-sync mode
WT-2333 Add a flag so drop doesn't block
WT-2335 NULL pointer crash in config_check_search with invalid configuration string
WT-2338 Disable using pre-allocated log files when backup cursor is open
WT-2339 format post-rebalance verify failure (stress run #11586)
WT-2340 Add logging guarantee assertions, whitespace
WT-2342 Enhance wtperf to support background create and drop operations
WT-2344 OS X compiler warning
WT-2347 Java: schema format edge cases
WT-2348 xargs -P isn't portable
WT-2355 Fix minor scratch buffer usage in logging
SERVER-21833 Compact does not release space to the system with WiredTiger
SERVER-21887 $sample takes disproportionately long time on newly created collection
SERVER-22064 Coverity analysis defect 77699: Unchecked return value
SERVER-21944 WiredTiger changes for 3.2.2
Diffstat (limited to 'src/third_party/wiredtiger/src/docs/tools')
-rwxr-xr-x | src/third_party/wiredtiger/src/docs/tools/doxfilter | 30 | ||||
-rwxr-xr-x | src/third_party/wiredtiger/src/docs/tools/doxfilter.py | 194 | ||||
-rwxr-xr-x | src/third_party/wiredtiger/src/docs/tools/doxypy.py | 414 | ||||
-rwxr-xr-x | src/third_party/wiredtiger/src/docs/tools/fixlinks.py | 67 | ||||
-rwxr-xr-x | src/third_party/wiredtiger/src/docs/tools/pyfilter | 30 |
5 files changed, 735 insertions, 0 deletions
diff --git a/src/third_party/wiredtiger/src/docs/tools/doxfilter b/src/third_party/wiredtiger/src/docs/tools/doxfilter new file mode 100755 index 00000000000..298e23f6cbb --- /dev/null +++ b/src/third_party/wiredtiger/src/docs/tools/doxfilter @@ -0,0 +1,30 @@ +#!/bin/sh +# +# Public Domain 2008-2012 WiredTiger, Inc. +# +# This is free and unencumbered software released into the public domain. +# +# Anyone is free to copy, modify, publish, use, compile, sell, or +# distribute this software, either in source code form or as a compiled +# binary, for any purpose, commercial or non-commercial, and by any +# means. +# +# In jurisdictions that recognize copyright laws, the author or authors +# of this software dedicate any and all copyright interest in the +# software to the public domain. We make this dedication for the benefit +# of the public at large and to the detriment of our heirs and +# successors. We intend this dedication to be an overt act of +# relinquishment in perpetuity of all present and future rights to this +# software under copyright law. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +# IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR +# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +# OTHER DEALINGS IN THE SOFTWARE. + +tooldir=`dirname $0` + +python $tooldir/doxfilter.py "$@" diff --git a/src/third_party/wiredtiger/src/docs/tools/doxfilter.py b/src/third_party/wiredtiger/src/docs/tools/doxfilter.py new file mode 100755 index 00000000000..b2d5f857df1 --- /dev/null +++ b/src/third_party/wiredtiger/src/docs/tools/doxfilter.py @@ -0,0 +1,194 @@ +#!/usr/bin/env python +# +# Public Domain 2014-2016 MongoDB, Inc. +# Public Domain 2008-2014 WiredTiger, Inc. +# +# This is free and unencumbered software released into the public domain. +# +# Anyone is free to copy, modify, publish, use, compile, sell, or +# distribute this software, either in source code form or as a compiled +# binary, for any purpose, commercial or non-commercial, and by any +# means. +# +# In jurisdictions that recognize copyright laws, the author or authors +# of this software dedicate any and all copyright interest in the +# software to the public domain. We make this dedication for the benefit +# of the public at large and to the detriment of our heirs and +# successors. We intend this dedication to be an overt act of +# relinquishment in perpetuity of all present and future rights to this +# software under copyright law. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +# IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR +# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +# OTHER DEALINGS IN THE SOFTWARE. + + +# This Python script is run as part of generating the documentation for the +# WiredTiger reference manual. It changes comments to Javadoc style +# (i.e., from "/*!" to "/**"), because the latter are configured to not +# search for brief descriptions at the beginning of pages. +# It also processes any page marked with @m_page specially to create +# multiple per-language versions of the page. + +import re, sys + +progname = 'doxfilter.py' +linenum = 0 +filename = '<unknown>' + +def err(arg): + sys.stderr.write(filename + ':' + str(linenum) + ': ERROR: ' + arg + '\n') + sys.exit(1) + +def java_post_substitutions(source): + result = source + for datatype in [['WT_CONNECTION', 'Connection'], + ['WT_CURSOR', 'Cursor'], + ['WT_SESSION', 'Session'], + ['WT_ASYNC_OPTYPE', 'AsyncOpType'], + ['WT_ASYNC_OP', 'AsyncOp']]: + fromdt = datatype[0] + todt = datatype[1] + + # e.g. replace("WT_CONNECTION::", "Connection."). + # replace("WT_CONNECTION", "Connection") + # replace("::WT_CONNECTION", "Connection") + # etc. + result = result.replace(fromdt + '::', todt + '.') + result = re.sub(r':*' + fromdt, todt, result) + + # We fix back any 'ref' entries, since we don't have + # many java versions of these refered-to pages. + # + # Ideally, we'd have a way ('@m_ref'?) to indicate which + # ones could refer to a Java version. + result = result.replace('ref ' + todt + '.', 'ref ' + fromdt + '::') + result = result.replace('::wiredtiger_open', '\c wiredtiger.open') + return result + +def process_lang(lang, lines): + result = '' + lang_ext = '.' + lang + class_suffix = None + if lang == 'c': + lang_suffix = "" + lang_desc="" + elif lang == 'java': + lang_suffix = "_lang_java" + lang_desc=" in Java" + else: + err('@m_page contains illegal lang: ' + lang) + condstack = [True] + linenum = 0 + mif_pat = re.compile(r'^\s*@m_if{([^,}]*)}') + melse_pat = re.compile(r'^\s*@m_else\s*$') + mendif_pat = re.compile(r'^\s*@m_endif\s*$') + mpage_pat = re.compile(r'@m_page{{([^}]*)},([^,}]*),([^}]*)}') + mpage_rep = r'@page \2' + lang_suffix + r' \3 ' + lang_desc + ref_pat = re.compile(r'@ref\s+(\w*)') + ref_rep = r'@ref \1' + lang_suffix + snip_pat = re.compile(r'@snippet ex_([^.]*)[.]c\s+(.*)') + snip_rep = r'@snippet ex_\1' + lang_ext + r' \2' + section_pat = re.compile(r'(^@\w*section)\s+(\w*)') + section_rep = r'\1 \2' + lang_suffix + subpage_pat = re.compile(r'@subpage\s+(\w*)') + subpage_rep = r'@subpage \1' + lang_suffix + exref_pat = re.compile(r'@ex_ref{ex_([^.]*)[.]c}') + if lang == 'c': + exref_rep = r'@ex_ref{ex_\1' + lang_ext + '}' + else: + # Though we have java examples, we don't have references + # to them working yet, so strip the @ex_ref. + exref_rep = r'ex_\1' + lang_ext + + # Any remaining @m_foo{...} aliases are + # diverted to @c_foo{...} or @java_foo{...} + mgeneric_pat = re.compile(r'@m_([^ }]*)') + mgeneric_rep = r'@' + lang + r'_\1' + for line in lines: + linenum += 1 + if lang != 'c': + line = re.sub(exref_pat, exref_rep, line) + line = re.sub(ref_pat, ref_rep, line) + line = re.sub(section_pat, section_rep, line) + line = re.sub(snip_pat, snip_rep, line) + line = re.sub(mpage_pat, mpage_rep, line) + line = re.sub(subpage_pat, subpage_rep, line) + if '@m_if' in line: + m = re.search(mif_pat, line) + if not m: + err('@m_if incorrect syntax') + iflang = m.groups()[0] + if iflang != 'java' and iflang != 'c': + err('@m_if unknown language') + condstack.append(iflang == lang) + elif '@m_else' in line: + if not re.search(melse_pat, line): + err('@m_else has extraneous stuff') + if len(condstack) <= 1: + err('@m_else missing if') + condstack[-1] = not condstack[-1] + elif '@m_endif' in line: + if not re.search(mendif_pat, line): + err('@m_endif has extraneous stuff') + if len(condstack) <= 1: + err('@m_endif missing if') + condstack.pop() + else: + if condstack[-1]: + # Do generic @m_... macros last + line = re.sub(mgeneric_pat, + mgeneric_rep, line) + result += line + '\n' + if lang == 'java': + result = java_post_substitutions(result) + if len(condstack) != 1: + err('non matching @m_if/@m_endif') + return result + +# Collect all lines between @m_page and the comment end to +# be processed multiple times, once for each language. +def process_multilang(source): + result = '' + in_mpage = False + mpage_content = [] + mpage_pat = re.compile(r'@m_page{{([^}]*)},([^,}]*),([^}]*)}') + mpage_end_pat = re.compile(r'\*/') + for line in source.split('\n'): + m = re.search(mpage_pat, line) + if line.count('@m_page') > 0 and not m: + err('@m_page incorrect syntax') + if m: + if in_mpage: + err('multiple @m_page without end of comment') + else: + in_mpage = True + langs = m.groups()[0].split(',') + if in_mpage: + mpage_content.append(line) + else: + result += line + '\n' + if re.search(mpage_end_pat, line): + if in_mpage: + in_mpage = False + for lang in langs: + result += process_lang(lang, mpage_content) + mpage_content = [] + return result + +def process(source): + source = source.replace(r'/*!', r'/**') + if '@m_' in source: + source = process_multilang(source) + return source + +if __name__ == '__main__': + for f in sys.argv[1:]: + filename = f + with open(f, 'r') as infile: + sys.stdout.write(process(infile.read())) + sys.exit(0) diff --git a/src/third_party/wiredtiger/src/docs/tools/doxypy.py b/src/third_party/wiredtiger/src/docs/tools/doxypy.py new file mode 100755 index 00000000000..54fef5f03a5 --- /dev/null +++ b/src/third_party/wiredtiger/src/docs/tools/doxypy.py @@ -0,0 +1,414 @@ +#!/usr/bin/env python + +__applicationName__ = "doxypy" +__blurb__ = """ +doxypy is an input filter for Doxygen. It preprocesses python +files so that docstrings of classes and functions are reformatted +into Doxygen-conform documentation blocks. +""" + +__doc__ = __blurb__ + \ +""" +In order to make Doxygen preprocess files through doxypy, simply +add the following lines to your Doxyfile: + FILTER_SOURCE_FILES = YES + INPUT_FILTER = "python /path/to/doxypy.py" +""" + +__version__ = "0.4.2" +__date__ = "14th October 2009" +__website__ = "http://code.foosel.org/doxypy" + +__author__ = ( + "Philippe 'demod' Neumann (doxypy at demod dot org)", + "Gina 'foosel' Haeussge (gina at foosel dot net)" +) + +__licenseName__ = "GPL v2" +__license__ = """This program is free software: you can redistribute it and/or modify +it under the terms of the GNU General Public License as published by +the Free Software Foundation, either version 2 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program. If not, see <http://www.gnu.org/licenses/>. +""" + +import sys +import re + +from optparse import OptionParser, OptionGroup + +class FSM(object): + """Implements a finite state machine. + + Transitions are given as 4-tuples, consisting of an origin state, a target + state, a condition for the transition (given as a reference to a function + which gets called with a given piece of input) and a pointer to a function + to be called upon the execution of the given transition. + """ + + """ + @var transitions holds the transitions + @var current_state holds the current state + @var current_input holds the current input + @var current_transition hold the currently active transition + """ + + def __init__(self, start_state=None, transitions=[]): + self.transitions = transitions + self.current_state = start_state + self.current_input = None + self.current_transition = None + + def setStartState(self, state): + self.current_state = state + + def addTransition(self, from_state, to_state, condition, callback): + self.transitions.append([from_state, to_state, condition, callback]) + + def makeTransition(self, input): + """Makes a transition based on the given input. + + @param input input to parse by the FSM + """ + for transition in self.transitions: + [from_state, to_state, condition, callback] = transition + if from_state == self.current_state: + match = condition(input) + if match: + self.current_state = to_state + self.current_input = input + self.current_transition = transition + if options.debug: + print >>sys.stderr, "# FSM: executing (%s -> %s) for line '%s'" % (from_state, to_state, input) + callback(match) + return + +class Doxypy(object): + def __init__(self): + string_prefixes = "[uU]?[rR]?" + + self.start_single_comment_re = re.compile("^\s*%s(''')" % string_prefixes) + self.end_single_comment_re = re.compile("(''')\s*$") + + self.start_double_comment_re = re.compile("^\s*%s(\"\"\")" % string_prefixes) + self.end_double_comment_re = re.compile("(\"\"\")\s*$") + + self.single_comment_re = re.compile("^\s*%s(''').*(''')\s*$" % string_prefixes) + self.double_comment_re = re.compile("^\s*%s(\"\"\").*(\"\"\")\s*$" % string_prefixes) + + self.defclass_re = re.compile("^(\s*)(def .+:|class .+:)") + self.empty_re = re.compile("^\s*$") + self.hashline_re = re.compile("^\s*#.*$") + self.importline_re = re.compile("^\s*(import |from .+ import)") + + self.multiline_defclass_start_re = re.compile("^(\s*)(def|class)(\s.*)?$") + self.multiline_defclass_end_re = re.compile(":\s*$") + + ## Transition list format + # ["FROM", "TO", condition, action] + transitions = [ + ### FILEHEAD + + # single line comments + ["FILEHEAD", "FILEHEAD", self.single_comment_re.search, self.appendCommentLine], + ["FILEHEAD", "FILEHEAD", self.double_comment_re.search, self.appendCommentLine], + + # multiline comments + ["FILEHEAD", "FILEHEAD_COMMENT_SINGLE", self.start_single_comment_re.search, self.appendCommentLine], + ["FILEHEAD_COMMENT_SINGLE", "FILEHEAD", self.end_single_comment_re.search, self.appendCommentLine], + ["FILEHEAD_COMMENT_SINGLE", "FILEHEAD_COMMENT_SINGLE", self.catchall, self.appendCommentLine], + ["FILEHEAD", "FILEHEAD_COMMENT_DOUBLE", self.start_double_comment_re.search, self.appendCommentLine], + ["FILEHEAD_COMMENT_DOUBLE", "FILEHEAD", self.end_double_comment_re.search, self.appendCommentLine], + ["FILEHEAD_COMMENT_DOUBLE", "FILEHEAD_COMMENT_DOUBLE", self.catchall, self.appendCommentLine], + + # other lines + ["FILEHEAD", "FILEHEAD", self.empty_re.search, self.appendFileheadLine], + ["FILEHEAD", "FILEHEAD", self.hashline_re.search, self.appendFileheadLine], + ["FILEHEAD", "FILEHEAD", self.importline_re.search, self.appendFileheadLine], + ["FILEHEAD", "DEFCLASS", self.defclass_re.search, self.resetCommentSearch], + ["FILEHEAD", "DEFCLASS_MULTI", self.multiline_defclass_start_re.search, self.resetCommentSearch], + ["FILEHEAD", "DEFCLASS_BODY", self.catchall, self.appendFileheadLine], + + ### DEFCLASS + + # single line comments + ["DEFCLASS", "DEFCLASS_BODY", self.single_comment_re.search, self.appendCommentLine], + ["DEFCLASS", "DEFCLASS_BODY", self.double_comment_re.search, self.appendCommentLine], + + # multiline comments + ["DEFCLASS", "COMMENT_SINGLE", self.start_single_comment_re.search, self.appendCommentLine], + ["COMMENT_SINGLE", "DEFCLASS_BODY", self.end_single_comment_re.search, self.appendCommentLine], + ["COMMENT_SINGLE", "COMMENT_SINGLE", self.catchall, self.appendCommentLine], + ["DEFCLASS", "COMMENT_DOUBLE", self.start_double_comment_re.search, self.appendCommentLine], + ["COMMENT_DOUBLE", "DEFCLASS_BODY", self.end_double_comment_re.search, self.appendCommentLine], + ["COMMENT_DOUBLE", "COMMENT_DOUBLE", self.catchall, self.appendCommentLine], + + # other lines + ["DEFCLASS", "DEFCLASS", self.empty_re.search, self.appendDefclassLine], + ["DEFCLASS", "DEFCLASS", self.defclass_re.search, self.resetCommentSearch], + ["DEFCLASS", "DEFCLASS_MULTI", self.multiline_defclass_start_re.search, self.resetCommentSearch], + ["DEFCLASS", "DEFCLASS_BODY", self.catchall, self.stopCommentSearch], + + ### DEFCLASS_BODY + + ["DEFCLASS_BODY", "DEFCLASS", self.defclass_re.search, self.startCommentSearch], + ["DEFCLASS_BODY", "DEFCLASS_MULTI", self.multiline_defclass_start_re.search, self.startCommentSearch], + ["DEFCLASS_BODY", "DEFCLASS_BODY", self.catchall, self.appendNormalLine], + + ### DEFCLASS_MULTI + ["DEFCLASS_MULTI", "DEFCLASS", self.multiline_defclass_end_re.search, self.appendDefclassLine], + ["DEFCLASS_MULTI", "DEFCLASS_MULTI", self.catchall, self.appendDefclassLine], + ] + + self.fsm = FSM("FILEHEAD", transitions) + self.outstream = sys.stdout + + self.output = [] + self.comment = [] + self.filehead = [] + self.defclass = [] + self.indent = "" + + def __closeComment(self): + """Appends any open comment block and triggering block to the output.""" + + if options.autobrief: + if len(self.comment) == 1 \ + or (len(self.comment) > 2 and self.comment[1].strip() == ''): + self.comment[0] = self.__docstringSummaryToBrief(self.comment[0]) + + if self.comment: + block = self.makeCommentBlock() + self.output.extend(block) + + if self.defclass: + self.output.extend(self.defclass) + + def __docstringSummaryToBrief(self, line): + """Adds \\brief to the docstrings summary line. + + A \\brief is prepended, provided no other doxygen command is at the + start of the line. + """ + stripped = line.strip() + if stripped and not stripped[0] in ('@', '\\'): + return "\\brief " + line + else: + return line + + def __flushBuffer(self): + """Flushes the current outputbuffer to the outstream.""" + if self.output: + try: + if options.debug: + print >>sys.stderr, "# OUTPUT: ", self.output + print >>self.outstream, "\n".join(self.output) + self.outstream.flush() + except IOError: + # Fix for FS#33. Catches "broken pipe" when doxygen closes + # stdout prematurely upon usage of INPUT_FILTER, INLINE_SOURCES + # and FILTER_SOURCE_FILES. + pass + self.output = [] + + def catchall(self, input): + """The catchall-condition, always returns true.""" + return True + + def resetCommentSearch(self, match): + """Restarts a new comment search for a different triggering line. + + Closes the current commentblock and starts a new comment search. + """ + if options.debug: + print >>sys.stderr, "# CALLBACK: resetCommentSearch" + self.__closeComment() + self.startCommentSearch(match) + + def startCommentSearch(self, match): + """Starts a new comment search. + + Saves the triggering line, resets the current comment and saves + the current indentation. + """ + if options.debug: + print >>sys.stderr, "# CALLBACK: startCommentSearch" + self.defclass = [self.fsm.current_input] + self.comment = [] + self.indent = match.group(1) + + def stopCommentSearch(self, match): + """Stops a comment search. + + Closes the current commentblock, resets the triggering line and + appends the current line to the output. + """ + if options.debug: + print >>sys.stderr, "# CALLBACK: stopCommentSearch" + self.__closeComment() + + self.defclass = [] + self.output.append(self.fsm.current_input) + + def appendFileheadLine(self, match): + """Appends a line in the FILEHEAD state. + + Closes the open comment block, resets it and appends the current line. + """ + if options.debug: + print >>sys.stderr, "# CALLBACK: appendFileheadLine" + self.__closeComment() + self.comment = [] + self.output.append(self.fsm.current_input) + + def appendCommentLine(self, match): + """Appends a comment line. + + The comment delimiter is removed from multiline start and ends as + well as singleline comments. + """ + if options.debug: + print >>sys.stderr, "# CALLBACK: appendCommentLine" + (from_state, to_state, condition, callback) = self.fsm.current_transition + + # single line comment + if (from_state == "DEFCLASS" and to_state == "DEFCLASS_BODY") \ + or (from_state == "FILEHEAD" and to_state == "FILEHEAD"): + # remove comment delimiter from begin and end of the line + activeCommentDelim = match.group(1) + line = self.fsm.current_input + self.comment.append(line[line.find(activeCommentDelim)+len(activeCommentDelim):line.rfind(activeCommentDelim)]) + + if (to_state == "DEFCLASS_BODY"): + self.__closeComment() + self.defclass = [] + # multiline start + elif from_state == "DEFCLASS" or from_state == "FILEHEAD": + # remove comment delimiter from begin of the line + activeCommentDelim = match.group(1) + line = self.fsm.current_input + self.comment.append(line[line.find(activeCommentDelim)+len(activeCommentDelim):]) + # multiline end + elif to_state == "DEFCLASS_BODY" or to_state == "FILEHEAD": + # remove comment delimiter from end of the line + activeCommentDelim = match.group(1) + line = self.fsm.current_input + self.comment.append(line[0:line.rfind(activeCommentDelim)]) + if (to_state == "DEFCLASS_BODY"): + self.__closeComment() + self.defclass = [] + # in multiline comment + else: + # just append the comment line + self.comment.append(self.fsm.current_input) + + def appendNormalLine(self, match): + """Appends a line to the output.""" + if options.debug: + print >>sys.stderr, "# CALLBACK: appendNormalLine" + self.output.append(self.fsm.current_input) + + def appendDefclassLine(self, match): + """Appends a line to the triggering block.""" + if options.debug: + print >>sys.stderr, "# CALLBACK: appendDefclassLine" + self.defclass.append(self.fsm.current_input) + + def makeCommentBlock(self): + """Indents the current comment block with respect to the current + indentation level. + + @returns a list of indented comment lines + """ + doxyStart = "##" + commentLines = self.comment + + commentLines = map(lambda x: "%s# %s" % (self.indent, x), commentLines) + l = [self.indent + doxyStart] + l.extend(commentLines) + + return l + + def parse(self, input): + """Parses a python file given as input string and returns the doxygen- + compatible representation. + + @param input the python code to parse + @returns the modified python code + """ + lines = input.split("\n") + + for line in lines: + self.fsm.makeTransition(line) + + if self.fsm.current_state == "DEFCLASS": + self.__closeComment() + + return "\n".join(self.output) + + def parseFile(self, filename): + """Parses a python file given as input string and returns the doxygen- + compatible representation. + + @param input the python code to parse + @returns the modified python code + """ + f = open(filename, 'r') + + for line in f: + self.parseLine(line.rstrip('\r\n')) + if self.fsm.current_state == "DEFCLASS": + self.__closeComment() + self.__flushBuffer() + f.close() + + def parseLine(self, line): + """Parse one line of python and flush the resulting output to the + outstream. + + @param line the python code line to parse + """ + self.fsm.makeTransition(line) + self.__flushBuffer() + +def optParse(): + """Parses commandline options.""" + parser = OptionParser(prog=__applicationName__, version="%prog " + __version__) + + parser.set_usage("%prog [options] filename") + parser.add_option("--autobrief", + action="store_true", dest="autobrief", + help="use the docstring summary line as \\brief description" + ) + parser.add_option("--debug", + action="store_true", dest="debug", + help="enable debug output on stderr" + ) + + ## parse options + global options + (options, filename) = parser.parse_args() + + if not filename: + print >>sys.stderr, "No filename given." + sys.exit(-1) + + return filename[0] + +def main(): + """Starts the parser on the file given by the filename as the first + argument on the commandline. + """ + filename = optParse() + fsm = Doxypy() + fsm.parseFile(filename) + +if __name__ == "__main__": + main() diff --git a/src/third_party/wiredtiger/src/docs/tools/fixlinks.py b/src/third_party/wiredtiger/src/docs/tools/fixlinks.py new file mode 100755 index 00000000000..7163246e3bd --- /dev/null +++ b/src/third_party/wiredtiger/src/docs/tools/fixlinks.py @@ -0,0 +1,67 @@ +#!/usr/bin/env python +# +# Public Domain 2014-2016 MongoDB, Inc. +# Public Domain 2008-2014 WiredTiger, Inc. +# +# This is free and unencumbered software released into the public domain. +# +# Anyone is free to copy, modify, publish, use, compile, sell, or +# distribute this software, either in source code form or as a compiled +# binary, for any purpose, commercial or non-commercial, and by any +# means. +# +# In jurisdictions that recognize copyright laws, the author or authors +# of this software dedicate any and all copyright interest in the +# software to the public domain. We make this dedication for the benefit +# of the public at large and to the detriment of our heirs and +# successors. We intend this dedication to be an overt act of +# relinquishment in perpetuity of all present and future rights to this +# software under copyright law. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +# IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR +# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +# OTHER DEALINGS IN THE SOFTWARE. +# +# This Python script is run as part of generating the documentation for the +# WiredTiger Python API. It runs after doxypy, and turns method names in +# comments into references to the corresponding function in the C API. + +import re, sys + +def process(source): + # Replace standard struct wrapper wording + source = re.sub(r'(\s+#.*)Proxy of C (\w+) struct', + lambda m: ('%sPython wrapper around C ::%s%s@copydoc ::%s' % + (m.group(1), m.group(2).strip('_').upper(), m.group(1), + m.group(2).strip('_').upper())), source) + + # Replace lowercase class names with upper case + source = re.sub(r'(\s+#.*)__(wt_\w+)::', + lambda m: ('%s%s::' % (m.group(1), m.group(2).upper())), source) + + # Replace "char" with "string" in comments + source = re.sub(r'(\s+#.*)\bconst char \*', r'\1string', source) + source = re.sub(r'(\s+#.*)\bchar const \*', r'\1string', source) + source = re.sub(r'(\s+#.*)\bchar\b', r'\1string', source) + + # Copy documentation -- methods, then global functions + source = re.sub(r'(\s+# )(\w+)\(self, (connection|cursor|session).*', + lambda m: ('%s%s%s@copydoc WT_%s::%s' % + (m.group(0), m.group(1), m.group(1), m.group(3).upper(), m.group(2))), + source) + + source = re.sub(r'(\s+# )(wiredtiger_\w+)\(.*', + lambda m: ('%s%s%s@copydoc ::%s' % + (m.group(0), m.group(1), m.group(1), m.group(2))), source) + + # Replace "self, handle" with "self" -- these are typedef'ed away + source = re.sub(r'(\s+#.*self), + (?:connection|cursor|session)', r'\1', source) + return source + +if __name__ == '__main__': + sys.stdout.write(process(sys.stdin.read())) diff --git a/src/third_party/wiredtiger/src/docs/tools/pyfilter b/src/third_party/wiredtiger/src/docs/tools/pyfilter new file mode 100755 index 00000000000..7ef80325bd1 --- /dev/null +++ b/src/third_party/wiredtiger/src/docs/tools/pyfilter @@ -0,0 +1,30 @@ +#!/bin/sh +# +# Public Domain 2008-2012 WiredTiger, Inc. +# +# This is free and unencumbered software released into the public domain. +# +# Anyone is free to copy, modify, publish, use, compile, sell, or +# distribute this software, either in source code form or as a compiled +# binary, for any purpose, commercial or non-commercial, and by any +# means. +# +# In jurisdictions that recognize copyright laws, the author or authors +# of this software dedicate any and all copyright interest in the +# software to the public domain. We make this dedication for the benefit +# of the public at large and to the detriment of our heirs and +# successors. We intend this dedication to be an overt act of +# relinquishment in perpetuity of all present and future rights to this +# software under copyright law. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +# IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR +# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +# OTHER DEALINGS IN THE SOFTWARE. + +tooldir=`dirname $0` + +python $tooldir/doxypy.py "$@" | python $tooldir/fixlinks.py |