diff options
50 files changed, 1882 insertions, 284 deletions
diff --git a/GNU-AGPL-3.0.txt b/GNU-AGPL-3.0.txt new file mode 100644 index 00000000000..dba13ed2ddf --- /dev/null +++ b/GNU-AGPL-3.0.txt @@ -0,0 +1,661 @@ + GNU AFFERO GENERAL PUBLIC LICENSE + Version 3, 19 November 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/> + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU Affero General Public License is a free, copyleft license for +software and other kinds of works, specifically designed to ensure +cooperation with the community in the case of network server software. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +our General Public Licenses are intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + Developers that use our General Public Licenses protect your rights +with two steps: (1) assert copyright on the software, and (2) offer +you this License which gives you legal permission to copy, distribute +and/or modify the software. + + A secondary benefit of defending all users' freedom is that +improvements made in alternate versions of the program, if they +receive widespread use, become available for other developers to +incorporate. Many developers of free software are heartened and +encouraged by the resulting cooperation. However, in the case of +software used on network servers, this result may fail to come about. +The GNU General Public License permits making a modified version and +letting the public access it on a server without ever releasing its +source code to the public. + + The GNU Affero General Public License is designed specifically to +ensure that, in such cases, the modified source code becomes available +to the community. It requires the operator of a network server to +provide the source code of the modified version running there to the +users of that server. Therefore, public use of a modified version, on +a publicly accessible server, gives the public access to the source +code of the modified version. + + An older license, called the Affero General Public License and +published by Affero, was designed to accomplish similar goals. This is +a different license, not a version of the Affero GPL, but Affero has +released a new version of the Affero GPL which permits relicensing under +this license. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU Affero General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Remote Network Interaction; Use with the GNU General Public License. + + Notwithstanding any other provision of this License, if you modify the +Program, your modified version must prominently offer all users +interacting with it remotely through a computer network (if your version +supports such interaction) an opportunity to receive the Corresponding +Source of your version by providing access to the Corresponding Source +from a network server at no charge, through some standard or customary +means of facilitating copying of software. This Corresponding Source +shall include the Corresponding Source for any work covered by version 3 +of the GNU General Public License that is incorporated pursuant to the +following paragraph. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the work with which it is combined will remain governed by version +3 of the GNU General Public License. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU Affero General Public License from time to time. Such new versions +will be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU Affero General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU Affero General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU Affero General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + <one line to give the program's name and a brief idea of what it does.> + Copyright (C) <year> <name of author> + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License + along with this program. If not, see <http://www.gnu.org/licenses/>. + +Also add information on how to contact you by electronic and paper mail. + + If your software can interact with users remotely through a computer +network, you should also make sure that it provides a way for users to +get its source. For example, if your program is a web application, its +interface could display a "Source" link that leads users to an archive +of the code. There are many ways you could offer source, and different +solutions will be better for different programs; see section 13 for the +specific requirements. + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU AGPL, see +<http://www.gnu.org/licenses/>. diff --git a/README b/README new file mode 100644 index 00000000000..bca1604b0e1 --- /dev/null +++ b/README @@ -0,0 +1,38 @@ +Mongo Database README + +DOCUMENTATION + +http://www.10gen.com/wiki/db + +ORGANIZATION + +db/ : core database source code +grid/ : socket messaging stuff +util/ : util classes + +BUILDING + +cd db +make clean && make + +RUNNING + +Run ./db and command line options will display. + +COMPILER VERSIONS + +Mongo has been tested with GCC 4.1.2 and Visual Studio 2008. Older versions +of GCC may not be happy. + +NOTES + +Mongo uses the 10gen appserver and the Java Virtual Machine to execute +user program code in the database. This feature is optional -- if you would +like to run the database without these installed, run with the --nojni +option. With --nojni specified, the db.eval() and $where features are no +longer available. + +Mongo uses memory mapped files. If built as a 32 bit executable, you will +not be able to work with large (multi-gigabyte) databases. However, 32 bit +builds should work fine with small development databases. + diff --git a/db/btree.cpp b/db/btree.cpp index 4d141722283..86dee51eaa4 100644 --- a/db/btree.cpp +++ b/db/btree.cpp @@ -1,5 +1,21 @@ // btree.cpp +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "stdafx.h" #include "btree.h" #include "pdfile.h" @@ -22,6 +38,10 @@ KeyNode::KeyNode(BucketBasics& bb, _KeyNode &k) : /* BucketBasics --------------------------------------------------- */ +int BucketBasics::Size() const { + assert( _Size == BucketSize ); + return _Size; +} inline void BucketBasics::setNotPacked() { flags &= ~Packed; } inline void BucketBasics::setPacked() { flags |= Packed; } @@ -91,11 +111,10 @@ int nDumped = 0; void BucketBasics::assertValid(bool force) { if( !debug && !force ) return; - wassert( n >= 0 && n < BucketSize ); + wassert( n >= 0 && n < Size() ); wassert( emptySize >= 0 && emptySize < BucketSize ); wassert( topSize >= n && topSize <= BucketSize ); - wassert( Size == BucketSize ); - if( 1 ) { + DEV { // slow: for( int i = 0; i < n-1; i++ ) { JSObj k1 = keyNode(i).key; @@ -113,7 +132,12 @@ void BucketBasics::assertValid(bool force) { break; } else if( z == 0 ) { - wassert( k(i).recordLoc < k(i+1).recordLoc ); + if( !(k(i).recordLoc < k(i+1).recordLoc) ) { + cout << "ERROR: btree key order corrupt (recordloc's wrong). Keys:" << endl; + cout << " k(" << i << "):" << keyNode(i).key.toString() << " RL:" << k(i).recordLoc.toString() << endl; + cout << " k(" << i+1 << "):" << keyNode(i+1).key.toString() << " RL:" << k(i+1).recordLoc.toString() << endl; + wassert( k(i).recordLoc < k(i+1).recordLoc ); + } } } } @@ -134,12 +158,12 @@ inline void BucketBasics::markUnused(int keypos) { } inline int BucketBasics::totalDataSize() const { - return Size - (data-(char*)this); + return Size() - (data-(char*)this); } void BucketBasics::init(){ parent.Null(); nextChild.Null(); - Size = BucketSize; + _Size = BucketSize; flags = Packed; n = 0; emptySize = totalDataSize(); topSize = 0; @@ -338,10 +362,18 @@ void BtreeBucket::delBucket(const DiskLoc& thisLoc, IndexDetails& id) { assert(false); } found: +#if 1 + /* as a temporary defensive measure, we zap the whole bucket, AND don't truly delete + it (meaning it is ineligible for reuse). temporary to see if it helps with some + issues. + */ + memset(this, 0, Size()); +#else //defensive: n = -1; parent.Null(); theDataFileMgr.deleteRecord(id.indexNamespace().c_str(), thisLoc.rec(), thisLoc); +#endif } /* note: may delete the entire bucket! this invalid upon return sometimes. */ @@ -515,7 +547,7 @@ void BtreeBucket::insertHere(DiskLoc thisLoc, int keypos, r->nextChild = nextChild; r->assertValid(); //r->dump(); - rLoc = theDataFileMgr.insert(idx.indexNamespace().c_str(), r, r->Size, true); + rLoc = theDataFileMgr.insert(idx.indexNamespace().c_str(), r, r->Size(), true); if( split_debug ) cout << " new rLoc:" << rLoc.toString() << endl; free(r); r = 0; @@ -536,7 +568,7 @@ void BtreeBucket::insertHere(DiskLoc thisLoc, int keypos, p->pushBack(middle.recordLoc, middle.key, thisLoc); p->nextChild = rLoc; p->assertValid(); - parent = idx.head = theDataFileMgr.insert(idx.indexNamespace().c_str(), p, p->Size, true); + parent = idx.head = theDataFileMgr.insert(idx.indexNamespace().c_str(), p, p->Size(), true); if( split_debug ) cout << " we were root, making new root:" << hex << parent.getOfs() << dec << endl; free(p); @@ -591,7 +623,8 @@ void BtreeBucket::insertHere(DiskLoc thisLoc, int keypos, /* start a new index off, empty */ DiskLoc BtreeBucket::addHead(IndexDetails& id) { BtreeBucket *p = allocTemp(); - DiskLoc loc = theDataFileMgr.insert(id.indexNamespace().c_str(), p, p->Size, true); + DiskLoc loc = theDataFileMgr.insert(id.indexNamespace().c_str(), p, p->Size(), true); + free(p); return loc; } @@ -608,7 +641,7 @@ DiskLoc BtreeBucket::advance(const DiskLoc& thisLoc, int& keyOfs, int direction, cout << " thisLoc: " << thisLoc.toString() << endl; cout << " keyOfs: " << keyOfs << " n:" << n << " direction: " << direction << endl; cout << bucketSummary() << endl; - assert( keyOfs >= 0 && keyOfs < n ); + assert(false); } int adj = direction < 0 ? 1 : 0; int ko = keyOfs + direction; @@ -866,34 +899,48 @@ void BtreeCursor::noteLocation() { } } -int clctr = 0; +/* Since the last noteLocation(), our key may have moved around, and that old cached + information may thus be stale and wrong (although often it is right). We check + that here; if we have moved, we have to search back for where we were at. -/* see if things moved around (deletes, splits, inserts) */ + i.e., after operations on the index, the BtreeCursor's cached location info may + be invalid. This function ensures validity, so you should call it before using + the cursor if other writers have used the database since the last noteLocation + call. +*/ void BtreeCursor::checkLocation() { - try { + { if( eof() ) return; - BtreeBucket *b = bucket.btree(); + BtreeBucket *b = bucket.btree(); + + assert( !keyAtKeyOfs.isEmpty() ); + + // Note keyAt() returns an empty JSObj if keyOfs is now out of range, + // which is possible as keys may have been deleted. if( b->keyAt(keyOfs).woEqual(keyAtKeyOfs) && b->k(keyOfs).recordLoc == locAtKeyOfs ) { - if( !b->k(keyOfs).isUsed() ) - checkUnused(); - return; + if( !b->k(keyOfs).isUsed() ) { + /* we were deleted but still exist as an unused + marker key. advance. + */ + checkUnused(); + } + return; } } - catch( AssertionException ) { - cout << "Caught exception in checkLocation(), that's maybe ok" << endl; - } + + /* normally we don't get to here. when we do, old position is no longer + valid and we must refind where we left off (which is expensive) + */ bool found; DiskLoc bold = bucket; -/* TODO: Switch to keep indexdetails and do idx.head! */ - /* didn't find, check from the top */ + /* TODO: Switch to keep indexdetails and do idx.head! */ DiskLoc head = bold.btree()->getHead(bold); bucket = head.btree()->locate(head, keyAtKeyOfs, keyOfs, found, locAtKeyOfs, direction); - if( clctr++ % 128 == 0 ) - cout << " key seems to have moved in the index, refinding. found:" << found << endl; + RARELY cout << " key seems to have moved in the index, refinding. found:" << found << endl; if( found ) checkUnused(); } diff --git a/db/btree.h b/db/btree.h index 063196f67b4..5e5b499f521 100644 --- a/db/btree.h +++ b/db/btree.h @@ -1,5 +1,21 @@ // btree.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #include "../stdafx.h" @@ -17,9 +33,15 @@ struct _KeyNode { unsigned short _kdo; void setKeyDataOfs(short s) { _kdo = s; assert(s>=0); } void setKeyDataOfsSavingUse(short s) { _kdo = s; assert(s>=0); } - void setUnused() { recordLoc.GETOFS() |= 1; } -// void setUsed() { _kdo &= 0x7fff; } - int isUnused() { return (recordLoc.getOfs() & 1); } + void setUnused() { + /* Setting ofs to odd is the sentinel for unused, as real recordLoc's are always + even numbers. + Note we need to keep its value basically the same as we use the recordLoc + as part of the key in the index (to handle duplicate keys efficiently). + */ + recordLoc.GETOFS() |= 1; + } + int isUnused() { return recordLoc.getOfs() & 1; } int isUsed() { return !isUnused(); } }; @@ -92,7 +114,7 @@ public: ss << " n: " << n << endl; ss << " parent: " << parent.toString() << endl; ss << " nextChild: " << parent.toString() << endl; - ss << " Size: " << Size << " flags:" << flags << endl; + ss << " Size: " << _Size << " flags:" << flags << endl; ss << " emptySize: " << emptySize << " topSize: " << topSize << endl; return ss.str(); } @@ -100,7 +122,8 @@ public: protected: void _shape(int level, stringstream&); DiskLoc nextChild; // child bucket off and to the right of the highest key. - int Size; // total size of this btree node in bytes. constant. + int _Size; // total size of this btree node in bytes. constant. + int Size() const; int flags; int emptySize; // size of the empty region int topSize; // size of the data at the top of the bucket (keys are at the beginning or 'bottom') diff --git a/db/clientcursor.cpp b/db/clientcursor.cpp index 6782d893ea1..5a6dece5a46 100644 --- a/db/clientcursor.cpp +++ b/db/clientcursor.cpp @@ -1,3 +1,19 @@ +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + /* clientcursor.cpp ClientCursor is a wrapper that represents a cursorid from our client @@ -42,6 +58,26 @@ void removedKey(const DiskLoc& btreeLoc, int keyPos) { // TODO!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! } +/* todo: this implementation is incomplete. we use it as a prefix for dropDatabase, which + works fine as the prefix will end with '.'. however, when used with drop and + deleteIndexes, this could take out cursors that belong to something else -- if you + drop "foo", currently, this will kill cursors for "foobar". +*/ +void ClientCursor::invalidate(const char *nsPrefix) { + vector<ClientCursor*> toDelete; + + int len = strlen(nsPrefix); + assert( len > 0 && strchr(nsPrefix, '.') ); + for( ByLoc::iterator i = byLoc.begin(); i != byLoc.end(); ++i ) { + ClientCursor *cc = i->second; + if( strncmp(nsPrefix, cc->ns.c_str(), len) == 0 ) + toDelete.push_back(i->second); + } + + for( vector<ClientCursor*>::iterator i = toDelete.begin(); i != toDelete.end(); ++i ) + delete (*i); +} + /* must call this on a delete so we clean up the cursors. */ void aboutToDelete(const DiskLoc& dl) { vector<ClientCursor*> toAdvance; @@ -92,7 +128,7 @@ long long ClientCursor::allocCursorId() { while( 1 ) { x = (((long long)rand()) << 32); x = x | ctm | 0x80000000; // OR to make sure not zero - if( ctm != ctmLast || ClientCursor::find(x) == 0 ) + if( ctm != ctmLast || ClientCursor::find(x, false) == 0 ) break; } ctmLast = ctm; @@ -143,4 +179,3 @@ class CursInspector : public SingleResultObjCursor { public: CursInspector() { reg("intr.cursors"); } } _ciproto; - diff --git a/db/clientcursor.h b/db/clientcursor.h index 46570c30f22..015b2e2c4b3 100644 --- a/db/clientcursor.h +++ b/db/clientcursor.h @@ -1,3 +1,19 @@ +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + /* clientcursor.h Cursor -- and its derived classes -- are our internal cursors. @@ -36,6 +52,11 @@ public: auto_ptr< set<string> > filter; // which fields query wants returned Message originalMessage; // this is effectively an auto ptr for data the matcher points to. + /* Get rid of cursors for namespaces that begin with nsprefix. + Used by drop, deleteIndexes, dropDatabase. + */ + static void invalidate(const char *nsPrefix); + static bool erase(CursorId id) { ClientCursor *cc = find(id); if( cc ) { @@ -45,10 +66,11 @@ public: return false; } - static ClientCursor* find(CursorId id) { + static ClientCursor* find(CursorId id, bool warn = true) { CCById::iterator it = clientCursorsById.find(id); if( it == clientCursorsById.end() ) { - cout << "ClientCursor::find(): cursor not found in map " << id << '\n'; + if( warn ) + OCCASIONALLY cout << "ClientCursor::find(): cursor not found in map " << id << " (ok after a drop)\n"; return 0; } return it->second; @@ -60,10 +82,5 @@ public: */ void updateLocation(); -//private: -// void addToByLocation(DiskLoc cl); void cleanupByLocation(DiskLoc loc); -//public: -// ClientCursor *nextAtThisLocation; }; - diff --git a/db/db.cpp b/db/db.cpp index 5241a15f9f4..89191ea7598 100644 --- a/db/db.cpp +++ b/db/db.cpp @@ -1,6 +1,22 @@ // db.cpp : Defines the entry point for the console application. // +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "stdafx.h" #include "db.h" #include "../grid/message.h" @@ -480,6 +496,9 @@ void jniCallback(Message& m, Message& out) } } +/* we create one thread for each connection from an app server client. + app server will open a pool of threads. +*/ void connThread() { try { @@ -927,8 +946,8 @@ int main(int argc, char* argv[], char *envp[] ) //#endif #undef exit -void dbexit(int rc) { - cout << " dbexit: flushing op log and files" << endl; +void dbexit(int rc, const char *why) { + cout << " dbexit: " << why << "; flushing op log and files" << endl; flushOpLog(); /* must do this before unmapping mem or you may get a seg fault */ @@ -1,3 +1,19 @@ +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "../stdafx.h" #include "../grid/message.h" diff --git a/db/db.vcproj.Dwight-PC.Dwight Merriman.user b/db/db.vcproj.Dwight-PC.Dwight Merriman.user deleted file mode 100644 index d729516a29f..00000000000 --- a/db/db.vcproj.Dwight-PC.Dwight Merriman.user +++ /dev/null @@ -1,65 +0,0 @@ -<?xml version="1.0" encoding="Windows-1252"?>
-<VisualStudioUserFile
- ProjectType="Visual C++"
- Version="9.00"
- ShowAllFiles="false"
- >
- <Configurations>
- <Configuration
- Name="Debug|Win32"
- >
- <DebugSettings
- Command="$(TargetPath)"
- WorkingDirectory=""
- CommandArguments="run"
- Attach="false"
- DebuggerType="3"
- Remote="1"
- RemoteMachine="DWIGHT-PC"
- RemoteCommand=""
- HttpUrl=""
- PDBPath=""
- SQLDebugging=""
- Environment="PATH=C:\Program Files\Java\jdk1.6.0_05\jre\bin\client"
- EnvironmentMerge="true"
- DebuggerFlavor="0"
- MPIRunCommand=""
- MPIRunArguments=""
- MPIRunWorkingDirectory=""
- ApplicationCommand=""
- ApplicationArguments=""
- ShimCommand=""
- MPIAcceptMode=""
- MPIAcceptFilter=""
- />
- </Configuration>
- <Configuration
- Name="Release|Win32"
- >
- <DebugSettings
- Command="$(TargetPath)"
- WorkingDirectory=""
- CommandArguments=""
- Attach="false"
- DebuggerType="3"
- Remote="1"
- RemoteMachine="DWIGHT-PC"
- RemoteCommand=""
- HttpUrl=""
- PDBPath=""
- SQLDebugging=""
- Environment=""
- EnvironmentMerge="true"
- DebuggerFlavor=""
- MPIRunCommand=""
- MPIRunArguments=""
- MPIRunWorkingDirectory=""
- ApplicationCommand=""
- ApplicationArguments=""
- ShimCommand=""
- MPIAcceptMode=""
- MPIAcceptFilter=""
- />
- </Configuration>
- </Configurations>
-</VisualStudioUserFile>
diff --git a/db/introspect.cpp b/db/introspect.cpp index 34db42a143d..e3e42ed0ff3 100644 --- a/db/introspect.cpp +++ b/db/introspect.cpp @@ -1,5 +1,21 @@ // introspect.cpp +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "stdafx.h" #include "introspect.h" #include "../util/builder.h" diff --git a/db/introspect.h b/db/introspect.h index 1dd7f3c8ee4..317e4fbcd5b 100644 --- a/db/introspect.h +++ b/db/introspect.h @@ -1,6 +1,22 @@ // introspect.h // system management stuff. +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #include "../stdafx.h" @@ -43,4 +59,3 @@ public: void profile(const char *str, int millis); - diff --git a/db/javajs.cpp b/db/javajs.cpp index d935d76f3ca..8d68c22458a 100644 --- a/db/javajs.cpp +++ b/db/javajs.cpp @@ -1,5 +1,21 @@ // java.cpp +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "stdafx.h" #include "javajs.h" #include <iostream> diff --git a/db/javajs.h b/db/javajs.h index 104b977a013..3c65bf8ffd9 100644 --- a/db/javajs.h +++ b/db/javajs.h @@ -1,5 +1,21 @@ // java.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #include "../stdafx.h" diff --git a/db/javatest.cpp b/db/javatest.cpp index 3e31c74b696..b4af248f7a9 100644 --- a/db/javatest.cpp +++ b/db/javatest.cpp @@ -1,5 +1,21 @@ // javatest.cpp +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "javajs.h" int main(){ diff --git a/db/jsobj.cpp b/db/jsobj.cpp index 50c7e2ae720..423adb07131 100644 --- a/db/jsobj.cpp +++ b/db/jsobj.cpp @@ -1,5 +1,21 @@ // jsobj.cpp +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "stdafx.h" #include "jsobj.h" #include "../util/goodies.h" @@ -95,6 +111,7 @@ public: JSMatcher::~JSMatcher() { for( int i = 0; i < nBuilders; i++ ) delete builders[i]; + delete in; delete where; } @@ -136,10 +153,18 @@ string Element::toString() { s << '"' << valuestr() << '"'; } break; + case DBRef: + s << fieldName(); + s << " : DBRef('" << valuestr() << "',"; + { + OID *x = (OID *) (valuestr() + valuestrsize()); + s << hex << x->a << x->b << dec << ')'; + } + break; case jstOID: - s << fieldName() << " : ObjId("; - s << hex << oid().a << hex << oid().b << ')'; - break; + s << fieldName() << " : ObjId("; + s << hex << oid().a << oid().b << dec << ')'; + break; default: s << fieldName() << ": ?type=" << type(); break; @@ -147,7 +172,7 @@ string Element::toString() { return s.str(); } -int Element::size() { +int Element::size() const { if( totalSize >= 0 ) return totalSize; @@ -194,7 +219,7 @@ int Element::size() { cout << "Element: bad type " << (int) type() << endl; assert(false); } - totalSize = x + fieldNameSize; + ((Element *) this)->totalSize = x + fieldNameSize; if( !eoo() ) { const char *next = data + totalSize; @@ -214,7 +239,7 @@ int Element::size() { } /* must be same type! */ -inline int compareElementValues(Element& l, Element& r) { +int compareElementValues(const Element& l, const Element& r) { int f; double x; switch( l.type() ) { @@ -289,15 +314,19 @@ int getGtLtOp(Element& e) { Element fe = e.embeddedObject().firstElement(); const char *fn = fe.fieldName(); - if( fn[0] == '$' && fn[1] && fn[2] == 't' ) { - if( fn[1] == 'g' ) { - if( fn[3] == 0 ) op = JSMatcher::GT; - else if( fn[3] == 'e' && fn[4] == 0 ) op = JSMatcher::GTE; - } - else if( fn[1] == 'l' ) { - if( fn[3] == 0 ) op = JSMatcher::LT; - else if( fn[3] == 'e' && fn[4] == 0 ) op = JSMatcher::LTE; + if( fn[0] == '$' && fn[1] ) { + if( fn[2] == 't' ) { + if( fn[1] == 'g' ) { + if( fn[3] == 0 ) op = JSMatcher::GT; + else if( fn[3] == 'e' && fn[4] == 0 ) op = JSMatcher::GTE; + } + else if( fn[1] == 'l' ) { + if( fn[3] == 0 ) op = JSMatcher::LT; + else if( fn[3] == 'e' && fn[4] == 0 ) op = JSMatcher::LTE; + } } + else if( fn[1] == 'i' && fn[2] == 'n' && fn[3] == 0 ) + op = JSMatcher::opIN; } return op; } @@ -305,7 +334,7 @@ int getGtLtOp(Element& e) { #include "pdfile.h" JSMatcher::JSMatcher(JSObj &_jsobj) : - where(0), jsobj(_jsobj), nRegex(0) + in(0), where(0), jsobj(_jsobj), nRegex(0) { nBuilders = 0; @@ -353,33 +382,64 @@ JSMatcher::JSMatcher(JSObj &_jsobj) : } // greater than / less than... - // { a : { $gt: 3 } } + // e.g., e == { a : { $gt : 3 } } + // or + // { a : { $in : [1,2,3] } } if( e.type() == Object ) { - Element fe = e.embeddedObject().firstElement(); - const char *fn = fe.fieldName(); - if( fn[0] == '$' && fn[1] && fn[2] == 't' ) { - int op = Equality; - if( fn[1] == 'g' ) { - if( fn[3] == 0 ) op = GT; - else if( fn[3] == 'e' && fn[4] == 0 ) op = GTE; - } - else if( fn[1] == 'l' ) { - if( fn[3] == 0 ) op = LT; - else if( fn[3] == 'e' && fn[4] == 0 ) op = LTE; + // e.g., fe == { $gt : 3 } + JSElemIter j(e.embeddedObject()); + bool ok = false; + while( j.more() ) { + Element fe = j.next(); + if( fe.eoo() ) + break; + // Element fe = e.embeddedObject().firstElement(); + const char *fn = fe.fieldName(); + if( fn[0] == '$' && fn[1] ) { + if( fn[2] == 't' ) { + int op = Equality; + if( fn[1] == 'g' ) { + if( fn[3] == 0 ) op = GT; + else if( fn[3] == 'e' && fn[4] == 0 ) op = GTE; + } + else if( fn[1] == 'l' ) { + if( fn[3] == 0 ) op = LT; + else if( fn[3] == 'e' && fn[4] == 0 ) op = LTE; + } + if( op && nBuilders < 8) { + JSObjBuilder *b = new JSObjBuilder(); + builders[nBuilders++] = b; + b->appendAs(fe, e.fieldName()); + toMatch.push_back( b->done().firstElement() ); + compareOp.push_back(op); + n++; + ok = true; + } + } + else if( fn[1] == 'i' && fn[2] == 'n' && fn[3] == 0 && fe.type() == Array ) { + // $in + assert( in == 0 ); // only one per query supported so far. finish... + in = new set<Element,element_lt>(); + JSElemIter i(fe.embeddedObject()); + while( i.more() ) + in->insert(i.next()); + toMatch.push_back(e); // not actually used at the moment + compareOp.push_back(opIN); + n++; + ok = true; + } } - if( op && nBuilders < 8) { - JSObjBuilder *b = new JSObjBuilder(); - builders[nBuilders++] = b; - b->appendAs(fe, e.fieldName()); - toMatch.push_back( b->done().firstElement() ); - compareOp.push_back(op); - n++; - continue; + else { + ok = false; + break; } } + if( ok ) + continue; } { + // normal, simple case e.g. { a : "foo" } toMatch.push_back(e); compareOp.push_back(Equality); n++; @@ -391,15 +451,35 @@ inline int JSMatcher::valuesMatch(Element& l, Element& r, int op) { if( op == 0 ) return l.valuesEqual(r); + if( op == opIN ) { + // { $in : [1,2,3] } + return in->count(l); + } + + /* check LT, GTE, ... */ if( l.type() != r.type() ) return false; - int c = compareElementValues(l, r); int z = 1 << (c+1); return (op & z); } -/* return value +/* Check if a particular field matches. + + fieldName - field to match "a.b" if we are reaching into an embedded object. + toMatch - element we want to match. + obj - database object to check against + compareOp - Equality, LT, GT, etc. + deep - out param. set to true/false if we scanned an array + isArr - + + Special forms: + + { "a.b" : 3 } means obj.a.b == 3 + { a : { $lt : 3 } } means obj.a < 3 + { a : { $in : [1,2] } } means [1,2].contains(obj.a) + + return value -1 mismatch 0 missing element 1 match @@ -503,15 +583,6 @@ bool JSMatcher::matches(JSObj& jsobj, bool *deep) { /* assuming there is usually only one thing to match. if more this could be slow sometimes. */ - for( int r = 0; r < nRegex; r++ ) { - RegexMatcher& rm = regexs[r]; - Element e = jsobj.getFieldDotted(rm.fieldName); - if( e.eoo() ) - return false; - if( !regexMatches(rm, e, deep) ) - return false; - } - // check normal non-regex cases: for( int i = 0; i < n; i++ ) { Element& m = toMatch[i]; @@ -524,35 +595,14 @@ bool JSMatcher::matches(JSObj& jsobj, bool *deep) { return false; } -/* - Element e = jsobj.getFieldDotted(m.fieldName(), arrayElName); - if( !e.eoo() ) { - if( valuesMatch(e, m, compareOp[i]) ) { - goto ok; - } - else if( e.type() == Array ) { - JSElemIter ai(e.embeddedObject()); - while( ai.more() ) { - Element z = ai.next(); - if( valuesMatch( z, m, compareOp[i]) ) { - if( deep ) - *deep = true; - goto ok; - } - } - } + for( int r = 0; r < nRegex; r++ ) { + RegexMatcher& rm = regexs[r]; + Element e = jsobj.getFieldDotted(rm.fieldName); + if( e.eoo() ) return false; - } -*/ - - /* missing. that is ok iff we were looking for null */ -// if( m.type() == jstNULL || m.type() == Undefined ) -// ; -////// else -// return false; -//ok: -// ; -// } + if( !regexMatches(rm, e, deep) ) + return false; + } if( where ) { if( where->func == 0 ) @@ -829,4 +879,3 @@ struct RXTest { assert( part.PartialMatch("dwight") ); } } rxtest; - diff --git a/db/jsobj.h b/db/jsobj.h index 1620298ec0b..135db6e4319 100644 --- a/db/jsobj.h +++ b/db/jsobj.h @@ -1,5 +1,21 @@ // jsobj.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #include "../stdafx.h" @@ -45,20 +61,19 @@ struct OID { totalSize includes itself. Data: - Bool: <byte> - EOO: nothing follows + Bool: <byte> + EOO: nothing follows Undefined: nothing follows - OID: an OID object - Number: <double> - String: <unsigned32 strsizewithnull><cstring> - Date: <8bytes> - Regex: <cstring regex><cstring options> - Object: a nested object, leading with its entire size, which terminates with EOO. - Array: same as object - BinData: - <int len> - <byte subtype> - <byte[len] data> + OID: an OID object + Number: <double> + String: <unsigned32 strsizewithnull><cstring> + Date: <8bytes> + Regex: <cstring regex><cstring options> + Object: a nested object, leading with its entire size, which terminates with EOO. + Array: same as object + DBRef: <strlen> <cstring ns> <oid> + DBRef is a database reference: basically a collection name plus an Object ID + BinData: <int len> <byte subtype> <byte[len] data> */ /* db operation message format @@ -88,9 +103,9 @@ class Element { friend class JSObj; public: string toString(); - JSType type() { return (JSType) *data; } - bool eoo() { return type() == EOO; } - int size(); + JSType type() const { return (JSType) *data; } + bool eoo() const { return type() == EOO; } + int size() const; // wrap this element up as a singleton object. JSObj wrap(); @@ -102,26 +117,27 @@ public: // raw data be careful: const char * value() const { return (data + fieldNameSize + 1); } - int valuesize() { return size() - fieldNameSize - 1; } + int valuesize() const { return size() - fieldNameSize - 1; } bool boolean() { return *value() ? true : false; } - unsigned long long date() { return *((unsigned long long*) value()); } + unsigned long long date() const { return *((unsigned long long*) value()); } double& number() { return *((double *) value()); } + double number() const { return *((double *) value()); } OID& oid() { return *((OID*) value()); } // for strings - int valuestrsize() { + int valuestrsize() const { return *((int *) value()); } // for objects the size *includes* the size of the size field - int objsize() { + int objsize() const { return *((int *) value()); } // for strings. also gives you start of the real data for an embedded object - const char * valuestr() { return value() + 4; } + const char * valuestr() const { return value() + 4; } JSObj embeddedObject(); @@ -163,14 +179,24 @@ private: } const char *data; int fieldNameSize; - int totalSize; + int totalSize; /* caches the computed size */ }; +/* l and r MUST have same type when called: check that first. */ +int compareElementValues(const Element& l, const Element& r); + class JSObj { friend class JSElemIter; class Details { public: - ~Details() { _objdata = 0; } + ~Details() { + // note refCount means two different things (thus the assert here) + assert(refCount <= 0); + if (owned()) { + free((void *)_objdata); + } + _objdata = 0; + } const char *_objdata; int _objsize; int refCount; // -1 == don't free @@ -180,6 +206,8 @@ class JSObj { details = new Details(); details->_objdata = data; details->_objsize = *((int*) data); + assert( details->_objsize > 0 ); + assert( details->_objsize <= 1024 * 1024 * 16 ); details->refCount = ifree ? 1 : -1; } public: @@ -224,8 +252,8 @@ public: JSObj extractFields(JSObj pattern, JSObjBuilder& b); const char *objdata() const { return details->_objdata; } - int objsize() const { return details->_objsize; } // includes the embedded size field - bool isEmpty() const { return details == 0 || objsize() <= 5; } + int objsize() const { return details ? details->_objsize : 0; } // includes the embedded size field + bool isEmpty() const { return objsize() <= 5; } /* this is broken if elements aren't in the same order. */ bool operator<(const JSObj& r) const { return woCompare(r) < 0; } @@ -235,7 +263,9 @@ public: */ int woCompare(const JSObj& r) const; bool woEqual(const JSObj& r) const { - return objsize()==r.objsize() && memcmp(objdata(),r.objdata(),objsize())==0; + int os = objsize(); + return os == r.objsize() && + (os == 0 || memcmp(objdata(),r.objdata(),os)==0); } bool operator==(const JSObj& r) const { return this->woEqual(r); @@ -384,6 +414,11 @@ private: BufBuilder b; }; +/* iterator for a JSObj + + Note each JSObj ends with an EOO element: so you will get more() on an empty + object, although next().eoo() will be true. +*/ class JSElemIter { public: JSElemIter(const JSObj& jso) { @@ -422,8 +457,9 @@ class Where; { a : 3 } is the pattern object. GT/LT: - { a : { $gt + { a : { $gt : 3 } } + TODO: we should rewrite the matcher to be more an AST style. */ class JSMatcher : boost::noncopyable { int matchesDotted( @@ -431,13 +467,24 @@ class JSMatcher : boost::noncopyable { Element& toMatch, JSObj& obj, int compareOp, bool *deep, bool isArr = false); + struct element_lt + { + bool operator()(const Element& l, const Element& r) const + { + int x = (int) l.type() - (int) r.type(); + if( x < 0 ) return true; + if( x > 0 ) return false; + return compareElementValues(l,r) < 0; + } + }; public: enum { Equality = 0, LT = 0x1, LTE = 0x3, GTE = 0x6, - GT = 0x4 + GT = 0x4, + opIN = 0x8 // { x : { $in : [1,2,3] } } }; static int opDirection(int op) { @@ -456,6 +503,7 @@ public: private: int valuesMatch(Element& l, Element& r, int op); + set<Element,element_lt> *in; Where *where; JSObj& jsobj; vector<Element> toMatch; diff --git a/db/json.cpp b/db/json.cpp index 6c875ab95d4..cdf72b508c4 100644 --- a/db/json.cpp +++ b/db/json.cpp @@ -1,5 +1,21 @@ // json.cpp +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "stdafx.h" #include "json.h" #include "../util/builder.h" diff --git a/db/json.h b/db/json.h index febee18940e..05cf415fdb5 100644 --- a/db/json.h +++ b/db/json.h @@ -1,5 +1,21 @@ // json.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #include "../stdafx.h" diff --git a/db/makefile b/db/makefile index f8ea9ab2f4b..59fec8d8404 100644 --- a/db/makefile +++ b/db/makefile @@ -1,9 +1,10 @@ # makefile for our db project -FLAGS= ${CFLAGS} -fPIC -ggdb -pthread -O3 -I .. -Isrc/p -I/src/p/db -L/usr/local/lib -L/usr/lib -rdynamic +FLAGS= ${CFLAGS} -fPIC -ggdb -pthread -O3 -I .. -Isrc/p -I/src/p/db -L/usr/local/lib -L/usr/lib -rdynamic LIB_DEPS = -lpcrecpp -lpcre LIB_BOOST = -lboost_thread -lboost_filesystem + LIBS= $(LIB_DEPS) $(LIB_BOOST) -lstdc++ JVM_LIBS = -L/opt/java/lib/ diff --git a/db/makefile.Darwin.i386 b/db/makefile.Darwin.i386 index 87a4018152e..b2dffcafaf4 100644 --- a/db/makefile.Darwin.i386 +++ b/db/makefile.Darwin.i386 @@ -1,4 +1,4 @@ -FLAGS += -I/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Headers/ -I/sw/include -L/sw/lib +FLAGS += -I/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Headers/ -I/sw/include -L/sw/lib -mmacosx-version-min=10.4 JVM_LIBS += -L/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Libraries/ -framework JavaVM diff --git a/db/minilex.h b/db/minilex.h index b63228f4822..637778441fc 100644 --- a/db/minilex.h +++ b/db/minilex.h @@ -1,6 +1,22 @@ // minilex.h // mini js lexical analyzer. idea is to be dumb and fast. +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + struct MiniLex { strhashmap reserved; bool ic[256]; // ic=Identifier Character @@ -93,5 +109,3 @@ struct MiniLex { reserved["with "] = true; } }; - - diff --git a/db/namespace.h b/db/namespace.h index d70ae6db3af..0469e128f5d 100644 --- a/db/namespace.h +++ b/db/namespace.h @@ -1,5 +1,21 @@ // namespace.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #include "../util/hashtab.h" @@ -77,6 +93,19 @@ public: return s; } + string indexName() { // e.g. "ts_1" + JSObj io = info.obj(); + return io.getStringField("name"); + } + + /* gets not our namespace name (indexNamespace for that), + but the collection we index, its name. + */ + string parentNS() { + JSObj io = info.obj(); + return io.getStringField("ns"); + } + /* delete this index. does NOT celan up the system catalog (system.indexes or system.namespaces) -- only NamespaceIndex. */ diff --git a/db/objwrappers.h b/db/objwrappers.h index 8868e9e7d4e..549b03dd25c 100644 --- a/db/objwrappers.h +++ b/db/objwrappers.h @@ -1,9 +1,22 @@ // objwrappers.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #include "../stdafx.h" #include "jsobj.h" - - - diff --git a/db/pdfile.cpp b/db/pdfile.cpp index 6563e41ae77..f9fc1250440 100644 --- a/db/pdfile.cpp +++ b/db/pdfile.cpp @@ -1,12 +1,27 @@ // pdfile.cpp +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + /* todo: -_ manage deleted records. bucket? -_ use deleted on inserts! -_ quantize allocations _ table scans must be sequential, not next/prev pointers -_ regex support +_ coalesce deleted + +_ disallow system* manipulations from the client. */ #include "stdafx.h" @@ -18,6 +33,7 @@ _ regex support #include "btree.h" #include <algorithm> #include <list> +#include "query.h" const char *dbpath = "/data/db/"; @@ -428,7 +444,7 @@ bool userCreateNS(const char *ns, JSObj& j, string& err) { void PhysicalDataFile::open(int fn, const char *filename) { int length; - + if( fn <= 4 ) { length = (64*1024*1024) << fn; if( strstr(filename, "alleyinsider") && length < 1024 * 1024 * 1024 ) { @@ -438,8 +454,19 @@ void PhysicalDataFile::open(int fn, const char *filename) { } } else length = 0x7ff00000; + + assert( length >= 64*1024*1024 ); - assert( length >= 64*1024*1024 && length % 4096 == 0 ); + if( strstr(filename, "_hudsonSmall") ) { + int mult = 1; + if ( fn > 1 && fn < 1000 ) + mult = fn; + length = 1024 * 512 * mult; + cout << "Warning : using small files for _hudsonSmall" << endl; + } + + + assert( length % 4096 == 0 ); assert(fn == fileNo); header = (PDFHeader *) mmf.map(filename, length); @@ -628,12 +655,41 @@ auto_ptr<Cursor> findTableScan(const char *ns, JSObj& order) { void aboutToDelete(const DiskLoc& dl); -/* delete this index. does NOT celan up the system catalog +/* drop a collection/namespace */ +void dropNS(string& nsToDrop) { + assert( strstr(nsToDrop.c_str(), ".system.") == 0 ); + { + // remove from the system catalog + JSObjBuilder b; + b.append("name", nsToDrop.c_str()); + JSObj cond = b.done(); // { name: "colltodropname" } + string system_namespaces = client->name + ".system.namespaces"; + int n = deleteObjects(system_namespaces.c_str(), cond, false, true); + wassert( n == 1 ); + } + // remove from the catalog hashtable + client->namespaceIndex.kill(nsToDrop.c_str()); +} + +/* delete this index. does NOT clean up the system catalog (system.indexes or system.namespaces) -- only NamespaceIndex. */ void IndexDetails::kill() { - string ns = indexNamespace(); - client->namespaceIndex.kill(ns.c_str()); + string ns = indexNamespace(); // e.g. foo.coll.$ts_1 + + { + // clean up in system.indexes + JSObjBuilder b; + b.append("name", indexName().c_str()); + b.append("ns", parentNS().c_str()); + JSObj cond = b.done(); // e.g.: { name: "ts_1", ns: "foo.coll" } + string system_indexes = client->name + ".system.indexes"; + int n = deleteObjects(system_indexes.c_str(), cond, false, true); + wassert( n == 1 ); + } + + dropNS(ns); + // client->namespaceIndex.kill(ns.c_str()); head.setInvalid(); info.setInvalid(); } @@ -645,6 +701,11 @@ void IndexDetails::kill() { */ void IndexDetails::getKeysFromObject(JSObj& obj, set<JSObj>& keys) { JSObj keyPattern = info.obj().getObjectField("key"); + if( keyPattern.objsize() == 0 ) { + cout << keyPattern.toString() << endl; + cout << info.obj().toString() << endl; + assert(false); + } JSObjBuilder b; JSObj key = obj.extractFields(keyPattern, b); if( key.isEmpty() ) @@ -717,8 +778,6 @@ void DataFileMgr::deleteRecord(const char *ns, Record *todelete, const DiskLoc& { dassert( todelete == dl.rec() ); - int tempextofs = todelete->extentOfs; - NamespaceDetails* d = nsdetails(ns); if( d->capped && !cappedOK ) { cout << "failing remove on a capped ns " << ns << endl; @@ -759,18 +818,18 @@ void DataFileMgr::deleteRecord(const char *ns, Record *todelete, const DiskLoc& { d->nrecords--; d->datasize -= todelete->netLength(); -/// DEBUGGING << "temp: dddelrec deleterecord " << ns << endl; -// if( todelete->extentOfs == 0xaca500 ) { -// cout << "break\n"; -// } -/* -TEMP: add deleted rec 0:aca5b0 aca500 -temp: adddelrec deleterecord admin.blog.posts -TEMP: add deleted rec 0:b9e750 b6a500 -temp: adddelrec deleterecord admin.blog.posts -*/ - - d->addDeletedRec((DeletedRecord*)todelete, dl); + /* temp: if in system.indexes, don't reuse, and zero out: we want to be + careful until validated more, as IndexDetails has pointers + to this disk location. so an incorrectly done remove would cause + a lot of problems. + */ + if( strstr(ns, ".system.indexes") ) { + memset(todelete, 0, todelete->lengthWithHeaders); + } + else { + DEV memset(todelete->data, 0, todelete->netLength()); // attempt to notice invalid reuse. + d->addDeletedRec((DeletedRecord*)todelete, dl); + } } } @@ -962,8 +1021,9 @@ DiskLoc DataFileMgr::insert(const char *ns, const void *buf, int len, bool god) tabletoidxns = io.getStringField("ns"); // table it indexes JSObj key = io.getObjectField("key"); if( name == 0 || *name == 0 || tabletoidxns == 0 || key.isEmpty() || key.objsize() > 2048 ) { - cout << "user warning: bad add index attempt name:" << (name?name:"") << " ns:" << - (tabletoidxns?tabletoidxns:"") << endl; + cout << "user warning: bad add index attempt name:" << (name?name:"") << "\n ns:" << + (tabletoidxns?tabletoidxns:"") << "\n ourns:" << ns; + cout << "\n idxobj:" << io.toString() << endl; return DiskLoc(); } tableToIndex = nsdetails(tabletoidxns); @@ -1062,3 +1122,23 @@ void pdfileInit() { // namespaceIndex.init(dbpath); theDataFileMgr.init(dbpath); } + +#include "clientcursor.h" + +void dropDatabase(const char *ns) { + // ns is of the form "<dbname>.$cmd" + char cl[256]; + nsToClient(ns, cl); + problem() << "dropDatabase " << cl << endl; + assert( client->name == cl ); + + /* important: kill all open cursors on the database */ + string prefix(cl); + prefix += '.'; + ClientCursor::invalidate(prefix.c_str()); + + clients.erase(cl); + delete client; // closes files + client = 0; + _deleteDataFiles(cl); +} diff --git a/db/pdfile.h b/db/pdfile.h index 8aa31ef26aa..89dcd602d33 100644 --- a/db/pdfile.h +++ b/db/pdfile.h @@ -1,3 +1,19 @@ +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + /* pdfile.h Files: @@ -23,6 +39,9 @@ class Extent; class Record; class Cursor; +void dropDatabase(const char *ns); +void dropNS(string& dropNs); + /*---------------------------------------------------------------------*/ class PDFHeader; @@ -447,17 +466,6 @@ inline void _deleteDataFiles(const char *client) { } } -inline void dropDatabase(const char *ns) { - char cl[256]; - nsToClient(ns, cl); - problem() << "dropDatabase " << cl << endl; - assert( client->name == cl ); - clients.erase(cl); - delete client; // closes files - client = 0; - _deleteDataFiles(cl); -} - inline NamespaceIndex* nsindex(const char *ns) { DEV { char buf[256]; diff --git a/db/query.cpp b/db/query.cpp index 42b8a4a4ae6..95aeccdaf70 100644 --- a/db/query.cpp +++ b/db/query.cpp @@ -1,5 +1,21 @@ // query.cpp +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "stdafx.h" #include "query.h" #include "pdfile.h" @@ -106,6 +122,26 @@ auto_ptr<Cursor> getIndexCursor(const char *ns, JSObj& query, JSObj& order, bool // compound keys with GT/LT not supported yet via index. goto fail; } + if( op >= JSMatcher::opIN ) { + // $in does not use an index (at least yet, should when # of elems is tiny) + goto fail; + } + + { + JSElemIter k(e.embeddedObject()); + k.next(); + if( !k.next().eoo() ) { + /* compound query like { $lt : 9, $gt : 2 } + for those our method below won't work. + need more work on "stopOnMiss" in general -- may + be issues with it. so fix this to use index after + that is fixed. + */ + OCCASIONALLY cout << "finish query optimizer for lt gt compound\n"; + goto fail; + } + } + int direction = - JSMatcher::opDirection(op); return auto_ptr<Cursor>( new BtreeCursor( d->indexes[i].head, @@ -150,25 +186,29 @@ auto_ptr<Cursor> getIndexCursor(const char *ns, JSObj& query, JSObj& order, bool } fail: - DEV cout << "getIndexCursor fail" << endl; + DEV cout << "getIndexCursor fail " << ns << '\n'; return auto_ptr<Cursor>(); } -void deleteObjects(const char *ns, JSObj pattern, bool justOne) { - - if( strstr(ns, ".system.") ) { - if( strstr(ns, ".system.namespaces") ){ +/* ns: namespace, e.g. <client>.<collection> + pattern: the "where" clause / criteria + justOne: stop after 1 match +*/ +int deleteObjects(const char *ns, JSObj pattern, bool justOne, bool god) { + if( strstr(ns, ".system.") && !god ) { + /*if( strstr(ns, ".system.namespaces") ){ cout << "info: delete on system namespace " << ns << '\n'; } else if( strstr(ns, ".system.indexes") ) { cout << "info: delete on system namespace " << ns << '\n'; } - else { + else*/ { cout << "ERROR: attempt to delete in system namespace " << ns << endl; - return; + return -1; } } + int nDeleted = 0; JSMatcher matcher(pattern); JSObj order; auto_ptr<Cursor> c = getIndexCursor(ns, pattern, order); @@ -196,17 +236,20 @@ void deleteObjects(const char *ns, JSObj pattern, bool justOne) { } else { assert( !deep || !c->getsetdup(rloc) ); // can't be a dup, we deleted it! -// cout << " found match to delete" << endl; if( !justOne ) c->noteLocation(); -_tempDelLoc = rloc; + _tempDelLoc = rloc; + theDataFileMgr.deleteRecord(ns, r, rloc); + nDeleted++; tempd = temp; if( justOne ) - return; + break; c->checkLocation(); } } + + return nDeleted; } struct Mod { @@ -427,9 +470,11 @@ string validateNS(const char *ns, NamespaceDetails *d) { ndel++; if( loc.questionable() ) { - ss << " ?bad deleted loc: " << loc.toString() << " bucket:" << i << " k:" << k << endl; - valid = false; - break; + if( loc.a() <= 0 || strstr(ns, "hudsonSmall") == 0 ) { + ss << " ?bad deleted loc: " << loc.toString() << " bucket:" << i << " k:" << k << endl; + valid = false; + break; + } } DeletedRecord *d = loc.drec(); @@ -551,6 +596,7 @@ inline bool _runCommands(const char *ns, JSObj& jsobj, stringstream& ss, BufBuil else if( e.type() == Number ) { if( strcmp(e.fieldName(), "dropDatabase") == 0 ) { if( 1 ) { + cout << "dropDatabase " << ns << endl; valid = true; int p = (int) e.number(); if( p != 1 ) { @@ -640,19 +686,30 @@ inline bool _runCommands(const char *ns, JSObj& jsobj, stringstream& ss, BufBuil } else if( strcmp( e.fieldName(), "drop") == 0 ) { valid = true; - string dropNs = us + '.' + e.valuestr(); - NamespaceDetails *d = nsdetails(dropNs.c_str()); - cout << "CMD: drop " << dropNs << endl; + string nsToDrop = us + '.' + e.valuestr(); + NamespaceDetails *d = nsdetails(nsToDrop.c_str()); + cout << "CMD: drop " << nsToDrop << endl; if( d == 0 ) { anObjBuilder.append("errmsg", "ns not found"); } else if( d->nIndexes != 0 ) { + // client is supposed to drop the indexes first anObjBuilder.append("errmsg", "ns has indexes (not permitted on drop)"); } else { ok = true; - anObjBuilder.append("ns", dropNs.c_str()); + anObjBuilder.append("ns", nsToDrop.c_str()); + ClientCursor::invalidate(nsToDrop.c_str()); + dropNS(nsToDrop); + /* + { + JSObjBuilder b; + b.append("name", dropNs.c_str()); + JSObj cond = b.done(); // { name: "colltodropname" } + deleteObjects("system.namespaces", cond, false, true); + } client->namespaceIndex.kill(dropNs.c_str()); + */ } } else if( strcmp( e.fieldName(), "validate") == 0 ) { @@ -679,7 +736,10 @@ inline bool _runCommands(const char *ns, JSObj& jsobj, stringstream& ss, BufBuil if( d ) { Element f = jsobj.findElement("index"); if( !f.eoo() ) { - // delete a specific index + + ClientCursor::invalidate(toDeleteNs.c_str()); + + // delete a specific index or all? if( f.type() == String ) { const char *idxName = f.valuestr(); if( *idxName == '*' && idxName[1] == 0 ) { @@ -709,7 +769,7 @@ inline bool _runCommands(const char *ns, JSObj& jsobj, stringstream& ss, BufBuil for( int i = x; i < d->nIndexes; i++ ) d->indexes[i] = d->indexes[i+1]; ok=true; - cout << " alpha implementation, space not reclaimed" << endl; + cout << " alpha implementation, space not reclaimed\n"; } else { cout << "deleteIndexes: " << idxName << " not found" << endl; } @@ -829,6 +889,7 @@ int runCount(const char *ns, JSObj& cmd, string& err) { QueryResult* runQuery(Message& message, const char *ns, int ntoskip, int _ntoreturn, JSObj jsobj, auto_ptr< set<string> > filter, stringstream& ss) { + time_t t = time(0); bool wantMore = true; int ntoreturn = _ntoreturn; if( _ntoreturn < 0 ) { @@ -836,10 +897,6 @@ QueryResult* runQuery(Message& message, const char *ns, int ntoskip, int _ntoret wantMore = false; } ss << "query " << ns << " ntoreturn:" << ntoreturn; - if( ntoskip ) - ss << " ntoskip:" << ntoskip; - if( client->profile ) - ss << "<br>query: " << jsobj.toString() << ' '; int n = 0; BufBuilder b(32768); @@ -859,6 +916,17 @@ QueryResult* runQuery(Message& message, const char *ns, int ntoskip, int _ntoret if( query.isEmpty() && order.isEmpty() ) query = jsobj; + /* The ElemIter will not be happy if this isn't really an object. So throw exception + here when that is true. + (Which may indicate bad data from appserver?) + */ + if( query.objsize() == 0 ) { + cout << "Bad query object?\n jsobj:"; + cout << jsobj.toString() << "\n query:"; + cout << query.toString() << endl; + assert(false); + } + auto_ptr<JSMatcher> matcher(new JSMatcher(query)); JSMatcher &debug1 = *matcher; assert( debug1.getN() < 5000 ); @@ -963,6 +1031,11 @@ assert( debug.getN() < 5000 ); qr->nReturned = n; b.decouple(); + if( (client && client->profile) || time(0)-t > 5 ) { + if( ntoskip ) + ss << " ntoskip:" << ntoskip; + ss << " <br>query: " << jsobj.toString() << ' '; + } ss << " nreturned:" << n; return qr; } @@ -995,6 +1068,7 @@ QueryResult* getMore(const char *ns, int ntoreturn, long long cursorid) { if( !cc ) { DEV cout << "getMore: cursorid not found " << ns << " " << cursorid << endl; + cursorid = 0; } else { start = cc->pos; diff --git a/db/query.h b/db/query.h index aa06fa04105..562a2ee4e39 100644 --- a/db/query.h +++ b/db/query.h @@ -1,5 +1,21 @@ // query.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #include "../stdafx.h" @@ -73,7 +89,6 @@ QueryResult* runQuery(Message&, const char *ns, int ntoskip, int ntoreturn, stringstream&); void updateObjects(const char *ns, JSObj updateobj, JSObj pattern, bool upsert, stringstream& ss); -void deleteObjects(const char *ns, JSObj pattern, bool justOne); +int deleteObjects(const char *ns, JSObj pattern, bool justOne, bool god=false); #include "clientcursor.h" - diff --git a/db/resource.h b/db/resource.h index 3041137c21a..979c5302b4f 100644 --- a/db/resource.h +++ b/db/resource.h @@ -2,6 +2,22 @@ // Microsoft Visual C++ generated include file. // Used by db.rc +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + // Next default values for new objects // #ifdef APSTUDIO_INVOKED diff --git a/db/send b/db/send Binary files differdeleted file mode 100644 index fe6a0899528..00000000000 --- a/db/send +++ /dev/null diff --git a/db/storage.h b/db/storage.h index c11a548f103..e1eea4cf287 100644 --- a/db/storage.h +++ b/db/storage.h @@ -1,3 +1,19 @@ +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + /* storage.h Storage subsystem management. diff --git a/db/tests.cpp b/db/tests.cpp index 1fac4d76573..f9d436b10fd 100644 --- a/db/tests.cpp +++ b/db/tests.cpp @@ -1,3 +1,19 @@ +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + /* tests.cpp unit test & such diff --git a/grid/message.cpp b/grid/message.cpp index d099ded693e..ccda8f647d7 100644 --- a/grid/message.cpp +++ b/grid/message.cpp @@ -1,3 +1,19 @@ +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + /* message todo: authenticate; encrypt? diff --git a/grid/message.h b/grid/message.h index 1f4df0d83bd..31b326cdb2d 100644 --- a/grid/message.h +++ b/grid/message.h @@ -1,5 +1,21 @@ // message.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #include "../util/sock.h" diff --git a/grid/protocol.h b/grid/protocol.h index e7d0298ec9b..8d652f476d3 100644 --- a/grid/protocol.h +++ b/grid/protocol.h @@ -1,5 +1,21 @@ // protocol.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once NOT USED diff --git a/grid/protoimpl.h b/grid/protoimpl.h index 9ca5277e527..8db57c05935 100644 --- a/grid/protoimpl.h +++ b/grid/protoimpl.h @@ -1,5 +1,21 @@ // protoimpl.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once /* packet dumping level of detail. */ diff --git a/grid/protorecv.cpp b/grid/protorecv.cpp index 49200d8ab56..9fa86021bd5 100644 --- a/grid/protorecv.cpp +++ b/grid/protorecv.cpp @@ -1,5 +1,21 @@ // protorecv.cpp +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "stdafx.h" #include "protocol.h" #include "boost/thread.hpp" diff --git a/grid/protosend.cpp b/grid/protosend.cpp index dfd31036127..88300928fd3 100644 --- a/grid/protosend.cpp +++ b/grid/protosend.cpp @@ -1,5 +1,21 @@ // protosend.cpp +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + /* todo: redo locking! */ #include "stdafx.h" diff --git a/stdafx.cpp b/stdafx.cpp index fc986b5f6dc..7d461b2e3cb 100644 --- a/stdafx.cpp +++ b/stdafx.cpp @@ -2,6 +2,22 @@ // db.pch will be the pre-compiled header // stdafx.obj will contain the pre-compiled type information +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "stdafx.h" // TODO: reference any additional headers you need in STDAFX.H @@ -3,6 +3,22 @@ // are changed infrequently // +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #if defined(_WIN32) @@ -11,6 +27,25 @@ const bool debug=true; const bool debug=false; #endif +#include <memory> + +extern void dbexit(int returnCode, const char *whyMsg = ""); + +inline void * ourmalloc(size_t size) { + void *x = malloc(size); + if( x == 0 ) dbexit(42, "malloc fails"); + return x; +} + +inline void * ourrealloc(void *ptr, size_t size) { + void *x = realloc(ptr, size); + if( x == 0 ) dbexit(43, "realloc fails"); + return x; +} + +#define malloc ourmalloc +#define realloc ourrealloc + #include "targetver.h" //#include "assert.h" @@ -104,7 +139,7 @@ extern const char *curNs; use this to log things just there. */ #if defined(_WIN32) -#define DEV if( 1 ) +#define DEV if( 0 ) #else #define DEV if( 0 ) #endif @@ -126,7 +161,6 @@ inline void our_debug_free(void *p) { #define free our_debug_free #endif -void dbexit(int resultcode); #define exit dbexit #undef yassert @@ -139,4 +173,3 @@ using namespace boost::filesystem; #include "util/goodies.h" #include "util/log.h" - diff --git a/targetver.h b/targetver.h index 11ff16bf5aa..8f994e4c608 100644 --- a/targetver.h +++ b/targetver.h @@ -1,5 +1,20 @@ +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #ifndef _WIN32_WINNT // Allow use of features specific to Windows Vista or later. #define _WIN32_WINNT 0x0600 // Change this to the appropriate value to target other versions of Windows. #endif - diff --git a/util/builder.h b/util/builder.h index 67b8c38d87f..d65c6d63196 100644 --- a/util/builder.h +++ b/util/builder.h @@ -1,3 +1,19 @@ +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + /* builder.h */ diff --git a/util/goodies.h b/util/goodies.h index 8bb25f31afa..43139d9c4a9 100644 --- a/util/goodies.h +++ b/util/goodies.h @@ -1,6 +1,22 @@ // goodies.h // miscellaneous junk +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #include "../stdafx.h" diff --git a/util/hashtab.h b/util/hashtab.h index ed3bde0c251..6b94050f492 100644 --- a/util/hashtab.h +++ b/util/hashtab.h @@ -1,3 +1,19 @@ +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + /* hashtab.h Simple, fixed size hash table. Darn simple. @@ -110,4 +126,3 @@ public: }; #pragma pack(pop) - diff --git a/util/log.h b/util/log.h index d4496de02c5..286d956ebb3 100644 --- a/util/log.h +++ b/util/log.h @@ -1,5 +1,21 @@ // log.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once class Nullstream { @@ -53,4 +69,3 @@ extern Logstream logstream; inline Logstream& problem() { return logstream.prolog(); } #define cout logstream - diff --git a/util/lruishmap.h b/util/lruishmap.h index eca27609897..f8e39e572f5 100644 --- a/util/lruishmap.h +++ b/util/lruishmap.h @@ -1,5 +1,21 @@ // lru-ish map.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #include "../stdafx.h" diff --git a/util/mmap.cpp b/util/mmap.cpp index 7a16b7f6a70..94f96d6fa59 100644 --- a/util/mmap.cpp +++ b/util/mmap.cpp @@ -1,5 +1,21 @@ // mmap.cpp +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "stdafx.h" #include "mmap.h" @@ -172,4 +188,3 @@ void MemoryMappedFile::flush(bool sync) { } #endif - diff --git a/util/mmap.h b/util/mmap.h index 6b8b8926373..1774840da8b 100644 --- a/util/mmap.h +++ b/util/mmap.h @@ -1,5 +1,21 @@ // mmap.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once class MemoryMappedFile { diff --git a/util/sock.cpp b/util/sock.cpp index 1b86edd4f5e..9c22fa58139 100644 --- a/util/sock.cpp +++ b/util/sock.cpp @@ -1,5 +1,21 @@ // sock.cpp +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "stdafx.h" #include "sock.h" diff --git a/util/sock.h b/util/sock.h index 3c00fe6a83f..d558c3a5f09 100644 --- a/util/sock.h +++ b/util/sock.h @@ -1,5 +1,21 @@ // sock.h +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #pragma once #include <stdio.h> diff --git a/util/util.cpp b/util/util.cpp index 3edb6f2b436..9c09267ae9c 100644 --- a/util/util.cpp +++ b/util/util.cpp @@ -1,3 +1,19 @@ +/** +* Copyright (C) 2008 10gen Inc. +* +* This program is free software: you can redistribute it and/or modify +* it under the terms of the GNU Affero General Public License, version 3, +* as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU Affero General Public License for more details. +* +* You should have received a copy of the GNU Affero General Public License +* along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + #include "stdafx.h" #include "goodies.h" |