QDoc: Remove `PureDocParser::m_tokenizer`

`PureCodeParser`, a class whose responsibility is to extract
documentation from non-programming-language files (e.g .qdoc files),
uses a `Tokenizer` instance to lex its processed source files.

A reference to the `Tokenizer` instance is stored in a member variable,
`m_tokenizer`, that is later accessed when the capabilities of the
`Tokenizer` are needed.

The `Tokenizer` instance that is referenced by `m_tokenizer` is created
on the stack every time `PureDocParser::parseSourceFile`, the entry
point to the parsing of an input file, is called.

As the instance will be destroyed after the call to
`PureDocParser::parseSourceFile` ends, `m_tokenizer` will be left
pointing to an invalid memory location, ensuing undefined behavior
should it be accessed outside of the scope of a call to
`PureDocParser::parseSourceFile`.

Since no access to `m_tokenizer` is performed outside this scope, no bug
was produced from it.

`m_tokenizer` only usage is in `PureDocParser::parseQdocComments`, the
method that actually performs the parsing and whose only caller is
`PureDocParser::parseSourceFile`, so that the instance-scope is
unrequired.

Hence, `m_tokenizer` was removed in favor of a smaller scope variable in
`PureCodeParser::processQdocComments`.

The initialization of `m_tokenizer` that was previously executed in
`PureDocParser::parseSourceFile` is now moved to
`PureDocParser::processQdocComments`, in a `tokenizer` variable that is
created in the scope of the method.

The usages of `m_tokenizer`, all local to
`PureDocParser::processQdocComments`, were modified to be usages of
`tokenizer`.

The usages are further modified to use the member of object access operator
instead of the member of pointer access operator due to the now
non-pointer type of `tokenizer`.

Thus, `m_tokenizer` was removed as dead-code.

Change-Id: I5c9e092cc5fd3b67ada8433b88fec2c42e4f6975
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>

2 files changed, 13 insertions, 14 deletions

diff --git a/src/qdoc/puredocparser.cpp b/src/qdoc/puredocparser.cpp
index 93616cb77..09f9fe30f 100644
--- a/src/qdoc/puredocparser.cpp
+++ b/src/qdoc/puredocparser.cpp
@@ -37,17 +37,13 @@ void PureDocParser::parseSourceFile(const Location &location, const QString &fil
         return;
     }
 
-    Location fileLocation(filePath);
-    Tokenizer fileTokenizer(fileLocation, in);
-    m_tokenizer = &fileTokenizer;
-
     /*
       The set of open namespaces is cleared before parsing
       each source file. The word "source" here means cpp file.
      */
     m_qdb->clearOpenNamespaces();
 
-    processQdocComments();
+    processQdocComments(in);
     in.close();
 }
 
@@ -56,19 +52,21 @@ void PureDocParser::parseSourceFile(const Location &location, const QString &fil
   and tree building. It only processes qdoc comments. It skips
   everything else.
  */
-void PureDocParser::processQdocComments()
+void PureDocParser::processQdocComments(QFile& input_file)
 {
+    Tokenizer tokenizer(Location{input_file.fileName()}, input_file);
+
     const QSet<QString> &commands = topicCommands() + metaCommands();
 
-    int token = m_tokenizer->getToken();
+    int token = tokenizer.getToken();
     while (token != Tok_Eoi) {
         if (token == Tok_Doc) {
-            QString comment = m_tokenizer->lexeme(); // returns an entire qdoc comment.
-            Location start_loc(m_tokenizer->location());
-            token = m_tokenizer->getToken();
+            QString comment = tokenizer.lexeme(); // returns an entire qdoc comment.
+            Location start_loc(tokenizer.location());
+            token = tokenizer.getToken();
 
             Doc::trimCStyleComment(start_loc, comment);
-            Location end_loc(m_tokenizer->location());
+            Location end_loc(tokenizer.location());
 
             // Doc constructor parses the comment.
             Doc doc(start_loc, end_loc, comment, commands, topicCommands());
@@ -90,7 +88,7 @@ void PureDocParser::processQdocComments()
             processTopicArgs(doc, topic, nodes, docs);
             processMetaCommands(nodes, docs);
         } else {
-            token = m_tokenizer->getToken();
+            token = tokenizer.getToken();
         }
     }
 }
diff --git a/src/qdoc/puredocparser.h b/src/qdoc/puredocparser.h
index e8da06e25..376836944 100644
--- a/src/qdoc/puredocparser.h
+++ b/src/qdoc/puredocparser.h
@@ -6,6 +6,8 @@
 
 #include "cppcodeparser.h"
 
+#include <QtCore/QFile>
+
 QT_BEGIN_NAMESPACE
 
 class Location;
@@ -20,8 +22,7 @@ public:
     void parseSourceFile(const Location &location, const QString &filePath) override;
 
 private:
-    void processQdocComments();
-    Tokenizer *m_tokenizer { nullptr };
+    void processQdocComments(QFile& input_file);
 };
 
 QT_END_NAMESPACE