Merge with CEDET upstream.

* admin/grammars/c.by (expr-binop): Add MOD.
(variablearg): Add 'opt-assign'.
(variablearg, varnamelist): Add default values so that it can be
later expanded into the tag.
(opt-stuff-after-symbol): Rename to 'brackets-after-symbol' and
remove empty match.
(multi-stage-dereference): Adapt to above rename.
(unaryexpression): Use 'symbol' instead of 'namespace-symbol',
since the latter also leads to an empty match at the end which
would make this too greedy.
(variablearg-opt-name): Support parsing of function pointers
inside an argument list.

* semantic/analyze.el
(semantic-analyze-find-tag-sequence-default): Always add scope to
the local miniscope for each type.  Otherwise, structure tags are
not analyzed correctly.  Also, always search the extended
miniscope even when not dealing with types.

* semantic/ctxt.el (semantic-get-local-variables-default): Also
try to parse local variables for buffers which are currently
marked as unparseable.  Otherwise, it is often impossible to
complete local variables.

* semantic/scope.el (semantic-analyze-scoped-types-default): If we
cannot find a type in the typecache, also look into the the types
we already found.  This is necessary since in C++, a 'using
namespace' can be dependend on a previous one.
(semantic-completable-tags-from-type): When creating the list of
completable types, pull in types which are referenced through
'using' statements, and also preserve their filenames.

* semanitc/bovine/c.el (semantic/analyze/refs): Require.
(semantic-analyze-tag-references): New override.  Mainly copied
from the default implementation, but if nothing could be found (or
just the tag itself), drop all namespaces from the scope and
search again.  This is necessary for implementations which are
defined outside of the namespace and only pull those in through
'using' statements.
(semantic-ctxt-scoped-types): Go through all tags around point and
search them for using statements.  In the case for using
statements outside of function scope, append them in the correct
order instead of using 'cons'.  This is important since using
statements may depend on previous ones.
(semantic-expand-c-tag-namelist): Do not try to parse struct
definitions as default values.  The grammar parser seems to return
the point positions slightly differently (as a cons instead of a
list).  Also, set parent for typedefs to 'nil'.  It does not
really make sense to set a parent class for typedefs, and it can
also lead to endless loops when calculating scope.
(semantic-c-reconstitute-token): Change handling of function
pointers; instead of seeing them as variables, handle them as
functions with a 'function-pointer' attribute.  Also, correctly
deal with function pointers as function arguments.
(semantic-c-reconstitute-function-arglist): New function to parse
function pointers inside an argument list.
(semantic-format-tag-name): Use 'function-pointer' attribute
instead of the old 'functionpointer-flag'.
(semantic-cpp-lexer): Use new `semantic-lex-spp-paren-or-list'.

* semantic/bovine/gcc.el (semantic-gcc-setup): Add 'features.h' to
the list of files whose preprocessor symbols are included.  This
pulls in things like __USE_POSIX and similar.

* semantic/format.el (semantic-format-tag-prototype-default):
Display default values if available.

* semantic/analyze/refs.el (semantic-analyze-refs-impl)
(semantic-analyze-refs-proto): Add 'default-value' as ignorable in
call to `semantic-tag-similar-p'.

* semantic/db-mode.el (semanticdb-semantic-init-hook-fcn): Always
set buffer for `semanticdb-current-table'.

* semantic/db.el (semanticdb-table::semanticdb-refresh-table): The
previous change turned up a bug in this method.  Since the current
table now correctly has a buffer set, the first clause in the
`cond' would be taken, but there was a `save-excursion' missing.

* semantic/lex-spp.el (semantic-c-end-of-macro): Declare.
(semantic-lex-spp-token-macro-to-macro-stream): Deal with macros
which open/close a scope.  For this, leave an overlay if we
encounter a single open paren and return a semantic-list in the
lexer.  When this list gets expanded, retrieve the old position
from the overlay.  See the comments in the function for further
details.
(semantic-lex-spp-find-closing-macro): New function to find the
next macro which closes scope (i.e., has a closing paren).
(semantic-lex-spp-replace-or-symbol-or-keyword): Go to end of
closing macro if necessary.
(semantic-lex-spp-paren-or-list): New lexer to specially deal with
parens in macro definitions.

* semantic/decorate/mode.el (semantic-decoration-mode): Do not
decorate available tags immediately but in an idle timer, since
EDE will usually not be activated yet, which will make it
impossible to find project includes.

* semantic/decorate/include.el
(semantic-decoration-on-includes-highlight-default): Remove
'unloaded' from throttle when decorating includes, otherwise all
would be loaded.  Rename 'table' to 'currenttable' to make things
clearer.

* ede/linux.el (cl): Require during compile.

* ede/linux.el (project-linux-build-directory-default)
(project-linux-architecture-default): Add customizable variables.
(ede-linux-project): Add additional slots to track Linux-specific
information (out-of-tree build directory and selected
architecture).
(ede-linux--get-build-directory, ede-linux--get-archs)
(ede-linux--detect-architecture, ede-linux--get-architecture)
(ede-linux--include-path): Added function to detect Linux-specific
information.
(ede-linux-load): Set new Linux-specific information when creating
a project.
(ede-expand-filename-impl): Use new and more accurate include
information.

* semantic/scope.el (semantic-calculate-scope): Return a clone of
the scopecache, so that everyone is working with its own (shallow)
copy.  Otherwise, if one caller is resetting the scope, it would
be reset for all others working with the scope cache as well.

1 files changed, 144 insertions, 10 deletions

diff --git a/lisp/cedet/semantic/lex-spp.el b/lisp/cedet/semantic/lex-spp.el
index 462e520654a..a77ebf98684 100644
--- a/lisp/cedet/semantic/lex-spp.el
+++ b/lisp/cedet/semantic/lex-spp.el
@@ -70,6 +70,8 @@
 (require 'semantic)
 (require 'semantic/lex)
 
+(declare-function semantic-c-end-of-macro "semantic/bovine/c")
+
 ;;; Code:
 (defvar semantic-lex-spp-macro-symbol-obarray nil
   "Table of macro keywords used by the Semantic Preprocessor.
@@ -527,16 +529,54 @@ and what valid VAL values are."
   ;;
   ;; Nested token FOO shows up in the table of macros, and gets replace
   ;; inline.  This is the same as case 2.
+  ;;
+  ;; CASE 5: Macros which open a scope without closing it
+  ;;
+  ;; #define __NAMESPACE_STD namespace std {
+  ;; #define __NAMESPACE_END }
+  ;;  ==>
+  ;; ((NAMESPACE "namespace" 140 . 149)
+  ;;  (symbol "std" 150 . 153)
+  ;;  (open-paren "{" 154 . 155))
+  ;;
+  ;; Note that we get a single 'open-paren' instead of a
+  ;; 'semantic-list', which is because we use
+  ;; 'semantic-lex-spp-paren-or-list' instead of
+  ;; 'semantic-lex-paren-or-list' in our spp-lexer.  To keep things
+  ;; reasonably simple, we assume that such an open scope will always
+  ;; be closed by another macro (see
+  ;; `semantic-lex-spp-find-closing-macro'). We generate a
+  ;; 'semantic-list' to this closing macro, and we leave an overlay
+  ;; which contains information how far we got into the macro's
+  ;; stream (since it might open several scopes).
+
+  (let* ((arglist (semantic-lex-spp-macro-with-args val))
+	 (argalist nil)
+	 (val-tmp nil)
+	 (v nil)
+	 (sppov (semantic-lex-spp-get-overlay beg))
+	 (sppinfo (when sppov (overlay-get sppov 'semantic-spp))))
+
+    ;; First, check if we were already here and left information
+    (when sppinfo
+      ;; Advance in the tokens as far as we got last time
+      (when (numberp (car sppinfo))
+	(while (and val
+		    (>= (car sppinfo) (car (last (car val)))))
+	    (setq val (cdr val))))
+      ;; And push an open paren
+      (semantic-lex-push-token
+       (semantic-lex-token 'open-paren beg (1+ beg) "{"))
+      (setq semantic-lex-current-depth (1+ semantic-lex-current-depth))
+      (unless val
+	;; We reached the end of this macro, so delete overlay
+	(delete-overlay sppov)))
 
-  (let ((arglist (semantic-lex-spp-macro-with-args val))
-	(argalist nil)
-	(val-tmp nil)
-	(v nil)
-	)
     ;; CASE 2: Dealing with the arg list.
-    (when arglist
+    (when (and val arglist)
       ;;  Skip the arg list.
-      (setq val (cdr val))
+      (when (eq (caar val) 'spp-arg-list)
+	(setq val (cdr val)))
 
       ;; Push args into the replacement list.
       (let ((AV argvalues))
@@ -616,7 +656,32 @@ and what valid VAL values are."
 	  (semantic-lex-push-token
 	   (semantic-lex-token (semantic-lex-token-class v) beg end txt))
 	  )
-
+	 ;; CASE 5: Macro which opens a scope
+	 ((eq (semantic-lex-token-class v) 'open-paren)
+	  ;; We assume that the scope will be closed by another macro.
+	  ;; (Everything else would be a terrible idea anyway.)
+	  (let* ((endpoint (semantic-lex-spp-find-closing-macro))
+		 (ov (when endpoint
+		       (or sppov
+			   (make-overlay beg end)))))
+	    (when ov
+	      ;; Generate a semantic-list which spans to the end of
+	      ;; the closing macro
+	      (semantic-lex-push-token
+	       (semantic-lex-token 'semantic-list beg endpoint))
+	      ;; The rest of the current macro's stream will be parsed
+	      ;; next time.
+	      (setq val-tmp nil)
+	      ;; Store our current state were we are in the macro and
+	      ;; the endpoint.
+	      (overlay-put ov 'semantic-spp
+			   (cons (car (last v)) endpoint)))))
+	 ((eq (semantic-lex-token-class v) 'close-paren)
+	  ;; Macro which closes a scope
+	  ;; Just push the close paren, but also decrease depth
+	  (semantic-lex-push-token
+	   (semantic-lex-token 'close-paren beg end txt))
+	  (setq semantic-lex-current-depth (1- semantic-lex-current-depth)))
 	 ;; CASE 1: Just another token in the stream.
 	 (t
 	  ;; Nothing new.
@@ -652,6 +717,37 @@ will return empty string instead.")
 	     txt
 	     ""))
 
+(defun semantic-lex-spp-find-closing-macro ()
+  "Find next macro which closes a scope through a close-paren.
+Returns position with the end of that macro."
+  (let ((macros (semantic-lex-spp-macros))
+	(cmacro-regexp "\\(")
+	(case-fold-search nil))
+    ;; Build a regexp which search for all macros with a closing
+    ;; paren, and search for it.
+    (dolist (cur macros)
+      (let ((stream (symbol-value cur)))
+	(when (and (listp stream) (listp (car stream)))
+	  (while stream
+	    (if (and (eq (caar stream) 'close-paren)
+		     (string= (nth 1 (car stream)) "}"))
+		(setq cmacro-regexp (concat cmacro-regexp (symbol-name cur) "\\|")
+		      stream nil)
+	      (setq stream (cdr-safe stream)))))))
+    (when cmacro-regexp
+      (save-excursion
+	(when (re-search-forward
+	       (concat (substring cmacro-regexp 0 -2) "\\)[^0-9a-zA-Z_]") nil t)
+	  (point))))))
+
+(defun semantic-lex-spp-get-overlay (&optional point)
+  "Return first overlay which has a 'semantic-spp property."
+  (let ((overlays (overlays-at (or point (point)))))
+    (while (and overlays
+		(null (overlay-get (car overlays) 'semantic-spp)))
+      (setq overlays (cdr overlays)))
+    (car-safe overlays)))
+
 ;;; Macro Merging
 ;;
 ;; Used when token streams from different macros include each other.
@@ -824,8 +920,46 @@ STR occurs in the current buffer between BEG and END."
   "\\(\\sw\\|\\s_\\)+"
   (let ((str (match-string 0))
 	(beg (match-beginning 0))
-	(end (match-end 0)))
-    (semantic-lex-spp-analyzer-push-tokens-for-symbol str beg end)))
+	(end (match-end 0))
+	sppov)
+      (semantic-lex-spp-analyzer-push-tokens-for-symbol str beg end)
+      (when (setq sppov (semantic-lex-spp-get-overlay beg))
+	(setq semantic-lex-end-point (cdr (overlay-get sppov 'semantic-spp))))))
+
+(define-lex-regex-analyzer semantic-lex-spp-paren-or-list
+  "Detect open parenthesis.
+Contrary to `semantic-lex-paren-or-list', this will push a single
+open-paren onto the stream if no closing paren can be found.
+This is important for macros which open a scope which is closed
+by another macro."
+  "\\s("
+  (if (or (not semantic-lex-maximum-depth)
+	  (< semantic-lex-current-depth semantic-lex-maximum-depth))
+      (progn
+	(setq semantic-lex-current-depth (1+ semantic-lex-current-depth))
+	(semantic-lex-push-token
+	 (semantic-lex-token
+	  'open-paren (match-beginning 0) (match-end 0))))
+    (save-excursion
+      (let ((start (match-beginning 0))
+	    (end (match-end 0))
+	    (peom (save-excursion (semantic-c-end-of-macro) (point))))
+	(condition-case nil
+	   (progn
+	     ;; This will throw an error if no closing paren can be found.
+	     (forward-list 1)
+	     (when (> (point) peom)
+	       ;; If we have left the macro, this is the wrong closing
+	       ;; paren, so error out as well.
+	       (error ""))
+	     (semantic-lex-push-token
+	      (semantic-lex-token
+	       'semantic-list start (point))))
+	  (error
+	   ;; Only push a single open-paren.
+	   (semantic-lex-push-token
+	    (semantic-lex-token
+	     'open-paren start end))))))))
 
 ;;; ANALYZERS FOR NEW MACROS
 ;;