git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@217656 91177308-0d34-0410-b5e6-96231b3b80d8

Unique_ptr creation stil needs to be moved earlier at some of the call sites.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@217474 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@216756 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@216715 91177308-0d34-0410-b5e6-96231b3b80d8

Overload SourceManager::overrideFileContents so that unconditionally passing ownership is explicitly done using unique_ptr.

Only those callers who are dynamically passing ownership should need the
3 argument form. Those accepting the default ("do pass ownership")
should do so explicitly with a unique_ptr now.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@216614 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@216585 91177308-0d34-0410-b5e6-96231b3b80d8

Add support for the static analyzer to synthesize function implementations from external model files.

Currently the analyzer lazily models some functions using 'BodyFarm',
which constructs a fake function implementation that the analyzer
can simulate that approximates the semantics of the function when
it is called.  BodyFarm does this by constructing the AST for
such definitions on-the-fly.  One strength of BodyFarm
is that all symbols and types referenced by synthesized function
bodies are contextual adapted to the containing translation unit.
The downside is that these ASTs are hardcoded in Clang's own
source code.

A more scalable model is to allow these models to be defined as source
code in separate "model" files and have the analyzer use those
definitions lazily when a function body is needed.  Among other things,
it will allow more customization of the analyzer for specific APIs
and platforms.

This patch provides the initial infrastructure for this feature.
It extends BodyFarm to use an abstract API 'CodeInjector' that can be
used to synthesize function bodies.  That 'CodeInjector' is
implemented using a new 'ModelInjector' in libFrontend, which lazily
parses a model file and injects the ASTs into the current translation
unit.  

Models are currently found by specifying a 'model-path' as an
analyzer option; if no path is specified the CodeInjector is not
used, thus defaulting to the current behavior in the analyzer.

Models currently contain a single function definition, and can
be found by finding the file <function name>.model.  This is an
initial starting point for something more rich, but it bootstraps
this feature for future evolution.

This patch was contributed by Gábor Horváth as part of his
Google Summer of Code project.

Some notes:

- This introduces the notion of a "model file" into
  FrontendAction and the Preprocessor.  This nomenclature
  is specific to the static analyzer, but possibly could be
  generalized.  Essentially these are sources pulled in
  exogenously from the principal translation.

  Preprocessor gets a 'InitializeForModelFile' and
  'FinalizeForModelFile' which could possibly be hoisted out
  of Preprocessor if Preprocessor exposed a new API to
  change the PragmaHandlers and some other internal pieces.  This
  can be revisited.

  FrontendAction gets a 'isModelParsingAction()' predicate function
  used to allow a new FrontendAction to recycle the Preprocessor
  and ASTContext.  This name could probably be made something
  more general (i.e., not tied to 'model files') at the expense
  of losing the intent of why it exists.  This can be revisited.

- This is a moderate sized patch; it has gone through some amount of
  offline code review.  Most of the changes to the non-analyzer
  parts are fairly small, and would make little sense without
  the analyzer changes.

- Most of the analyzer changes are plumbing, with the interesting
  behavior being introduced by ModelInjector.cpp and
  ModelConsumer.cpp.

- The new functionality introduced by this change is off-by-default.
  It requires an analyzer config option to enable.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@216550 91177308-0d34-0410-b5e6-96231b3b80d8

And in the process, discover that FileManager::removeStatCache had a
double-delete when removing an element from the middle of the list (at
the beginning or the end of the list, there was no problem) and add a
unit test to exercise the code path (which successfully crashed when run
(with modifications to match the old API) without this patch applied)

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@215388 91177308-0d34-0410-b5e6-96231b3b80d8

to destroy one of these.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@213837 91177308-0d34-0410-b5e6-96231b3b80d8

Remove pointless MICache: it only ever contained up to 1 object, and was only
non-empty when recovering from an error. There's no performance or memory win
from maintaining this cache.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@213825 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@211763 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@210448 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@209083 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@209061 91177308-0d34-0410-b5e6-96231b3b80d8

It makes more sense to just overload createFileID().

Gardening only.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@209002 91177308-0d34-0410-b5e6-96231b3b80d8

MacroArgs are owned by TokenLexer, and when a TokenLexer is destroyed, it'll
call its MacroArgs's destroy() method.  destroy() only appends the MacroArg to
Preprocessor's MacroArgCache list, and Preprocessor's destructor then calls
deallocate() on all MacroArgs in that list.  This method then ends up freeing
the MacroArgs's memory.

In a code completion context, Parser::cutOffParsing() gets called when a code
completion token is hit, which changes the type of the current token to
tok::eof.  eof tokens aren't always ConsumeToken()ed, so
Preprocessor::HandleEndOfFile() isn't always called, and that function is
responsible for popping the macro stack.

Due to this, Preprocessor::CurTokenLexer can be non-NULL when
~Preprocessor runs.  It's a unique_ptr, so it ended up being destructed after
~Preprocessor completed, and its MacroArgs thus got added to the freelist after
the code freeing things on the freelist had already completed.  The fix is to
explicitly call reset() before the freelist processing happens.  (See the bug
for more notes.)


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@208438 91177308-0d34-0410-b5e6-96231b3b80d8

Landing this separately to make the previous commits easy to follow at home.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@207826 91177308-0d34-0410-b5e6-96231b3b80d8

The Preprocessor::Initialize() function already offers a clear interface to
achieve this, further reducing the confusing number of states a newly
constructed preprocessor can have.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@207825 91177308-0d34-0410-b5e6-96231b3b80d8

Preprocessor::enableIncrementalProcessing() provides a consistent interface to
enable the feature.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@207824 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@204007 91177308-0d34-0410-b5e6-96231b3b80d8

avoid warning for unused macros.

rdar://15034698

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@203213 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@202006 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@201532 91177308-0d34-0410-b5e6-96231b3b80d8

These features are new in VS 2013 and are necessary in order to layout
std::ostream correctly.  Currently we have an ABI incompatibility when
self-hosting with the 2013 stdlib in our convertible_fwd_ostream wrapper
in gtest.

This change adds another implicit attribute, MSVtorDispAttr, because
implicit attributes are currently the best way to make sure the
information stays on class templates through instantiation.

Reviewers: majnemer

Differential Revision: http://llvm-reviews.chandlerc.com/D2746

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@201274 91177308-0d34-0410-b5e6-96231b3b80d8

just storing a flag indicating if there was one), and include it in the 'end of
module' annotation. No functionality change.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@200573 91177308-0d34-0410-b5e6-96231b3b80d8

encodes the canonical rules for LLVM's style. I noticed this had drifted
quite a bit when cleaning up LLVM, so wanted to clean up Clang as well.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@198686 91177308-0d34-0410-b5e6-96231b3b80d8

module. Use the marker to diagnose cases where we try to transition between
submodules when not at the top level (most likely because a closing brace was
missing at the end of a header file, but is also possible if submodule headers
attempt to do something fundamentally non-modular, like our .def files).


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@195543 91177308-0d34-0410-b5e6-96231b3b80d8

The preprocessor currently recognizes module declarations to load a
module based on seeing the 'import' keyword followed by an
identifier. This sequence is fairly unlikely in C (one would need a
type named 'import'), but is more common in Objective-C (where a
variable named 'import' can cause problems). Since import declarations
currently require a leading '@', recognize that in the preprocessor as
well. Fixes <rdar://problem/15084587>.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@194225 91177308-0d34-0410-b5e6-96231b3b80d8

Before this patch, Lex() would recurse whenever the current lexer changed (e.g.
upon entry into a macro). This patch turns the recursion into a loop: the
various lex routines now don't return a token when the current lexer changes,
and at the top level Preprocessor::Lex() now loops until it finds a token.
Normally, the recursion wouldn't end up being very deep, but the recursion depth
can explode in edge cases like a bunch of consecutive macros which expand to
nothing (like in the testcase test/Preprocessor/macro_expand_empty.c in this
patch).

<rdar://problem/14569770>

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@190980 91177308-0d34-0410-b5e6-96231b3b80d8

This patch was created by Lawrence Crowl and reviewed in:
http://llvm-reviews.chandlerc.com/D963

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@187738 91177308-0d34-0410-b5e6-96231b3b80d8

[Preprocessor] For the MacroExpands preprocessor callback, also pass the MacroArgs object that provides information about
the argument tokens for a function macro.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@181065 91177308-0d34-0410-b5e6-96231b3b80d8

Deserialized MacroInfos were not destroyed and if their SmallVector did heap allocation,
it was leaked.

rdar://13768967

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@180771 91177308-0d34-0410-b5e6-96231b3b80d8

It's not used anymore.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@178106 91177308-0d34-0410-b5e6-96231b3b80d8

[Preprocessor/Modules] Separate the macro directives kinds into their own MacroDirective's subclasses.

For each macro directive (define, undefine, visibility) have a separate object that gets chained
to the macro directive history. This has several benefits:

-No need to mutate a MacroDirective when there is a undefine/visibility directive. Stuff like
 PPMutationListener become unnecessary.
-No need to keep extra source locations for the undef/visibility locations for the define directive object
 (which is the majority of the directives)
-Much easier to hide/unhide a section in the macro directive history.
-Easier to track the effects of the directives across different submodules.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@178037 91177308-0d34-0410-b5e6-96231b3b80d8

full information about the macro (e.g if it was imported and where).

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@175978 91177308-0d34-0410-b5e6-96231b3b80d8

for the data specific to a macro definition (e.g. what the tokens are), and
MacroDirective class which encapsulates the changes to the "macro namespace"
(e.g. the location where the macro name became active, the location where it was undefined, etc.)

(A MacroDirective always points to a MacroInfo object.)

Usually a macro definition (MacroInfo) is where a macro name becomes active (MacroDirective) but
splitting the concepts allows us to better model the effect of modules to the macro namespace
(also as a bonus it allows better modeling of push_macro/pop_macro #pragmas).
Modules can have their own macro history, separate from the local (current translation unit)
macro history; MacroDirectives will be used to model the macro history (changes to macro namespace).

For example, if "@import A;" imports macro FOO, there will be a new local MacroDirective created
to indicate that "FOO" became active at the import location. Module "A" itself will contain another
MacroDirective in its macro history (at the point of the definition of FOO) and both MacroDirectives
will point to the same MacroInfo object.

Introducing the separation of macro concepts is the first part towards better modeling of module macros.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@175585 91177308-0d34-0410-b5e6-96231b3b80d8

a source location came from the predefines buffer.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@174190 91177308-0d34-0410-b5e6-96231b3b80d8

Compilation always sets this explicitly, but creating a preprocessor
manually should still put the 'IsPreprocessedOutput' flag in a valid state.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@174077 91177308-0d34-0410-b5e6-96231b3b80d8

This is required to use them in TableGen.


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@173924 91177308-0d34-0410-b5e6-96231b3b80d8

This is a missing piece for C99 conformance.

This patch handles UCNs by adding a '\\' case to LexTokenInternal and
LexIdentifier -- if we see a backslash, we tentatively try to read in a UCN.
If the UCN is not syntactically well-formed, we fall back to the old
treatment: a backslash followed by an identifier beginning with 'u' (or 'U').

Because the spelling of an identifier with UCNs still has the UCN in it, we
need to convert that to UTF-8 in Preprocessor::LookUpIdentifierInfo.

Of course, valid code that does *not* use UCNs will see only a very minimal
performance hit (checks after each identifier for non-ASCII characters,
checks when converting raw_identifiers to identifiers that they do not
contain UCNs, and checks when getting the spelling of an identifier that it
does not contain a UCN).

This patch also adds basic support for actual UTF-8 in the source. This is
treated almost exactly the same as UCNs except that we consider stray
Unicode characters to be mistakes and offer a fixit to remove them.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@173369 91177308-0d34-0410-b5e6-96231b3b80d8