This is a fixup for the unit tests from r298278 (originally r298165).

Since the buffer that RawB2 pointed at was later deleted, a new call to
getBuffer may very well return a buffer at the same/old address.  Which is
fine.  Just delete the spurious check.

A Windows bot was occasionally hitting this in practice:
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/7086

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@298414 91177308-0d34-0410-b5e6-96231b3b80d8

This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338).  The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.

Original commit message follows:

----

Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands).  Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly.  Furthermore, the LockFileManager is not
robust in every environment.  Other -cc1 commands can stall until
timeout (after about eight minutes).

This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack.  The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module.  Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout.  Perhaps we should
reconsider blocking at all.

This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.

The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename.  Its ownership is shared by the
CompilerInstance and ModuleManager.

- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.

- When modules fail to validate, they're removed from the cache.

- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.

- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.

Original patch by Manman Ren; most testcases by Adrian Prantl!

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@298278 91177308-0d34-0410-b5e6-96231b3b80d8

This reverts commit r298165, as it broke the ARM builds.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@298185 91177308-0d34-0410-b5e6-96231b3b80d8

Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands).  Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly.  Furthermore, the LockFileManager is not
robust in every environment.  Other -cc1 commands can stall until
timeout (after about eight minutes).

This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack.  The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module.  Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout.  Perhaps we should
reconsider blocking at all.

This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.

The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename.  Its ownership is shared by the
CompilerInstance and ModuleManager.

  - The ModuleManager stores PCMs there that it loads from disk, never
    touching the disk if the cache is hot.

  - When modules fail to validate, they're removed from the cache.

  - When a CompilerInstance is spawned to build a new module, each
    already-loaded PCM is assumed to be valid, and is frozen to avoid
    the use-after-free.

  - Any newly-built module is written directly to the cache to avoid the
    round-trip to the filesystem, making lock files unnecessary for
    correctness.

Original patch by Manman Ren; most testcases by Adrian Prantl!

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@298165 91177308-0d34-0410-b5e6-96231b3b80d8

clang-format treats MSVC `__super` keyword like all other keywords adding
a single space after. This change disables this behavior for `__super`.

Patch originally by jutocz (thanks!).

Differential Revision: https://reviews.llvm.org/D30932

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297936 91177308-0d34-0410-b5e6-96231b3b80d8

Before (even violating the column limit):
  auto Diag =
      diag()
      << aaaaaaaaaaaaaaaa(aaaaaaaaaaaa, aaaaaaaaaaaaa, aaaaaaaaaaaaaaaaaaaaaaaaaa);

After:
  auto Diag = diag() << aaaaaaaaaaaaaaaa(aaaaaaaaaaaa, aaaaaaaaaaaaa,
                                         aaaaaaaaaaaaaaaaaaaaaaaaaa);

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297931 91177308-0d34-0410-b5e6-96231b3b80d8

Add AST matchers for ObjCProtocolDecl, ObjCCategoryDecl, ObjCMethodDecl, ObjCIvarDecl, and ObjCPropertyDecl.

Patch by Dave Lee.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297882 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297759 91177308-0d34-0410-b5e6-96231b3b80d8

This prevents unwanted fallout from r296664. Specifically in proto formatting,
this changed:
  optional Aaaaaaaa aaaaaaaa = 12 [
    (aaa) = aaaa,
    (bbbbbbbbbbbbbbbbbbbbbbbbbb) = {
      aaaaaaaaaaaaaaaaa: true,
      aaaaaaaaaaaaaaaa: true
    }
  ];

Into:
  optional Aaaaaaaa aaaaaaaa = 12 [
    (aaa) = aaaa,
    (bbbbbbbbbbbbbbbbbbbbbbbbbb) =
        {aaaaaaaaaaaaaaaaa: true, aaaaaaaaaaaaaaaa: true}
  ];

Which is considered less readable. Generally, it seems preferable to
format such dict literals as blocks rather than contract them to one
line.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297696 91177308-0d34-0410-b5e6-96231b3b80d8

Modified the tests to accept any iteration order, to run only on Unix, and added
additional error reporting to investigate SystemZ bot issue.

The VFS directory iterator and recursive directory iterator behave differently
from the LLVM counterparts. Once the VFS iterators hit a broken symlink they
immediately abort. The LLVM counterparts don't stat entries unless they have to
descend into the next directory, which allows to recover from this issue by
clearing the error code and skipping to the next entry.

This change adds similar behavior to the VFS iterators. There should be no
change in current behavior in the current CLANG source base, because all
clients have loop exit conditions that also check the error code.

This fixes rdar://problem/30934619.

Differential Revision: https://reviews.llvm.org/D30768

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297693 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
@see is special among JSDoc tags in that it is commonly followed by URLs. The JSDoc spec suggests that users should wrap URLs in an additional {@link url...} tag (@see http://usejsdoc.org/tags-see.html), but this is very commonly violated, with @see being followed by a "naked" URL.

This change special cases all JSDoc lines that contain an @see not to be wrapped to account for that.

Reviewers: djasper

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D30883

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297607 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
Previously clang-format would not break after any !. However in TypeScript, ! can be used as a post fix operator for non-nullability:
    x.foo()!.bar()!;

With this change, clang-format will wrap after the ! if it is likely a post-fix non null operator.

Reviewers: djasper

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D30705

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297606 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
`interface` and `type` are pseudo keywords and cause automatic semicolon
insertion when followed by a line break:

    interface  // gets parsed as a long variable access to "interface"
        VeryLongInterfaceName {

    }

With this change, clang-format not longer wraps after `interface` or `type`.

Reviewers: djasper

Subscribers: cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D30874

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297605 91177308-0d34-0410-b5e6-96231b3b80d8

Still broken on Windows and SystemZ bot ... sorry for the noise.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297533 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297531 91177308-0d34-0410-b5e6-96231b3b80d8

Modified the tests to accept any iteration order.

The VFS directory iterator and recursive directory iterator behave differently
from the LLVM counterparts. Once the VFS iterators hit a broken symlink they
immediately abort. The LLVM counterparts allow to recover from this issue by
clearing the error code and skipping to the next entry.

This change adds the same functionality to the VFS iterators. There should be
no change in current behavior in the current CLANG source base, because all
clients have loop exit conditions that also check the error code.

This fixes rdar://problem/30934619.

Differential Revision: https://reviews.llvm.org/D30768

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297528 91177308-0d34-0410-b5e6-96231b3b80d8

The tests are failing on one of the bots.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297517 91177308-0d34-0410-b5e6-96231b3b80d8

The VFS directory iterator and recursive directory iterator behave differently
from the LLVM counterparts. Once the VFS iterators hit a broken symlink they
immediately abort. The LLVM counterparts allow to recover from this issue by
clearing the error code and skipping to the next entry.

This change adds the same functionality to the VFS iterators. There should be
no change in current behavior in the current CLANG source base, because all
clients have loop exit conditions that also check the error code.

This fixes rdar://problem/30934619.

Differential Revision: https://reviews.llvm.org/D30768

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297510 91177308-0d34-0410-b5e6-96231b3b80d8

Differential Revision: https://reviews.llvm.org/D30487

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297467 91177308-0d34-0410-b5e6-96231b3b80d8

Summary: Just realized the implementation is missing...

Reviewers: klimek

Reviewed By: klimek

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D30735

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297289 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
This patch makes ContinuationIndenter call breakProtrudingToken only if
NoLineBreak and NoLineBreakInOperand is false.

Previously, clang-format required two runs to converge on the following example with 24 columns:
Note that the second operand shouldn't be splitted according to NoLineBreakInOperand, but the
token breaker doesn't take that into account:
```
func(a, "long long long long", c);
```
After first run:
```
func(a, "long long "
        "long long",
         c);
```
After second run, where NoLineBreakInOperand is taken into account:
```
func(a,
     "long long "
     "long long",
     c);
```

With the patch, clang-format now obtains in one run:
```
func(a,
     "long long long"
     "long",
     c);
```
which is a better token split overall.

Reviewers: djasper

Reviewed By: djasper

Subscribers: cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D30575

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297274 91177308-0d34-0410-b5e6-96231b3b80d8

Before:
  void f() { MACRO(A * const a); }

After:
  void f() { MACRO(A *const a); }

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297268 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
This patch enables comment reflowing of lines not matching the comment pragma regex
in multiline comments containing comment pragma lines. Previously, these comments
were dumped without being reindented to the result.

Reviewers: djasper, mprobst

Reviewed By: mprobst

Subscribers: klimek, mprobst, cfe-commits

Differential Revision: https://reviews.llvm.org/D30697

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297261 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297148 91177308-0d34-0410-b5e6-96231b3b80d8

[clang-format] Fixed indent issue when adding a comment at the end of a return type in named function declaration.

Differential Revision: https://reviews.llvm.org/D30646

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297143 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
This patch adds support for namespaces ending in semicolon to the namespace comment fixer.
source:
```
namespace A {
  int i;
  int j;
};
```
clang-format before:
```
namespace A {
  int i;
  int j;
} // namespace A;
```
clang-format after:
```
namespace A {
  int i;
  int j;
}; // namespace A
```

Reviewers: djasper

Reviewed By: djasper

Subscribers: cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D30688

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297140 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
I've included a unit test with a function template containing a variable
of incomplete type. Clang compiles this without errors (the standard
does not require a diagnostic in this case). Without the fix, this case
triggers the crash.

Reviewers: klimek

Reviewed By: klimek

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D30636

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297129 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297034 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
Until now, NamespaceEndCommentFixer was adding missing comments for every run,
which results in multiple end comments for:
```
namespace {
  int i;
  int j;
}
#if A
  int a = 1;
#else
  int a = 2;
#endif
```
result before:

```
namespace {
  int i;
  int j;
}// namespace // namespace
#if A
  int a = 1;
#else
  int a = 2;
#endif
```
result after:
```
namespace {
  int i;
  int j;
}// namespace
#if A
  int a = 1;
#else
  int a = 2;
#endif
```

Reviewers: djasper

Reviewed By: djasper

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D30659

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@297028 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
This patch makes the namespace comment fixer use the number of unwrapped lines
that a namespace spans to detect it that namespace is short, thus not needing
end comments to be added.
This is needed to ensure clang-format is idempotent. Previously, a short namespace
was detected by the original source code lines. This has the effect of requiring two
runs for this example:
```
namespace { class A; }
```
after first run:
```
namespace {
class A;
}
```
after second run:
```
namespace {
class A;
} // namespace
```

Reviewers: djasper

Reviewed By: djasper

Subscribers: cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D30528

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@296736 91177308-0d34-0410-b5e6-96231b3b80d8

Before:
  return {
    a,
    b: 'b', c,
  };

After:
  return {
    a,
    b: 'b',
    c,
  };

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@296664 91177308-0d34-0410-b5e6-96231b3b80d8

Many things were wrong:
- We didn't always allow wrapping after "as", which can be necessary.
- We used to Undestand the identifier after "as" as a start of a name.
- We didn't properly parse the structure of the expression with "as"
  having the precedence of relational operators

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@296659 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@296638 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
This patch enables namespace end comments under a new flag FixNamespaceComments,
which is enabled for the LLVM and Google styles.

Reviewers: djasper

Reviewed By: djasper

Subscribers: cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D30405

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@296632 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
An AtomicChange is used to create and group a set of source edits, e.g.
replacements or header insertions. Edits in an AtomicChange should be related,
e.g. replacements for the same type reference and the corresponding header
insertion/deletion.

An AtomicChange is uniquely identified by a key position and will either be
fully applied or not applied at all. The key position should be the location
of the key syntactical element that is being changed, e.g. the call to a
refactored method.

Next step: add a tool that applies AtomicChange.

Reviewers: klimek, djasper

Reviewed By: klimek

Subscribers: alexshap, cfe-commits, djasper, mgorny

Differential Revision: https://reviews.llvm.org/D27054

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@296616 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@296608 91177308-0d34-0410-b5e6-96231b3b80d8

Those blocks are used if C++ code is SWIG-wrapped (see swig.org) and
usually do not contain C++ code. Also cleanup the implementation of for #if 0
and #if false a bit.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@296605 91177308-0d34-0410-b5e6-96231b3b80d8

Before:
  public enum VeryLongEnum {
    ENUM_WITH_MANY_PARAMETERS("aaaaaaaaaaaaaaaaaaaaaa",
                              "bbbbbbbbbbbbbbbbbbbbbbbbbbb",
                              "ccccccccccccccccccc")
    ,
    SECOND_ENUM("a", "b", "c");

    private VeryLongEnum(String a, String b, String c) {}
  }

After:
  public enum VeryLongEnum {
    ENUM_WITH_MANY_PARAMETERS("aaaaaaaaaaaaaaaaaaaaaa",
                              "bbbbbbbbbbbbbbbbbbbbbbbbbbb",
                              "ccccccccccccccccccc") ,
    SECOND_ENUM("a", "b", "c");

    private VeryLongEnum(String a, String b, String c) {}
  }

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@296499 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
Also limits the blacklisting to only apply when the tag is actually
followed by a parameter in curly braces.

    /** @mods {long.type.must.not.wrap} */

vs

    /** @const this is a long description that may wrap. */

Reviewers: djasper

Subscribers: klimek, krasimir, cfe-commits

Differential Revision: https://reviews.llvm.org/D30452

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@296467 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
This patch adds a NamespaceEndCommentsFixer TokenAnalyzer for clang-format,
which fixes end namespace comments.
It currently supports inserting and updating existing wrong comments.

Example source:
```
namespace A {
int i;
}

namespace B {
int j;
} // namespace A
```

after formatting:
```
namespace A {
int i;
} // namespace A

namespace B {
int j;
} // namespace B
```

Reviewers: klimek, djasper

Reviewed By: djasper

Subscribers: klimek, mgorny

Differential Revision: https://reviews.llvm.org/D30269

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@296341 91177308-0d34-0410-b5e6-96231b3b80d8