Implements parsing of lambdas in the UnwrappedLineParser.
This introduces the correct line breaks; the formatting of
lambda captures are still incorrect, and the braces are also
still formatted as if they were braced init lists instead of
blocks.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@189818 91177308-0d34-0410-b5e6-96231b3b80d8

With this patch, clang-format can be configured to:
* not indent in namespace at all (former behavior).
* indent in namespace as in other blocks.
* indent only in inner namespaces (as required by WebKit style).

Also fix alignment of access specifiers in WebKit style.

Patch started by Marek Kurdej. Thank you!

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@187540 91177308-0d34-0410-b5e6-96231b3b80d8

Before:
someFunction(OtherParam, BracedList{
                           // comment 1 (Forcing intersting break)
                           param1, param2,
                           // comment 2
                           param3, param4
             });
After:
someFunction(OtherParam, BracedList{
                           // comment 1 (Forcing intersting break)
                           param1, param2,
                           // comment 2
                           param3, param4
                         });

To do so, the UnwrappedLineParser now stores the information about the
kind of brace in the FormatToken.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@185914 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@185823 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@183115 91177308-0d34-0410-b5e6-96231b3b80d8

Gets rid of AnnotatedToken, putting everything into FormatToken.
FormatTokens are created once, and only referenced by pointer.  This
enables multiple future features, like having tokens shared between
multiple UnwrappedLines (while there's still work to do to fully enable
that).

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182859 91177308-0d34-0410-b5e6-96231b3b80d8

The FormatToken is now not copyable any more.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182772 91177308-0d34-0410-b5e6-96231b3b80d8

With this patch, we create all tokens in one go before parsing and pass
an ArrayRef<FormatToken*> to the UnwrappedLineParser. The
UnwrappedLineParser is switched to use pointer-to-token internally.

The UnwrappedLineParser still copies the tokens into the UnwrappedLines.
This will be fixed in an upcoming patch.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182768 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182742 91177308-0d34-0410-b5e6-96231b3b80d8

Unify handling of whitespace when breaking protruding tokens with other
whitespace replacements.

As a side effect, the BreakableToken structure changed significantly:
- have a common base class for single-line breakable tokens, as they are
  much more similar
- revamp handling of multi-line comments; we now calculate the
  information about lines in multi-line comments similar to normal
  tokens, and always issue replacements

As a result, we were able to get rid of special casing of trailing
whitespace deletion for comments in the whitespace manager and the
BreakableToken and fixed bugs related to tab handling and escaped
newlines.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182738 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182732 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
Added stack of preprocessor branching directives, and ignore all tokens
inside #if 0 except for preprocessor directives.

Reviewers: klimek, djasper

Reviewed By: klimek

CC: cfe-commits

Differential Revision: http://llvm-reviews.chandlerc.com/D855

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182658 91177308-0d34-0410-b5e6-96231b3b80d8

Replaces the use of WhitespaceStart + WhitspaceLength.
This made a bug in the formatter obvous where we would incorrectly
calculate the next column.

FIXME: There's a similar bug left regarding TokenLength. We should
probably also move to have a TokenRange instead.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182572 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182569 91177308-0d34-0410-b5e6-96231b3b80d8

Allows formatting of C++11 braced init list constructs, like:
vector<int> v { 1, 2, 3 };
f({ 1, 2 });

This involves some changes of how tokens are handled in the
UnwrappedLineFormatter. Note that we have a plan to evolve the
design of the token flow into one where we create all tokens
up-front and then annotate them in the various layers (as we
currently already have to create all tokens at once anyway, the
current abstraction does not help). Thus, this introduces
FIXMEs towards that goal.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182568 91177308-0d34-0410-b5e6-96231b3b80d8

We only ever implemented one and that one is not actually all that
helpful (e.g. gets incorrectly triggered by macros).

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@181871 91177308-0d34-0410-b5e6-96231b3b80d8

Summary:
Added BreakableLineComment, moved common code from
BreakableBlockComment to newly added BreakableComment. As a side-effect of the
rewrite, found another problem with escaped newlines and had to change
code which removes trailing whitespace from line comments not to break after
this patch.

Reviewers: klimek, djasper

Reviewed By: klimek

CC: cfe-commits

Differential Revision: http://llvm-reviews.chandlerc.com/D682

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@179693 91177308-0d34-0410-b5e6-96231b3b80d8

Previously we'd only detect structural errors on the very first level.
This leads to incorrectly balanced braces not being discovered, and thus
incorrect indentation.

This change fixes the problem by:
- changing the parser to use an error state that can be detected
  anywhere inside the productions, for example if we get an eof on
  SOME_MACRO({ some block <eof>
- previously we'd never break lines when we discovered a structural
  error; now we break even in the case of a structural error if there
  are two unwrapped lines within the same line; thus,
  void f() { while (true) { g(); y(); } }
  will still be re-formatted, even if there's missing braces somewhere
  in the file
- still exclude macro definitions from generating structural error;
  macro definitions are inbalanced snippets

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@179379 91177308-0d34-0410-b5e6-96231b3b80d8

With [] marking the selected range, clang-format invoked on

    [  ]   int a;

Would so far not reformat anything. With this patch, it formats a
line if its leading whitespace is touched.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@176435 91177308-0d34-0410-b5e6-96231b3b80d8

No functional changes. Also removed experimental-warning from all of
clang-format's files, as it is no longer accurate.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@173830 91177308-0d34-0410-b5e6-96231b3b80d8

This gives us the ability to guess better defaults for whether a *
between identifiers is a pointer dereference or binary operator.

Now correctly formats:
void f(a *b);
void f() { f(a * b); }

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@173243 91177308-0d34-0410-b5e6-96231b3b80d8

Changing nextToken() in the UnwrappedLineParser to get the next
non-comment token. This allows us to correctly layout a whole class of
snippets, like:

if /* */(/* */ a /* */) /* */
  f() /* */; /* */
else /* */
  g();

Fixes a bug in the formatter where we would assume there is a previous
non-comment token.
Also adds the indent level of an unwrapped line to the debug output in
the parser.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@173168 91177308-0d34-0410-b5e6-96231b3b80d8

Previously, we would not detect brace initializer lists in return
statements, thus:
 return (a)(b) { 1, 2, 3 };
would put the semicolon onto the next line.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@173017 91177308-0d34-0410-b5e6-96231b3b80d8

Manually fix the order of UnwrappedLineParser.cpp as that one didn't
have its associated header as the first header.

This also uncovered a subtle inclusion order dependency as CLog.h didn't
include LLVM.h to pick up using declarations it relied upon.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@172892 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@172831 91177308-0d34-0410-b5e6-96231b3b80d8

This patch prepares being able to test for and fix more problems (see
FIXME in the test for example).

Previously we would output unwrapped lines for preprocessor directives
at the point where we also parsed the hash token. Since often
projections only terminate (and thus output their own unwrapped line)
after peeking at the next token, this would lead to the formatter seeing
the preprocessor directives out-of-order (slightly earlier). To be able
to correctly identify lines to merge, the formatter needs a well-defined
order of unwrapped lines, which this patch introduces.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@172819 91177308-0d34-0410-b5e6-96231b3b80d8

This makes the tedious fitsIntoLimit() method unnecessary and I can
replace one hack (constructor initializers) by a slightly better hack.

Furthermore, this will enable calculating whether a certain part of a
line fits into the limit for future modifications.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@172604 91177308-0d34-0410-b5e6-96231b3b80d8

It was quite convoluted leading to us accidentally introducing O(N^2)
complexity while copying from UnwrappedLine to AnnotatedLine. We might
still want to improve the datastructure in AnnotatedLine (most
importantly not put them in a vector where they need to be copied on
vector resizing but that will be done as a follow-up.

This fixes most of the regression in llvm.org/PR14959.

No formatting changes intended.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@172602 91177308-0d34-0410-b5e6-96231b3b80d8

This switches to parsing record definitions only if we can clearly
identify them. We're specifically allowing common patterns for
visibility control through macros and attributes, but we cannot
currently fix all instances. This fixes all known bugs we have though.

Before:
static class A f() {
  return g();
} int x;

After:
static class A f() {
  return g();
}
int x;

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@172530 91177308-0d34-0410-b5e6-96231b3b80d8

brought into 'clang' namespace by clang/Basic/LLVM.h


git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@172323 91177308-0d34-0410-b5e6-96231b3b80d8

Now we correctly parse and format:
verifyFormat("struct foo a = { bar };
int n;

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@172229 91177308-0d34-0410-b5e6-96231b3b80d8

Summary: Uses DiagnosticsEngine to output diagnostics.

Reviewers: djasper, klimek

Reviewed By: djasper

CC: cfe-commits

Differential Revision: http://llvm-reviews.chandlerc.com/D278

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@172071 91177308-0d34-0410-b5e6-96231b3b80d8

Previously, we would not indent:
SOME_MACRO({
  int i;
});
correctly. This is fixed by adding the trailing }); to the unwrapped
line starting with SOME_MACRO({, so the formatter can correctly match
the braces and indent accordingly.

Also fixes incorrect parsing of initializer lists, like:
int a[] = { 1 };

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@172058 91177308-0d34-0410-b5e6-96231b3b80d8

Just reuse the @interface code for this. It accepts slightly more than
necessary (@implementation cannot have protocol lists), but that's ok.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@172019 91177308-0d34-0410-b5e6-96231b3b80d8

Pull pieces of the @interface code into reusable methods.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@172001 91177308-0d34-0410-b5e6-96231b3b80d8

Previously:
@interface Foo + (id)init; @end

Now:
@interface Foo
+ (id)init;
@end

Some tweaking remains, but this is a good first step.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@171995 91177308-0d34-0410-b5e6-96231b3b80d8

Previously, we'd always start at indent level 0 after a preprocessor
directive, now we layout the following snippet (column limit 69) as
follows:

functionCallTo(someOtherFunction(
    withSomeParameters, whichInSequence,
    areLongerThanALine(andAnotherCall,
  B
                       withMoreParamters,
                       whichStronglyInfluenceTheLayout),
    andMoreParameters),
               trailing);

Note that the different jumping indent is a different issue that will be
addressed separately.

This is the first step towards handling #ifdef->#else->#endif chains
correctly.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@171974 91177308-0d34-0410-b5e6-96231b3b80d8

This is a first step towards supporting more complex structures such
as #ifs inside unwrapped lines. This patch mostly converts the array-based
UnwrappedLine into a linked-list-based UnwrappedLine. Future changes will
allow multiple children for each Token turning the UnwrappedLine into a
tree.

No functional changes intended.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@171856 91177308-0d34-0410-b5e6-96231b3b80d8

Previous indent:
class A {
}
a;
void f() {
};

With this patch:
class A {
} a;
void f() {
}
;

The patch introduces a production for classes and structs, and parses
the rest of the line to the semicolon after the class scope.
This allowed us to remove a long-standing wart in the parser that would
just much the semicolon after any block.
Due to this suboptimal formating some tests were broken.

Some unrelated formatting tests broke; those hit a bug in the ast
printing, and need to be fixed separately.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@171761 91177308-0d34-0410-b5e6-96231b3b80d8

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@171737 91177308-0d34-0410-b5e6-96231b3b80d8