LanguageExtensions.rst

=================================

Clang supports additional attributes for checking basic locking policies in
multithreaded programs.  Clang currently parses the following list of
attributes, although **the implementation for these annotations is currently in
development.** For more details, see the `GCC implementation
<http://gcc.gnu.org/wiki/ThreadSafetyAnnotation>`_.

``no_thread_safety_analysis``
-----------------------------

Use ``__attribute__((no_thread_safety_analysis))`` on a function declaration to
specify that the thread safety analysis should not be run on that function.
This attribute provides an escape hatch (e.g. for situations when it is
difficult to annotate the locking policy).

``lockable``
------------

Use ``__attribute__((lockable))`` on a class definition to specify that it has
a lockable type (e.g. a Mutex class).  This annotation is primarily used to
check consistency.

``scoped_lockable``
-------------------

Use ``__attribute__((scoped_lockable))`` on a class definition to specify that
it has a "scoped" lockable type.  Objects of this type will acquire the lock
upon construction and release it upon going out of scope.  This annotation is
primarily used to check consistency.

``guarded_var``
---------------

Use ``__attribute__((guarded_var))`` on a variable declaration to specify that
the variable must be accessed while holding some lock.

``pt_guarded_var``
------------------

Use ``__attribute__((pt_guarded_var))`` on a pointer declaration to specify
that the pointer must be dereferenced while holding some lock.

``guarded_by(l)``
-----------------

Use ``__attribute__((guarded_by(l)))`` on a variable declaration to specify
that the variable must be accessed while holding lock ``l``.

``pt_guarded_by(l)``
--------------------

Use ``__attribute__((pt_guarded_by(l)))`` on a pointer declaration to specify
that the pointer must be dereferenced while holding lock ``l``.

``acquired_before(...)``
------------------------

Use ``__attribute__((acquired_before(...)))`` on a declaration of a lockable
variable to specify that the lock must be acquired before all attribute
arguments.  Arguments must be lockable type, and there must be at least one
argument.

``acquired_after(...)``
-----------------------

Use ``__attribute__((acquired_after(...)))`` on a declaration of a lockable
variable to specify that the lock must be acquired after all attribute
arguments.  Arguments must be lockable type, and there must be at least one
argument.

``exclusive_lock_function(...)``
--------------------------------

Use ``__attribute__((exclusive_lock_function(...)))`` on a function declaration
to specify that the function acquires all listed locks exclusively.  This
attribute takes zero or more arguments: either of lockable type or integers
indexing into function parameters of lockable type.  If no arguments are given,
the acquired lock is implicitly ``this`` of the enclosing object.

``shared_lock_function(...)``
-----------------------------

Use ``__attribute__((shared_lock_function(...)))`` on a function declaration to
specify that the function acquires all listed locks, although the locks may be
shared (e.g. read locks).  This attribute takes zero or more arguments: either
of lockable type or integers indexing into function parameters of lockable
type.  If no arguments are given, the acquired lock is implicitly ``this`` of
the enclosing object.

``exclusive_trylock_function(...)``
-----------------------------------

Use ``__attribute__((exclusive_lock_function(...)))`` on a function declaration
to specify that the function will try (without blocking) to acquire all listed
locks exclusively.  This attribute takes one or more arguments.  The first
argument is an integer or boolean value specifying the return value of a
successful lock acquisition.  The remaining arugments are either of lockable
type or integers indexing into function parameters of lockable type.  If only
one argument is given, the acquired lock is implicitly ``this`` of the
enclosing object.

``shared_trylock_function(...)``
--------------------------------

Use ``__attribute__((shared_lock_function(...)))`` on a function declaration to
specify that the function will try (without blocking) to acquire all listed
locks, although the locks may be shared (e.g. read locks).  This attribute
takes one or more arguments.  The first argument is an integer or boolean value
specifying the return value of a successful lock acquisition.  The remaining
arugments are either of lockable type or integers indexing into function
parameters of lockable type.  If only one argument is given, the acquired lock
is implicitly ``this`` of the enclosing object.

``unlock_function(...)``
------------------------

Use ``__attribute__((unlock_function(...)))`` on a function declaration to
specify that the function release all listed locks.  This attribute takes zero
or more arguments: either of lockable type or integers indexing into function
parameters of lockable type.  If no arguments are given, the acquired lock is
implicitly ``this`` of the enclosing object.

``lock_returned(l)``
--------------------

Use ``__attribute__((lock_returned(l)))`` on a function declaration to specify
that the function returns lock ``l`` (``l`` must be of lockable type).  This
annotation is used to aid in resolving lock expressions.

``locks_excluded(...)``
-----------------------

Use ``__attribute__((locks_excluded(...)))`` on a function declaration to
specify that the function must not be called with the listed locks.  Arguments
must be lockable type, and there must be at least one argument.

``exclusive_locks_required(...)``
---------------------------------

Use ``__attribute__((exclusive_locks_required(...)))`` on a function
declaration to specify that the function must be called while holding the
listed exclusive locks.  Arguments must be lockable type, and there must be at
least one argument.

``shared_locks_required(...)``
------------------------------

Use ``__attribute__((shared_locks_required(...)))`` on a function declaration
to specify that the function must be called while holding the listed shared
locks.  Arguments must be lockable type, and there must be at least one
argument.

Consumed Annotation Checking
============================

Clang supports additional attributes for checking basic resource management
properties, specifically for unique objects that have a single owning reference.
The following attributes are currently supported, although **the implementation
for these annotations is currently in development and are subject to change.**

``consumable``
--------------

Each class that uses any of the following annotations must first be marked
using the consumable attribute.  Failure to do so will result in a warning.

``set_typestate(new_state)``
----------------------------

Annotate methods that transition an object into a new state with
``__attribute__((set_typestate(new_state)))``.  The new new state must be
unconsumed, consumed, or unknown.

``callable_when(...)``
----------------------

Use ``__attribute__((callable_when(...)))`` to indicate what states a method
may be called in.  Valid states are unconsumed, consumed, or unknown.  Each
argument to this attribute must be a quoted string.  E.g.:

``__attribute__((callable_when("unconsumed", "unknown")))``

``tests_typestate(tested_state)``
---------------------------------

Use ``__attribute__((tests_typestate(tested_state)))`` to indicate that a method
returns true if the object is in the specified state..

``param_typestate(expected_state)``
-----------------------------------

This attribute specifies expectations about function parameters.  Calls to an
function with annotated parameters will issue a warning if the corresponding
argument isn't in the expected state.  The attribute is also used to set the
initial state of the parameter when analyzing the function's body.

``return_typestate(ret_state)``
-------------------------------

The ``return_typestate`` attribute can be applied to functions or parameters.
When applied to a function the attribute specifies the state of the returned
value.  The function's body is checked to ensure that it always returns a value
in the specified state.  On the caller side, values returned by the annotated
function are initialized to the given state.

If the attribute is applied to a function parameter it modifies the state of
an argument after a call to the function returns.  The function's body is
checked to ensure that the parameter is in the expected state before returning. 

Type Safety Checking
====================

Clang supports additional attributes to enable checking type safety properties
that can't be enforced by the C type system.  Use cases include:

* MPI library implementations, where these attributes enable checking that
  the buffer type matches the passed ``MPI_Datatype``;
* for HDF5 library there is a similar use case to MPI;
* checking types of variadic functions' arguments for functions like
  ``fcntl()`` and ``ioctl()``.

You can detect support for these attributes with ``__has_attribute()``.  For
example:

.. code-block:: c++

  #if defined(__has_attribute)
  #  if __has_attribute(argument_with_type_tag) && \
        __has_attribute(pointer_with_type_tag) && \
        __has_attribute(type_tag_for_datatype)
  #    define ATTR_MPI_PWT(buffer_idx, type_idx) __attribute__((pointer_with_type_tag(mpi,buffer_idx,type_idx)))
  /* ... other macros ...  */
  #  endif
  #endif

  #if !defined(ATTR_MPI_PWT)
  # define ATTR_MPI_PWT(buffer_idx, type_idx)
  #endif

  int MPI_Send(void *buf, int count, MPI_Datatype datatype /*, other args omitted */)
      ATTR_MPI_PWT(1,3);

``argument_with_type_tag(...)``
-------------------------------

Use ``__attribute__((argument_with_type_tag(arg_kind, arg_idx,
type_tag_idx)))`` on a function declaration to specify that the function
accepts a type tag that determines the type of some other argument.
``arg_kind`` is an identifier that should be used when annotating all
applicable type tags.

This attribute is primarily useful for checking arguments of variadic functions
(``pointer_with_type_tag`` can be used in most non-variadic cases).

For example:

.. code-block:: c++

  int fcntl(int fd, int cmd, ...)
      __attribute__(( argument_with_type_tag(fcntl,3,2) ));

``pointer_with_type_tag(...)``
------------------------------

Use ``__attribute__((pointer_with_type_tag(ptr_kind, ptr_idx, type_tag_idx)))``
on a function declaration to specify that the function accepts a type tag that
determines the pointee type of some other pointer argument.

For example:

.. code-block:: c++

  int MPI_Send(void *buf, int count, MPI_Datatype datatype /*, other args omitted */)
      __attribute__(( pointer_with_type_tag(mpi,1,3) ));

``type_tag_for_datatype(...)``
------------------------------

Clang supports annotating type tags of two forms.

* **Type tag that is an expression containing a reference to some declared
  identifier.** Use ``__attribute__((type_tag_for_datatype(kind, type)))`` on a
  declaration with that identifier:

  .. code-block:: c++

    extern struct mpi_datatype mpi_datatype_int
        __attribute__(( type_tag_for_datatype(mpi,int) ));
    #define MPI_INT ((MPI_Datatype) &mpi_datatype_int)

* **Type tag that is an integral literal.** Introduce a ``static const``
  variable with a corresponding initializer value and attach
  ``__attribute__((type_tag_for_datatype(kind, type)))`` on that declaration,
  for example:

  .. code-block:: c++

    #define MPI_INT ((MPI_Datatype) 42)
    static const MPI_Datatype mpi_datatype_int
        __attribute__(( type_tag_for_datatype(mpi,int) )) = 42

The attribute also accepts an optional third argument that determines how the
expression is compared to the type tag.  There are two supported flags:

* ``layout_compatible`` will cause types to be compared according to
  layout-compatibility rules (C++11 [class.mem] p 17, 18).  This is
  implemented to support annotating types like ``MPI_DOUBLE_INT``.

  For example:

  .. code-block:: c++

    /* In mpi.h */
    struct internal_mpi_double_int { double d; int i; };
    extern struct mpi_datatype mpi_datatype_double_int
        __attribute__(( type_tag_for_datatype(mpi, struct internal_mpi_double_int, layout_compatible) ));

    #define MPI_DOUBLE_INT ((MPI_Datatype) &mpi_datatype_double_int)

    /* In user code */
    struct my_pair { double a; int b; };
    struct my_pair *buffer;
    MPI_Send(buffer, 1, MPI_DOUBLE_INT /*, ...  */); // no warning

    struct my_int_pair { int a; int b; }
    struct my_int_pair *buffer2;
    MPI_Send(buffer2, 1, MPI_DOUBLE_INT /*, ...  */); // warning: actual buffer element
                                                      // type 'struct my_int_pair'
                                                      // doesn't match specified MPI_Datatype

* ``must_be_null`` specifies that the expression should be a null pointer
  constant, for example:

  .. code-block:: c++

    /* In mpi.h */
    extern struct mpi_datatype mpi_datatype_null
        __attribute__(( type_tag_for_datatype(mpi, void, must_be_null) ));

    #define MPI_DATATYPE_NULL ((MPI_Datatype) &mpi_datatype_null)

    /* In user code */
    MPI_Send(buffer, 1, MPI_DATATYPE_NULL /*, ...  */); // warning: MPI_DATATYPE_NULL
                                                        // was specified but buffer
                                                        // is not a null pointer

Format String Checking
======================

Clang supports the ``format`` attribute, which indicates that the function
accepts a ``printf`` or ``scanf``-like format string and corresponding
arguments or a ``va_list`` that contains these arguments.

Please see `GCC documentation about format attribute
<http://gcc.gnu.org/onlinedocs/gcc/Function-Attributes.html>`_ to find details
about attribute syntax.

Clang implements two kinds of checks with this attribute.

#. Clang checks that the function with the ``format`` attribute is called with
   a format string that uses format specifiers that are allowed, and that
   arguments match the format string.  This is the ``-Wformat`` warning, it is
   on by default.

#. Clang checks that the format string argument is a literal string.  This is
   the ``-Wformat-nonliteral`` warning, it is off by default.

   Clang implements this mostly the same way as GCC, but there is a difference
   for functions that accept a ``va_list`` argument (for example, ``vprintf``).
   GCC does not emit ``-Wformat-nonliteral`` warning for calls to such
   fuctions.  Clang does not warn if the format string comes from a function
   parameter, where the function is annotated with a compatible attribute,
   otherwise it warns.  For example:

   .. code-block:: c

     __attribute__((__format__ (__scanf__, 1, 3)))
     void foo(const char* s, char *buf, ...) {
       va_list ap;
       va_start(ap, buf);

       vprintf(s, ap); // warning: format string is not a string literal
     }

   In this case we warn because ``s`` contains a format string for a
   ``scanf``-like function, but it is passed to a ``printf``-like function.

   If the attribute is removed, clang still warns, because the format string is
   not a string literal.

   Another example:

   .. code-block:: c

     __attribute__((__format__ (__printf__, 1, 3)))
     void foo(const char* s, char *buf, ...) {
       va_list ap;
       va_start(ap, buf);

       vprintf(s, ap); // warning
     }

   In this case Clang does not warn because the format string ``s`` and
   the corresponding arguments are annotated.  If the arguments are
   incorrect, the caller of ``foo`` will receive a warning.