A number of semaphore definitions exist, although the general concept remains as introduced by Dijkstra in 1968. We assume the reader has some familiarity with semaphores and their semantics, and so we will not cover them here. Many books are available for those requiring additional information. See especially Principles of Concurrent and Distributed Programming by M. Ben-Ari.

The GNAT.Semaphores package defines two semaphore abstractions, both in terms of protected types. We use protected types, rather than defining abstract data types based on operating-system facilities, for the sake of portability. A simple wrapper around the facilities of an operating system or RTOS would be reasonable, but the resulting interface (and implementation, of course) would not have the portability required of the general-purpose GNAT hierarchy.

The first abstraction defines a “counting” semaphore, in which an integer count is maintained by the operations, such that the acquire operation only executes (and thus returns, allowing the caller to continue) when the count is greater than zero. Counting semaphores are convenient for expressing condition synchronization in terms of the availability of some number of resources. For example, with a circular bounded buffer, a semaphore’s count can represent the number of available buffer slots. Callers must wait for the buffer to be non-full when attempting to insert a new item, and the blocking call to acquire the semaphore will achieve that effect directly.

The declaration for the counting semaphore is as follows. Note that we have removed the in-line comments for the sake of brevity, but will cover their content.

   protected type Counting_Semaphore
      (Initial_Value : Natural;
      Ceiling : System.Priority)
   is
      pragma Priority (Ceiling);

      entry Seize;
      procedure Release;

   private
      Count : Natural := Initial_Value;
   end Counting_Semaphore;

The second type implements “binary” semaphores. This kind of semaphore is less flexible than the counting semaphore, in that no notion of a count is maintained. The abstraction is simply that of a flag indicating whether or not the semaphore is available. (If the value of a counting semaphore varied between zero and one, the effect would be the same.)

   protected type Binary_Semaphore
     (Initially_Available : Boolean;
      Ceiling : System.Priority)
   is
      pragma Priority (Ceiling);

      entry Seize;
      procedure Release;

   private
      Available : Boolean := Initially_Available;
   end Binary_Semaphore;

In both cases, the provided operations consist of an entry “Seize” to acquire the semaphore with mutually exclusive access, and a protected procedure “Release” to release it. The Seize operation must be an entry for the sake of the barrier expressing the required condition synchronization. Releasing the semaphore has no such requirement, so it need not be an entry.

Both types are visibly defined as protected types so that users can make conditional and timed calls when appropriate. This capability addresses one of the portability problems with semaphores. Although the basic “acquire” and “release” operations will always be provided (by whatever name), there is no “standard” interface for other forms of interaction. Language-defined constructs provide some of those kinds of interactions, and do so portably.

Both types require discriminants. The counting semaphore type uses a discriminant to specify the initial nonnegative value of the count. The binary semaphore type uses a Boolean discriminant to specify whether the semaphore should be initially available. For example, a binary semaphore would be initially available when used to express mutual exclusion, but might not be initially available when used to express condition synchronization.

In addition, both types use another discriminant to specify the ceiling priority for objects of the type. The discriminant is then passed as the argument to pragma Priority. If the Real-Time Systems Annex is in force this part is essential; otherwise it has no effect. Note that a default value cannot be provided for the Ceiling discriminant because that would require a default for the other discriminant, too. The best we can do is to define a convenient value to be used in declarations of individual objects when the Real-Time Annex is not in force, so the following constant is provided in GNAT.Semaphores:

   Default_Ceiling : constant System.Priority := System.Default_Priority;

However, subtypes can be used to enhance convenience, readability and robustness. The earlier Gem (#70) provided an example:

   subtype Mutual_Exclusion is Binary_Semaphore
     (Initially_Available => True,
      Ceiling             => Default_Ceiling);

The subtype ensures that corresponding objects are initially available, but also conveniently supplies the Ceiling discriminant value (assuming the Real-Time Annex is not in force) and gives the reader an indication of the intended use.

As you can see, the declarations of protected types can be very expressive. The discriminants are the part you might not have considered using, although that is by no means an unusual approach. The protected bodies of both types are trivial and will not be shown here. You can see them in the GNAT hierarchy, along with the implementations of all the other GNAT hierarchy packages.

Introduction

Ada 95 brought us the predefined package Interfaces, with its standard definitions for types like Unsigned_32 and so on. It also defines various functions to do common shift and rotate operations on those types, such as:

   function Shift_Left
     (Value  : Unsigned_32;
      Amount : Natural) return Unsigned_32;

In GNAT Pro, these function declarations are also followed by a curious-looking pragma:

   pragma Import (Intrinsic, Shift_Left);

The Ada 95 RM (6.3.1) says:

‘The intrinsic calling convention represents subprograms that are “built in” to the compiler.’

This basically means that the code generator “knows” how to implement these functions. In the case of shifts and rotates, this means that a call to one of these functions results in very few (possibly just one) machine instructions — giving the performance you’d expect from such a simple operation.

Shifts and rotates are useful in a wide variety of applications, but are endemic in cryptographic algorithms, where they appear all over the place.

Unfortunately, the standard specification of package Interfaces is not legal SPARK, since the various shift and rotate functions are overloaded for each of the modular types, and overloading within a scope is not allowed. Darn…

To get round this, we first introduce a “shadow” specification of Interfaces that supplies the legal SPARK bits that we’re interested in — specifically the type declarations, thus:

   package Interfaces is
      type Unsigned_8  is mod 2**8;
      type Unsigned_16 is mod 2**16;
      type Unsigned_32 is mod 2**32;
      type Unsigned_64 is mod 2**64;
   end Interfaces;

This is legal SPARK, and goes in a file called interfaces.shs. We then use the Examiner’s index-file mechanism to make sure that the Examiner sees this version of the package specification.

To get at the shift and rotate functions, we need to introduce a new package that “de-overloads” the names. Let’s call this package “SR” — this is also a “shadow”, so it goes in sr.shs or a similar file. It looks like this:

   with Interfaces;
   --# inherit Interfaces;
   package SR is

      function Rotate_Left_16
        (Value  : Interfaces.Unsigned_16;
         Amount : Natural) return Interfaces.Unsigned_16;

      function Rotate_Left_32
        (Value  : Interfaces.Unsigned_32;
         Amount : Natural) return Interfaces.Unsigned_32;

      -- ...and so on for all required functions...
   end SR;

Note how we removed the overloading of the function names.

OK… so how do we glue package SR to the predefined intrinsic functions in package Interfaces?

It’s tempting to just implement the body of SR (in Ada, not SPARK) to “call through” to the appropriate function in Interfaces, but this might involve the overhead of a full-blown function call/return sequence, losing all that juicy performance that the intrinsic functions give us. We could try using pragma Inline to get rid of that overhead, but, as car salesmen say, “your mileage may vary” with that idea.

Fortunately, there is a way out that preserves the efficiency of the intrinsic version. Remember that the specification of SR above is a shadow? We supply the following slightly different version to the compiler in sr.ads:

   with Interfaces;
   package SR is

      function Rotate_Left_16
        (Value  : Interfaces.Unsigned_16;
         Amount : Natural) return Interfaces.Unsigned_16 renames Interfaces.Rotate_Left;

      function Rotate_Left_32
        (Value  : Interfaces.Unsigned_32;
         Amount : Natural) return Interfaces.Unsigned_32 renames Interfaces.Rotate_Left;

      -- ...and so on
   end SR;

The renaming here ensures that our SPARK-friendly “un-overloaded” functions are synonymous with the intrinsic “built in” functions, getting us the best of both worlds — the type safety of SPARK, with the efficiency of intrinsic machine code!

The full article can be found at:
http://embedded-computing.com/integrating-static-analysis-a-compiler-database

This Month’s Challenge
This code defines a function Cruise.Control which computes a mode for automatically controlling a car speed, given the information provided by sensors on the position and speed of surrounding vehicles. The mode distinguishes various levels of breaking and speeding.

To see CodePeer in action please click here.

Part III: External Tools

The previous two Gems on memory monitoring explained the use of GNAT.Debug_Pools and GNATCOLL.Memory to monitor memory access and detect memory leaks.

Some very powerful external tools exist on some systems that can be used to achieve similar goals. Very often, these tools will simply replace the system calls malloc and free, although in some cases they will in fact emulate a virtual machine in which your application is run.

One common kind of tool, available on most systems, indicates how much memory your application is using (such as the Process Manager on Windows or “top” on Unix systems). This is however a very limited tool, since memory that is properly freed by your application might in fact not be given back to the system (for performance reasons, malloc keeps it and reuses it for the next allocation). So it is very hard to discover memory leaks that way, and of course even if you can see that there is a leak, you have no way of knowing where it is in your code.

One very useful Linux application is valgrind. This is a virtual machine, that you start with various tools. One of them is a memory checker, which can detect invalid memory accesses, double deallocation, use of uninitialized variables, and memory leaks. You do not need to do anything special when compiling your application and you can simply run it as follows:

valgrind --tool=memcheck your_app app_arguments

If you also want to detect memory leaks, start your application with:

valgrind --tool=memcheck --leak-check=full --leak-resolution=med your_app

Compared to the techniques we discussed in the previous Gems, valgrind is much slower (since all code runs in a virtual machine), but more accurate. For instance, when it reports memory leaks, it will only report those chunks of memory that are no longer referenced anywhere in your application. For instance, if you have a global constant initialized once by calling “new …”, the Ada packages would report it as a leak, whereas valgrind by default knows it is still referenced and will not bother you with it (although, of course, you have an option to see it, namely –show-reachable). Since it is often very hard to free such memory (you have to do it at finalization time), and generally not worth it since the memory will be reclaimed by the system anyway, you generally want to ignore those chunks.

Likewise, if you have an allocated object that itself contains accesses to dynamically allocated memory, valgrind will by default only report the root object as a leak, since the other chunks are accessible from the first. When you fix the leak for the first, it is likely that you will at the same time fix the other leaks as well.

Other sources of memory leaks, much harder to detect, are those that occur in the graphical part of your application. On most systems, graphical rendering is a client-server process, and the memory is allocated on the server, not on the client. Therefore, tools like valgrind or the GNAT packages would not be able to report it as a leak (for instance if you have allocated a big pixmap but never freed it). On Windows these are called “GID”, and are visible in the Process Manager, so that you can actually monitor whether your application seems to have such leaks. Once again, though, this provides no detail as to the origin of the leak.

On Unix, you can use the “xrestop” application, which can tell you the number of windows, cursors, graphic context, etc. that you have allocated.

Many such tools like this exist, both commercial and free. Let us know if you routinely use such tools, as they could be useful to other readers of this Gem.

Unless your coding standard forbids any dynamic allocation, memory management is a constant concern during system development. You might want to limit the amount of memory that your application requires, or you might have memory leaks (allocation chunks that are never returned to the system). The latter is a critical concern for long-running applications.

Part II: System.Memory

In the previous Gem we discussed the use of GNAT.Debug_Pools to detect memory problems. Another approach that is somewhat easier to use, because it doesn’t require changes to the application, is to override the System.Memory package. In GNAT and its run-time, all actual memory allocations are done via this package, which among other things does the low-level system calls to malloc() and free().

If you create your own version of System.Memory, you will in fact short-circuit all memory allocation and deallocation, and replace it with your own. To do so, copy the file s-memory.adb to one of your source directories, modify it as appropriate, and compile your application, passing the “-a” switch to gnatmake (gprbuild currently does not have an equivalent switch, although using project files should work as expected). This ensures that GNAT will recompile the library with your modified System.

Using this package does not provide the same safety as the debug pools, since it does not check that dereferences are valid, and so your code could still be accessing invalid memory. On the other hand, the use of System.Memory is much less intrusive in your code. System.Memory is best viewed as a performance analysis tool rather than a debugging tool, although it will allow you to monitor your code for memory leaks.

The GNATCOLL library (a recent addition to the GNAT technology, and part of the latest customer and public releases) provides such an implementation in the form of the GNATCOLL.Memory package. This package is not a direct replacement for System.Memory, but only a minimal amount of work is needed to make use of it, by creating a version of s-memory.adb file that contains the following:

with GNATCOLL.Memory;
package body System.Memory is

   package M renames GNATCOLL.Memory;

   function Alloc (Size : size_t) return System.Address is
   begin
      return M.Alloc (M.size_t (Size));
   end Alloc;

   procedure Free (Ptr : System.Address) renames M.Free;

   function Realloc
      (Ptr  : System.Address;
       Size : size_t)
      return System.Address
   is
   begin
      return M.Realloc (Ptr, M.size_t (Size));
   end Realloc;

end System.Memory;

You then need to modify your code so that it properly initializes GNATCOLL.Memory, which is done via a call like the following:

   GNATCOLL.Memory.Configure (Activate_Monitor => True);

The monitoring provided by this GNATCOLL package is not enabled by default, to limit overhead on a running program. In your application, monitoring could be activated through a command-line switch, or by means of a specific environment variable. (This is all fully under your control, though, so you’ll have to do the actual call to Getenv and then to Configure.)

You can then instrument your code in one or more places to dump the memory usage at that point. This is done through a call such as the following:

   GNATCOLL.Memory.Dump (Size => 3, Report => Memory_Usage);

Such a call will print on the console three backtraces for the code that allocated the most memory (among the currently allocated memory). Variants exist to dump the backtraces that executed the greatest number of allocations (as opposed to the largest allocation size), or the total amount of memory, even if that memory has since been released.

Such a dump includes a backtrace with addresses, which you can convert to a symbolic backtrace by using the external tool addr2line as follows:

    addr2line -e  

This library has very light overhead (in particular when not activated) so that you can distribute your application with support for GNATCOLL.Memory built in, and then investigate any issues that arise in production code. In fact, our own GPS IDE now includes this support. Activation is controlled by an external variable, and dumping the state of memory can be done through a Python command at any point in time without the need to recompile GPS. (Note that GNATCOLL also provides an interface to Python.)

Another feature of GNATCOLL.Memory is the capability of resetting all counters to zero. For example, let’s assume we want to investigate memory leaks while opening and closing editors in GPS. If we look at the places that allocate memory, the biggest allocations that are displayed in the console do not concern the editor itself, but rather memory allocated when GPS started (if you are curious, this is generally memory that is related to the cross-reference database). Therefore, we would do the following: start GPS, reset GNATCOLL.Memory counters to 0, open and close an editor, and dump memory usage. At that point, if Dump prints any information on the console, we know that this is memory that has been allocated since the call to Reset, and that wasn’t freed when the editor was closed, and therefore is most likely a memory leak.

The appropriate use of Reset and Dump therefore allows the monitoring of memory usage in specific parts of the code.

A separate tool called gnatmem is also distributed with GNAT. When you link your application with the -lgmem switch, it will transparently instrument all calls to the standard malloc and free (from the Ada code), in a fashion similar to GNATCOLL.Memory. On program exit, a disk file is created that you can then analyze using gnatmem, to highlight the sources of memory leaks in your application. This tool requires no change to your code at all, but, on the other hand, does not provide a way to monitor specific sections of your code like GNATCOLL.Memory does.

Gnatmem provides a number of command-line switches to control the display of information. For instance, the switch “-m 0″ lets you view all places in your code that ever allocated memory, even if that memory was properly deallocated afterwards. When one such place is doing millions of allocations, it might sometimes be more efficient to use a custom Storage_Pool and avoid the system call to malloc, by reusing memory. The “-s” switch allows you to sort the output in various ways.

In Part III of this series we will take a look at various commercially available system tools for monitoring and analyzing memory usage.

Unless your coding standard forbids any dynamic allocation, memory management is a constant concern during system development. You might want to limit the amount of memory that your application requires, or you might have memory leaks (allocation chunks that are never returned to the system). The latter is a critical concern for long-running applications.

Part I: Storage Pools

The standard Ada memory management mechanism is the storage pool, as defined in package System.Storage_Pools. A storage pool is a tagged type that allows you to override the standard “new” operator and the associated Unchecked_Deallocation procedure. A given pool can be associated with one or more access types. The GNAT run-time comes with a number of predefined storage pools, and you can also create your own. One basic implementation, for instance, would be to instrument the pool operations to print a trace to the console every time memory is allocated or freed, and then post-process those traces with an external tool afterward.

This is of course a little tedious, so GNAT provides the package GNAT.Debug_Pools as a much more advanced storage pool implementation that can, at any time during program execution, display the currently allocated memory (along with a backtrace of the code at the point of allocation). It will also detect invalid memory references (for instance, attempting to dereference a pointer to already deallocated memory). The implementation is efficient andl imposes only a very small overhead on your application.

Here is a short example demonstrating the use of debug pools:

with GNAT.Debug_Pools;

package My_Package is
   Pool : GNAT.Debug_Pools.Debug_Pool;

   type Integer_Access is access Integer;
   for Integer_Access'Storage_Pool use Pool;
end My_Package;

with My_Package;
with Ada.Unchecked_Deallocation;
procedure Main is
   procedure Unchecked_Free is
      new Ada.Unchecked_Deallocation (Integer, Integer_Access);
   Ptr : Integer_Access;
begin
   Ptr := new Integer;
   Ptr.all := 1;
   Unchecked_Free (Ptr);
   Ptr.all := 2;  --  raises exception
end Main;

The variable My_Package.Pool should be shared as much as possible among all your access types. It’s not necessary to create one per access type.

As noted in the main procedure, the last reference to Ptr is invalid, and will result in an exception raised from the debug pool (rather than some erroneous behavior depending on the system).

GNAT.Debug_Pools provides various subprograms to analyze current memory usage, in particular the total amount of memory currently allocated, as well as which part of the code did the allocations. The backtraces are also useful when analyzing double-deallocation scenarios, since a debug pool shows both where the memory was allocated and by what piece of code it was first deallocated.

However, GNAT’s debug pools are rather heavy to put in place in existing code, since you need to add a “for Type_Name’Storage_Pool use Pool” to every access type, and there is no mechanism to define a single default storage pool for all types. GNAT.Debug_Pools can also give false warnings when dereferencing a pointer to aliased data on the stack (which was never allocated via a “new” operator, but was accessed via an ‘Access attribute).

In the next Gem we will discuss an alternative approach to controlling and instrumenting dynamic allocation and deallocation, by overriding the low-level memory management support itself.

Agile methods promote a disciplined project management process that encourages frequent inspection and adaptation, a leadership philosophy that encourages teamwork, self-organization and accountability, a set of engineering best practices that allow for rapid delivery of high-quality software, and a business approach that aligns development with customer needs and company goals. Formal Methods Formal methods and tools promise to find tough corner-case bugs and provide exhaustive proofs of software properties, based on mathematical techniques that are used both for the specification of properties and their verification. Although formal methods and tools don’t necessarily require PhDs in formal verification, they definitely require some expertise: someone has to know the tool well, understand how to write good formal properties, and know what to do when proofs don’t complete.

This conference brings together experts from the two fields and asks the question “Can Formality and Agility be combined?” to develop software that has to reach the highest levels of safety and security. Talks will examine the theoretical consolidation between these software development principles and industrial case studies will show how they are used in practice.

Speakers include Paul Boca, Jim Woodcock, Herve Delseny, Peter Gardner, Rod Chapman, Cyrille Comar.