A number of semaphore definitions exist, although the general concept remains as introduced by Dijkstra in 1968. We assume the reader has some familiarity with semaphores and their semantics, and so we will not cover them here. Many books are available for those requiring additional information. See especially Principles of Concurrent and Distributed Programming by M. Ben-Ari.

The GNAT.Semaphores package defines two semaphore abstractions, both in terms of protected types. We use protected types, rather than defining abstract data types based on operating-system facilities, for the sake of portability. A simple wrapper around the facilities of an operating system or RTOS would be reasonable, but the resulting interface (and implementation, of course) would not have the portability required of the general-purpose GNAT hierarchy.

The first abstraction defines a “counting” semaphore, in which an integer count is maintained by the operations, such that the acquire operation only executes (and thus returns, allowing the caller to continue) when the count is greater than zero. Counting semaphores are convenient for expressing condition synchronization in terms of the availability of some number of resources. For example, with a circular bounded buffer, a semaphore’s count can represent the number of available buffer slots. Callers must wait for the buffer to be non-full when attempting to insert a new item, and the blocking call to acquire the semaphore will achieve that effect directly.

The declaration for the counting semaphore is as follows. Note that we have removed the in-line comments for the sake of brevity, but will cover their content.

   protected type Counting_Semaphore
      (Initial_Value : Natural;
      Ceiling : System.Priority)
   is
      pragma Priority (Ceiling);

      entry Seize;
      procedure Release;

   private
      Count : Natural := Initial_Value;
   end Counting_Semaphore;

The second type implements “binary” semaphores. This kind of semaphore is less flexible than the counting semaphore, in that no notion of a count is maintained. The abstraction is simply that of a flag indicating whether or not the semaphore is available. (If the value of a counting semaphore varied between zero and one, the effect would be the same.)

   protected type Binary_Semaphore
     (Initially_Available : Boolean;
      Ceiling : System.Priority)
   is
      pragma Priority (Ceiling);

      entry Seize;
      procedure Release;

   private
      Available : Boolean := Initially_Available;
   end Binary_Semaphore;

In both cases, the provided operations consist of an entry “Seize” to acquire the semaphore with mutually exclusive access, and a protected procedure “Release” to release it. The Seize operation must be an entry for the sake of the barrier expressing the required condition synchronization. Releasing the semaphore has no such requirement, so it need not be an entry.

Both types are visibly defined as protected types so that users can make conditional and timed calls when appropriate. This capability addresses one of the portability problems with semaphores. Although the basic “acquire” and “release” operations will always be provided (by whatever name), there is no “standard” interface for other forms of interaction. Language-defined constructs provide some of those kinds of interactions, and do so portably.

Both types require discriminants. The counting semaphore type uses a discriminant to specify the initial nonnegative value of the count. The binary semaphore type uses a Boolean discriminant to specify whether the semaphore should be initially available. For example, a binary semaphore would be initially available when used to express mutual exclusion, but might not be initially available when used to express condition synchronization.

In addition, both types use another discriminant to specify the ceiling priority for objects of the type. The discriminant is then passed as the argument to pragma Priority. If the Real-Time Systems Annex is in force this part is essential; otherwise it has no effect. Note that a default value cannot be provided for the Ceiling discriminant because that would require a default for the other discriminant, too. The best we can do is to define a convenient value to be used in declarations of individual objects when the Real-Time Annex is not in force, so the following constant is provided in GNAT.Semaphores:

   Default_Ceiling : constant System.Priority := System.Default_Priority;

However, subtypes can be used to enhance convenience, readability and robustness. The earlier Gem (#70) provided an example:

   subtype Mutual_Exclusion is Binary_Semaphore
     (Initially_Available => True,
      Ceiling             => Default_Ceiling);

The subtype ensures that corresponding objects are initially available, but also conveniently supplies the Ceiling discriminant value (assuming the Real-Time Annex is not in force) and gives the reader an indication of the intended use.

As you can see, the declarations of protected types can be very expressive. The discriminants are the part you might not have considered using, although that is by no means an unusual approach. The protected bodies of both types are trivial and will not be shown here. You can see them in the GNAT hierarchy, along with the implementations of all the other GNAT hierarchy packages.

Introduction

Ada 95 brought us the predefined package Interfaces, with its standard definitions for types like Unsigned_32 and so on. It also defines various functions to do common shift and rotate operations on those types, such as:

   function Shift_Left
     (Value  : Unsigned_32;
      Amount : Natural) return Unsigned_32;

In GNAT Pro, these function declarations are also followed by a curious-looking pragma:

   pragma Import (Intrinsic, Shift_Left);

The Ada 95 RM (6.3.1) says:

‘The intrinsic calling convention represents subprograms that are “built in” to the compiler.’

This basically means that the code generator “knows” how to implement these functions. In the case of shifts and rotates, this means that a call to one of these functions results in very few (possibly just one) machine instructions — giving the performance you’d expect from such a simple operation.

Shifts and rotates are useful in a wide variety of applications, but are endemic in cryptographic algorithms, where they appear all over the place.

Unfortunately, the standard specification of package Interfaces is not legal SPARK, since the various shift and rotate functions are overloaded for each of the modular types, and overloading within a scope is not allowed. Darn…

To get round this, we first introduce a “shadow” specification of Interfaces that supplies the legal SPARK bits that we’re interested in — specifically the type declarations, thus:

   package Interfaces is
      type Unsigned_8  is mod 2**8;
      type Unsigned_16 is mod 2**16;
      type Unsigned_32 is mod 2**32;
      type Unsigned_64 is mod 2**64;
   end Interfaces;

This is legal SPARK, and goes in a file called interfaces.shs. We then use the Examiner’s index-file mechanism to make sure that the Examiner sees this version of the package specification.

To get at the shift and rotate functions, we need to introduce a new package that “de-overloads” the names. Let’s call this package “SR” — this is also a “shadow”, so it goes in sr.shs or a similar file. It looks like this:

   with Interfaces;
   --# inherit Interfaces;
   package SR is

      function Rotate_Left_16
        (Value  : Interfaces.Unsigned_16;
         Amount : Natural) return Interfaces.Unsigned_16;

      function Rotate_Left_32
        (Value  : Interfaces.Unsigned_32;
         Amount : Natural) return Interfaces.Unsigned_32;

      -- ...and so on for all required functions...
   end SR;

Note how we removed the overloading of the function names.

OK… so how do we glue package SR to the predefined intrinsic functions in package Interfaces?

It’s tempting to just implement the body of SR (in Ada, not SPARK) to “call through” to the appropriate function in Interfaces, but this might involve the overhead of a full-blown function call/return sequence, losing all that juicy performance that the intrinsic functions give us. We could try using pragma Inline to get rid of that overhead, but, as car salesmen say, “your mileage may vary” with that idea.

Fortunately, there is a way out that preserves the efficiency of the intrinsic version. Remember that the specification of SR above is a shadow? We supply the following slightly different version to the compiler in sr.ads:

   with Interfaces;
   package SR is

      function Rotate_Left_16
        (Value  : Interfaces.Unsigned_16;
         Amount : Natural) return Interfaces.Unsigned_16 renames Interfaces.Rotate_Left;

      function Rotate_Left_32
        (Value  : Interfaces.Unsigned_32;
         Amount : Natural) return Interfaces.Unsigned_32 renames Interfaces.Rotate_Left;

      -- ...and so on
   end SR;

The renaming here ensures that our SPARK-friendly “un-overloaded” functions are synonymous with the intrinsic “built in” functions, getting us the best of both worlds — the type safety of SPARK, with the efficiency of intrinsic machine code!

Part III: External Tools

The previous two Gems on memory monitoring explained the use of GNAT.Debug_Pools and GNATCOLL.Memory to monitor memory access and detect memory leaks.

Some very powerful external tools exist on some systems that can be used to achieve similar goals. Very often, these tools will simply replace the system calls malloc and free, although in some cases they will in fact emulate a virtual machine in which your application is run.

One common kind of tool, available on most systems, indicates how much memory your application is using (such as the Process Manager on Windows or “top” on Unix systems). This is however a very limited tool, since memory that is properly freed by your application might in fact not be given back to the system (for performance reasons, malloc keeps it and reuses it for the next allocation). So it is very hard to discover memory leaks that way, and of course even if you can see that there is a leak, you have no way of knowing where it is in your code.

One very useful Linux application is valgrind. This is a virtual machine, that you start with various tools. One of them is a memory checker, which can detect invalid memory accesses, double deallocation, use of uninitialized variables, and memory leaks. You do not need to do anything special when compiling your application and you can simply run it as follows:

valgrind --tool=memcheck your_app app_arguments

If you also want to detect memory leaks, start your application with:

valgrind --tool=memcheck --leak-check=full --leak-resolution=med your_app

Compared to the techniques we discussed in the previous Gems, valgrind is much slower (since all code runs in a virtual machine), but more accurate. For instance, when it reports memory leaks, it will only report those chunks of memory that are no longer referenced anywhere in your application. For instance, if you have a global constant initialized once by calling “new …”, the Ada packages would report it as a leak, whereas valgrind by default knows it is still referenced and will not bother you with it (although, of course, you have an option to see it, namely –show-reachable). Since it is often very hard to free such memory (you have to do it at finalization time), and generally not worth it since the memory will be reclaimed by the system anyway, you generally want to ignore those chunks.

Likewise, if you have an allocated object that itself contains accesses to dynamically allocated memory, valgrind will by default only report the root object as a leak, since the other chunks are accessible from the first. When you fix the leak for the first, it is likely that you will at the same time fix the other leaks as well.

Other sources of memory leaks, much harder to detect, are those that occur in the graphical part of your application. On most systems, graphical rendering is a client-server process, and the memory is allocated on the server, not on the client. Therefore, tools like valgrind or the GNAT packages would not be able to report it as a leak (for instance if you have allocated a big pixmap but never freed it). On Windows these are called “GID”, and are visible in the Process Manager, so that you can actually monitor whether your application seems to have such leaks. Once again, though, this provides no detail as to the origin of the leak.

On Unix, you can use the “xrestop” application, which can tell you the number of windows, cursors, graphic context, etc. that you have allocated.

Many such tools like this exist, both commercial and free. Let us know if you routinely use such tools, as they could be useful to other readers of this Gem.

Unless your coding standard forbids any dynamic allocation, memory management is a constant concern during system development. You might want to limit the amount of memory that your application requires, or you might have memory leaks (allocation chunks that are never returned to the system). The latter is a critical concern for long-running applications.

Part II: System.Memory

In the previous Gem we discussed the use of GNAT.Debug_Pools to detect memory problems. Another approach that is somewhat easier to use, because it doesn’t require changes to the application, is to override the System.Memory package. In GNAT and its run-time, all actual memory allocations are done via this package, which among other things does the low-level system calls to malloc() and free().

If you create your own version of System.Memory, you will in fact short-circuit all memory allocation and deallocation, and replace it with your own. To do so, copy the file s-memory.adb to one of your source directories, modify it as appropriate, and compile your application, passing the “-a” switch to gnatmake (gprbuild currently does not have an equivalent switch, although using project files should work as expected). This ensures that GNAT will recompile the library with your modified System.

Using this package does not provide the same safety as the debug pools, since it does not check that dereferences are valid, and so your code could still be accessing invalid memory. On the other hand, the use of System.Memory is much less intrusive in your code. System.Memory is best viewed as a performance analysis tool rather than a debugging tool, although it will allow you to monitor your code for memory leaks.

The GNATCOLL library (a recent addition to the GNAT technology, and part of the latest customer and public releases) provides such an implementation in the form of the GNATCOLL.Memory package. This package is not a direct replacement for System.Memory, but only a minimal amount of work is needed to make use of it, by creating a version of s-memory.adb file that contains the following:

with GNATCOLL.Memory;
package body System.Memory is

   package M renames GNATCOLL.Memory;

   function Alloc (Size : size_t) return System.Address is
   begin
      return M.Alloc (M.size_t (Size));
   end Alloc;

   procedure Free (Ptr : System.Address) renames M.Free;

   function Realloc
      (Ptr  : System.Address;
       Size : size_t)
      return System.Address
   is
   begin
      return M.Realloc (Ptr, M.size_t (Size));
   end Realloc;

end System.Memory;

You then need to modify your code so that it properly initializes GNATCOLL.Memory, which is done via a call like the following:

   GNATCOLL.Memory.Configure (Activate_Monitor => True);

The monitoring provided by this GNATCOLL package is not enabled by default, to limit overhead on a running program. In your application, monitoring could be activated through a command-line switch, or by means of a specific environment variable. (This is all fully under your control, though, so you’ll have to do the actual call to Getenv and then to Configure.)

You can then instrument your code in one or more places to dump the memory usage at that point. This is done through a call such as the following:

   GNATCOLL.Memory.Dump (Size => 3, Report => Memory_Usage);

Such a call will print on the console three backtraces for the code that allocated the most memory (among the currently allocated memory). Variants exist to dump the backtraces that executed the greatest number of allocations (as opposed to the largest allocation size), or the total amount of memory, even if that memory has since been released.

Such a dump includes a backtrace with addresses, which you can convert to a symbolic backtrace by using the external tool addr2line as follows:

    addr2line -e  

This library has very light overhead (in particular when not activated) so that you can distribute your application with support for GNATCOLL.Memory built in, and then investigate any issues that arise in production code. In fact, our own GPS IDE now includes this support. Activation is controlled by an external variable, and dumping the state of memory can be done through a Python command at any point in time without the need to recompile GPS. (Note that GNATCOLL also provides an interface to Python.)

Another feature of GNATCOLL.Memory is the capability of resetting all counters to zero. For example, let’s assume we want to investigate memory leaks while opening and closing editors in GPS. If we look at the places that allocate memory, the biggest allocations that are displayed in the console do not concern the editor itself, but rather memory allocated when GPS started (if you are curious, this is generally memory that is related to the cross-reference database). Therefore, we would do the following: start GPS, reset GNATCOLL.Memory counters to 0, open and close an editor, and dump memory usage. At that point, if Dump prints any information on the console, we know that this is memory that has been allocated since the call to Reset, and that wasn’t freed when the editor was closed, and therefore is most likely a memory leak.

The appropriate use of Reset and Dump therefore allows the monitoring of memory usage in specific parts of the code.

A separate tool called gnatmem is also distributed with GNAT. When you link your application with the -lgmem switch, it will transparently instrument all calls to the standard malloc and free (from the Ada code), in a fashion similar to GNATCOLL.Memory. On program exit, a disk file is created that you can then analyze using gnatmem, to highlight the sources of memory leaks in your application. This tool requires no change to your code at all, but, on the other hand, does not provide a way to monitor specific sections of your code like GNATCOLL.Memory does.

Gnatmem provides a number of command-line switches to control the display of information. For instance, the switch “-m 0″ lets you view all places in your code that ever allocated memory, even if that memory was properly deallocated afterwards. When one such place is doing millions of allocations, it might sometimes be more efficient to use a custom Storage_Pool and avoid the system call to malloc, by reusing memory. The “-s” switch allows you to sort the output in various ways.

In Part III of this series we will take a look at various commercially available system tools for monitoring and analyzing memory usage.

Unless your coding standard forbids any dynamic allocation, memory management is a constant concern during system development. You might want to limit the amount of memory that your application requires, or you might have memory leaks (allocation chunks that are never returned to the system). The latter is a critical concern for long-running applications.

Part I: Storage Pools

The standard Ada memory management mechanism is the storage pool, as defined in package System.Storage_Pools. A storage pool is a tagged type that allows you to override the standard “new” operator and the associated Unchecked_Deallocation procedure. A given pool can be associated with one or more access types. The GNAT run-time comes with a number of predefined storage pools, and you can also create your own. One basic implementation, for instance, would be to instrument the pool operations to print a trace to the console every time memory is allocated or freed, and then post-process those traces with an external tool afterward.

This is of course a little tedious, so GNAT provides the package GNAT.Debug_Pools as a much more advanced storage pool implementation that can, at any time during program execution, display the currently allocated memory (along with a backtrace of the code at the point of allocation). It will also detect invalid memory references (for instance, attempting to dereference a pointer to already deallocated memory). The implementation is efficient andl imposes only a very small overhead on your application.

Here is a short example demonstrating the use of debug pools:

with GNAT.Debug_Pools;

package My_Package is
   Pool : GNAT.Debug_Pools.Debug_Pool;

   type Integer_Access is access Integer;
   for Integer_Access'Storage_Pool use Pool;
end My_Package;

with My_Package;
with Ada.Unchecked_Deallocation;
procedure Main is
   procedure Unchecked_Free is
      new Ada.Unchecked_Deallocation (Integer, Integer_Access);
   Ptr : Integer_Access;
begin
   Ptr := new Integer;
   Ptr.all := 1;
   Unchecked_Free (Ptr);
   Ptr.all := 2;  --  raises exception
end Main;

The variable My_Package.Pool should be shared as much as possible among all your access types. It’s not necessary to create one per access type.

As noted in the main procedure, the last reference to Ptr is invalid, and will result in an exception raised from the debug pool (rather than some erroneous behavior depending on the system).

GNAT.Debug_Pools provides various subprograms to analyze current memory usage, in particular the total amount of memory currently allocated, as well as which part of the code did the allocations. The backtraces are also useful when analyzing double-deallocation scenarios, since a debug pool shows both where the memory was allocated and by what piece of code it was first deallocated.

However, GNAT’s debug pools are rather heavy to put in place in existing code, since you need to add a “for Type_Name’Storage_Pool use Pool” to every access type, and there is no mechanism to define a single default storage pool for all types. GNAT.Debug_Pools can also give false warnings when dereferencing a pointer to aliased data on the stack (which was never allocated via a “new” operator, but was accessed via an ‘Access attribute).

In the next Gem we will discuss an alternative approach to controlling and instrumenting dynamic allocation and deallocation, by overriding the low-level memory management support itself.

Error message information leak occurs when secure data is leaked, through error messages, to unauthorised users, and is one of the top twenty-five most dangerous programming errors according to SANS Institute. The general problem is ensuring that information flow adheres to certain policies — for example, certain data should never be written in an error message to a log file that may be accessible by unauthorised users.

The objective of this Gem is to demonstrate that the Examiner detects information flow violations.

Step-by-Step Instructions

Step 1: Study a Contract from an Information Flow Perspective

The code below is from the procedure Verify in bio.adb. The out variable MatchResult returns the result of whether a person’s fingerprint matched their template. The local variable NumericReturn is set to the enumerated value BioApiOk if the fingerprint successfully matched; otherwise it returns an error code.

When a match is unsuccessful, a log record is written including the variable NumericReturn, which is derived from the person’s Template.

      221  procedure Verify(Template       : in     IandATypes.TemplateT;
      222                   MaxFAR         : in     IandATypes.FarT;
      223                   MatchResult    :    out IandATypes.MatchResultT;
      224                   AchievedFAR    :    out IandATypes.FarT)
             ...
      230  --# derives AuditLog.State,
      231  --#         AuditLog.FileState from AuditLog.State,
      232  --#                                 AuditLog.FileState,
      233  --#                                 Template,
      234  --#                                 Clock.Now,
      235  --#                                 ConfigData.State,
      236  --#                                 Interface.Input &
             ...
      242  is
      243     NumericReturn : BasicTypes.Unsigned32T;
      244  begin
      245     Interface.Verify(Template     => Template,
      246                      MaxFAR       => MaxFAR,
      247                      MatchResult  => MatchResult,
      248                      AchievedFAR  => AchievedFAR,
      249                      BioReturn    => NumericReturn);
      250
      251     if NumericReturn /= ValueOf(BioAPIOk) then
      252        -- An error occurred, overwrite match information.

      253        MatchResult := IandATypes.NoMatch;
      254        AuditLog.AddElementToLog
      255          (ElementID    => AuditTypes.SystemFault,
      256           Severity     => AuditTypes.Warning,
      257           User         => AuditTypes.NoUser,
      258           Description  => MakeDescription ("Biometric device failure ",
      259                                            NumericReturn));
      260        end if;
      261  end Verify;

If the log were accessible to potential hackers, which is not the case for Tokeneer, then this would be an example of an error message information leak. Fingerprint templates should never be accessible by hackers.

Step 2: Change an Information Flow Aspect of the Contract

Change the contract for the procedure Verify to specify that no information written to the log is derived from Template by deleting line 233.

       230   --# derives AuditLog.State,
       231   --#         AuditLog.FileState from AuditLog.State,
       232   --#                                 AuditLog.FileState,
       233   --#                                                     
       234   --#                                 Clock.Now,
       235   --#                                 ConfigData.State,
       236   --#                                 Interface.Input &

The corresponding line (line 73) of code needs to be removed from the file bio.ads.

       60   procedure Verify(Template       : in     IandATypes.TemplateT;
       61                    MaxFAR         : in     IandATypes.FarT;
       62                    MatchResult    :    out IandATypes.MatchResultT;
       63                    AchievedFAR    :    out IandATypes.FarT);
            ...
       69   --# derives AuditLog.State,
       70   --#         AuditLog.FileState from Input,
       71   --#                                 AuditLog.State,
       72   --#                                 AuditLog.FileState,
       73   --#                                                     
       74   --#                                 Clock.Now,
       75   --#                                 ConfigData.State &

Step 3: Use the SPARK Tools to Detect the Information Leak

Examine the file bio.adb and notice the Examiner reports the error that information derived from the variable Template is written to the log. This means that data derived from the template is being written to the log!

bio.adb:261:8:
    Flow Error 601 - AuditLog.State may be derived from the imported value(s) of Template.
bio.adb:261:8:
    Flow Error 601 - AuditLog.FileState may be derived from the imported value(s) of Template.

Step 4: Introduce a Malicious Hack

The Examiner also detects when data has been incorrectly used. We now add back door code in the procedure Verify (lines 248-251 below). It returns a positive match independent of the user’s fingerprint when the clock is at midnight.

      223  procedure Verify(Template       : in     IandATypes.TemplateT;
      224                   MaxFAR         : in     IandATypes.FarT;
      225                   MatchResult    :    out IandATypes.MatchResultT;
      226                   AchievedFAR    :    out IandATypes.FarT)
            ...
      232  --# derives AuditLog.State,
      233  --#         AuditLog.FileState from AuditLog.State,
      234  --#                                 AuditLog.FileState,
      235  --#                                 Template,
      236  --#                                 Clock.Now,
      237  --#                                 ConfigData.State,
      238  --#                                 Interface.Input &
               ...
      244  is
      245     NumericReturn : BasicTypes.Unsigned32T;
      246     T             : Clock.TimeT;
      247  begin
      248     T := Clock.GetNow;
      249     if T = Clock.ZeroTime then
      250        MatchResult := IandATypes.Match;
      251        AchievedFAR := 0;
      252     else
      253
      254        Interface.Verify(Template     => Template,
      255                         MaxFAR       => MaxFAR,
      256                         MatchResult  => MatchResult,
      257                         AchievedFAR  => AchievedFAR,
      258                         BioReturn    => NumericReturn);
      259
              ...
      272    end if; 
             ...
      275  end Verify;

Step 5: Use the SPARK Tools to Detect the Weakness

Examine the file bio_bad.adb and notice that the Examiner reports the error that MatchResult is dependent on the current time (Clock.now), which is inconsistent with the procedure’s contract. The Examiner has identified an information flow inconsistency due to the presence of the back door.

bio_bad.adb:243:40:
    Flow Error   4 - The dependency of the exported value of AuditLog.State on the imported value
    of  Verify.Template has not been previously stated.
bio_bad.adb:243:40:
    Flow Error   4 - The dependency of the exported value of AuditLog.FileState on the imported value
    of Verify.Template has not been previously stated.
bio_bad.adb:275:8:
    Flow Error 601 - MatchResult may be derived from the imported value(s) of Clock.Now.
bio_bad.adb:275:8:
    Flow Error 601 - AchievedFAR may be derived from the imported value(s) of Clock.Now.

Summary

In this Gem we have learnt about information flow contracts and we have seen the SPARK tools detect a malicious hack. SPARK programs are free from information leaks when the contract accurately specifies the desired information flow between variables.

An overflow error occurs when the capacity of a device is exceeded. Overflow errors are a source of quality and security concerns. For instance, when an arithmetic overflow occurs, a calculated value does not fit in its specified size, and the calculation (and the program) just stops. Buffer overflow happens when a process stores data in a buffer outside of the memory that the programmer set aside for it. Buffer overflow errors are widely known to present a vulnerability to malicious hackers, who might exploit the error to sneak their own code onto a victim’s disk, storing it outside of the intended buffer.

The SPARK tools detect all potential arithmetic and buffer overflow errors. In the Gem about input validation, we saw an example of an arithmetic overflow. In this Gem, we will study how the SPARK tools find a buffer overflow error that we have injected into the Tokeneer code.

Step-by-Step Instructions

We will introduce a buffer overflow error into auditlog.adb and show how the SPARK Toolset detects it.

Step 1: Inject a Buffer Overflow Error

The procedure AddElementToCurrentFile increments the element indexed by CurrentLogFile in the array LogFileEntries – see below.

      775  --# post LogFileEntries(CurrentLogFile) =
      776  --#             LogFileEntries~(CurrentLogFile) + 1;
              ...
      790     LogFileEntries(CurrentLogFile) := LogFileEntries(CurrentLogFile) + 1;

Change the code on line 790 and the postcondition on 775 to 776 so the procedure copies the value in the CurrentLogFile+1th element into the CurrentLogFileth element of the array. The modified code is shown below.

      774  --# pre LogFileEntries(CurrentLogFile) < MaxLogFileEntries;
      775  --# post LogFileEntries(CurrentLogFile) =
      776  --#             LogFileEntries~(CurrentLogFile+1);
           ...
      790  LogFileEntries(CurrentLogFile) := LogFileEntries(CurrentLogFile+1) ;

Step 2: Analyse and Study the Verification Output

Analyse Tokeneer. The SPARK Toolset identifies, on lines 587 to 597 (see below), that there is a potential problem with the procedure AddElementToCurrentFile.

 659    VCs for procedure_addelementtocurrentfile :
 660    ----------------------------------------------------------------------------
 661          |       |                     |  -----Proved In-----  |       |       |
 662    #    | From  | To                  | vcg | siv | plg | prv | False | TO DO |
 663    ----------------------------------------------------------------------------
 664     1    | start | rtc check @ 781     |     | YES |     |     |       |       | 
 665     2    | start | rtc check @ 782     |     | YES |     |     |       |       | 
 666     3    | start | rtc check @ 788     |     | YES |     |     |       |       | 
 667     4    | start | rtc check @ 790     |     |     |     |     |       |  YES  |
 668     5    | start |    assert @ finish  |     | YES |     |     |       |       | 
 669    ----------------------------------------------------------------------------

The Simplifier failed to show that CurrentLogFile+1 is always within range of the index type, because it is not true – it goes outside its range when CurrentLogFile = LogFileIndexType’Last, which then causes a buffer overflow.

Summary

Buffer overflow errors are common and present a security vulnerability. The SPARK Toolset can verify that a SPARK program is free from arithmetic as well as buffer overflow errors. In this Gem we have seen how the SPARK tools can be used to detect a buffer overflow error. In the next Gem, we will see how SPARK can be used in Ensuring Secure Information Flow.

In this Gem, we will see how the SPARK tools detect any differences between a program’s intended behaviour, as specified in its contract, and its actual behaviour, as implemented in the code. Thus the SPARK tools can be used either to find defects in the contract, to find defects in the implementation, to find defects in both, or to show conformance between intended and actual behaviour.

Step-by-Step Instructions

Step 1: Analyse the Correct Version of the Code

The precondition for procedure AddElementToCurrentFile in auditlog.adb (lines 772 – 774 in the code below) specifies that the array element LogFileEntries (CurrentLogFile) is less than MaxLogFileEntries and the postcondition specifies that the array element is incremented by 1.

      751        procedure AddElementToCurrentFile
      752          (ElementID    : in     AuditTypes.ElementT;
      753           Severity     : in     AuditTypes.SeverityT;
      754           User         : in     AuditTypes.UserTextT;
      755           Description  : in     AuditTypes.DescriptionT)
      756          --# global in     Clock.Now;
      757          --#        in     CurrentLogFile;
      758          --#        in out AuditSystemFault;
      759          --#        in out LogFiles;
      760          --#        in out LogFileEntries;
      761          --# derives AuditSystemFault,
      762          --#         LogFiles         from *,
      763          --#                               Description,
      764          --#                               LogFiles,
      765          --#                               Clock.Now,
      766          --#                               ElementID,
      767          --#                               Severity,
      768          --#                               User,
      769          --#                               CurrentLogFile &
      770          --#         LogFileEntries   from *,
      771          --#                               CurrentLogFile;
      772          --# pre LogFileEntries(CurrentLogFile) < MaxLogFileEntries;
      773          --# post LogFileEntries(CurrentLogFile) =
      774          --#             LogFileEntries~(CurrentLogFile) + 1;
      775  
      776        is
      777           TheFile : File.T ;
      778        begin
      779           TheFile := LogFiles (CurrentLogFile);
      780           AddElementToFile
      781             (TheFile => TheFile,
      782              ElementID    => ElementID,
      783              Severity     => Severity,
      784              User         => User,
      785              Description  => Description);
      786           LogFiles (CurrentLogFile) := TheFile;
      787  
      788           LogFileEntries(CurrentLogFile) := LogFileEntries(CurrentLogFile) + 1;
      789        end AddElementToCurrentFile;

Analyse Tokeneer with the SPARK Toolset. The result of the analysis shows (see below) that the SPARK Toolset has identified no problems with the code in the procedure AddElementToLogFile – the columns False and TO DO are empty.

 659    VCs for procedure_addelementtocurrentfile :
 660    ----------------------------------------------------------------------------
 661          |       |                     |  -----Proved In-----  |       |       |
 662     #    | From  | To                  | vcg | siv | plg | prv | False | TO DO |
 663    ----------------------------------------------------------------------------
 664     1    | start | rtc check @ 781     |     | YES |     |     |       |       | 
 665     2    | start | rtc check @ 782     |     | YES |     |     |       |       | 
 666     3    | start | rtc check @ 788     |     | YES |     |     |       |       | 
 667     4    | start | rtc check @ 790     |     | YES |     |     |       |       | 
 668     5    | start |    assert @ finish  |     | YES |     |     |       |       | 
 669    ----------------------------------------------------------------------------

Step 2: Change the Contract – But not the Implementation

Let us change the postcondition so the procedure’s contract no longer matches its implementation. The SPARK Toolset will then detect the inconsistency. Change the postcondition to specify that the array element LogFileEntries(CurrentLogFile) should be incremented by 10. The modified code is shown below.

      772          --# pre LogFileEntries(CurrentLogFile) < MaxLogFileEntries;
      773          --# post LogFileEntries(CurrentLogFile) =
      774          --#             LogFileEntries~(CurrentLogFile) + 10;
      775  

Step 3: Re-Analyse and Study the Results

Re-analyse Tokeneer. The results of the analysis for the procedure AddElementToLogFile has changed – the columns False and TO DO are no longer empty (see below).

 659    VCs for procedure_addelementtocurrentfile :
 660    ----------------------------------------------------------------------------
 661          |       |                     |  -----Proved In-----  |       |       |
 662    #    | From  | To                  | vcg | siv | plg | prv | False | TO DO |
 663    ----------------------------------------------------------------------------
 664     1    | start | rtc check @ 781     |     | YES |     |     |       |       | 
 665     2    | start | rtc check @ 782     |     | YES |     |     |       |       | 
 666     3    | start | rtc check @ 788     |     | YES |     |     |       |       | 
 667     4    | start | rtc check @ 790     |     | YES |     |     |       |       | 
 668     5    | start |    assert @ finish  |     |     |     |     |  YES  |       | 
 669    ----------------------------------------------------------------------------

The SPARK Toolset has identified a potential problem with the code. The problem is that the procedure’s contract and implementation don’t match.

Step 4: Revert the Contract – But Change the Implementation

Undo the changes to the postcondition and change line 788 so that the input value is preserved.

     774           --# pre LogFileEntries(CurrentLogFile) < MaxLogFileEntries;
     775           --# post LogFileEntries(CurrentLogFile) =
     776           --#             LogFileEntries~(CurrentLogFile) + 1 
                     ...
     791           LogFileEntries(CurrentLogFile) := LogFileEntries(CurrentLogFile) + 0;
     792        end AddElementToCurrentFile;

Re-analyse Tokeneer. The analysis of the procedure is unchanged, as the implementation, like previously, does not match the procedure’s contract.

Step 5: Strengthen the Contract

Revert the code to its original state.

In SPARK, we can strengthen the procedure’s contract and say more about the properties of the procedure. Let’s add extra code assigning the value 10 to the first element of the array LogFileEntries if CurrentLogFile is not LogFileIndexType’First (lines 789 – 791 below).

      788         LogFileEntries(CurrentLogFile) := LogFileEntries(CurrentLogFile) + 1;
      789         if CurrentLogFile /= LogFileIndexType'First then
      790            LogFileEntries(LogFileIndexType'First) := 10;
      791         end if;
      792      end AddElementToCurrentFile;

Re-analyse Tokeneer and notice that no errors are reported, as the procedure’s implementation is not inconsistent with its contract. The postcondition says nothing about the effects of the procedure on any of the array elements except the one indexed by CurrentLogEntry.

We can strengthen the postcondition (lines 775 and 776 below) to specify that only the entry indexed by CurrentLogFile is incremented and all other elements remain unchanged.

 772   --# pre LogFileEntries(CurrentLogFile) < MaxLogFileEntries;
 773   --# post LogFileEntries = LogFileEntries~[CurrentLogFile => LogFileEntries~(CurrentLogFile)+1]; 

Re-analyse Tokeneer and notice, on lines 659 to 671, that the mismatch between the implementation and contract has been detected.

 657    VCs for procedure_addelementtocurrentfile :
 658    ----------------------------------------------------------------------------
 659          |       |                     |  -----Proved In-----  |       |       |
 660     #    | From  | To                  | vcg | siv | plg | prv | False | TO DO |
 661    ----------------------------------------------------------------------------
 662     1    | start | rtc check @ 783     |     | YES |     |     |       |       | 
 663     2    | start | rtc check @ 784     |     | YES |     |     |       |       | 
 664     3    | start | rtc check @ 790     |     | YES |     |     |       |       | 
 665     4    | start | rtc check @ 792     |     | YES |     |     |       |       | 
 666     5    | start | rtc check @ 794     |     | YES |     |     |       |       | 
 667     6    | start |    assert @ finish  |     |     |     |     |       |  YES  |
 668     7    | start |    assert @ finish  |     | YES |     |     |       |       | 
 669    ----------------------------------------------------------------------------

Summary

This Gem demonstrates that the more precise the specification, the more bugs the SPARK Toolset can detect. The use of the SPARK Toolset during development to verify code is, in our experience, more effective than compiling and testing, since the analysis is for all input data and not just a few specific test cases. In the next Gem we will see how SPARK deals with overflow errors.

Input validation ensures that your program’s input conforms to expectations – for example, to ensure that the input has the right type. But validation requirements can be much more complicated than that. Incorrect input validation can lead to security and safety problems since many applications live in a “hostile” environment and the input might be constructed by an attacker. “It’s the number one killer of healthy software…” according to the CWE/SANS list of the top twenty-five most dangerous programming errors.

For example, consider the following few lines of code from the original release of the Tokeneer code:

         233      if Success and then
         234         (RawDuration * 10 <= Integer(DurationT'Last) and 
         235          RawDuration * 10 >= Integer(DurationT'First)) then 
         236         Value := DurationT(RawDuration * 10);
         237      else

This code has a check that the input RawDuration is in the right range before the value is updated – an example of so called defensive coding, according to the advice from the software experts who compiled the list of dangerous programming errors. Can you see the problem with this code?

The SPARK tools will identify a serious defect in this code, which could impact on security.

In this Gem, the above code will be investigated using the SPARK tools. This Gem shows the challenges in ensuring the absence of input validation errors, and the benefits of using SPARK to do so.

First, we will familiarize ourselves with two more advanced SPARK tools by running them on the correct version of the code. Then we inject the defect shown above into the Tokeneer code, rerun the SPARK tools, and interpret the results.

Step 1: Analyse the Correct Version of the Code

The correct lines of code are the following:

         233      if Success and then
         234         (RawDuration <= Integer(DurationT'Last) / 10 and 
         235          RawDuration >= Integer(DurationT'First) / 10) then 
         236         Value := DurationT(RawDuration * 10);
         237      else

Analyse Tokeneer using the following steps:

* Run the Examiner on the file configdata.adb.
* Run the Simplifier, a more advanced tool in the SPARK tool suite, which tries to show the absence of certain run-time errors by theorem proving.
* Run POGS, which gives a summary of the verification just performed.

Now let us inspect the verification summary, where an overall summary of the Tokeneer verification is given. It shows that there are no errors which is expected as we ran the tools on the correct version of the code. Furthermore, it shows a number of tables describing details for the verification. For example, lines 2750 to 2766 of core.sum, shown below, are the results from the analysis of the procedure ReadDuration.

2750    VCs for procedure_readduration :
2751    ----------------------------------------------------------------------------
2752          |       |                     |  -----Proved In-----  |       |       |
2753     #    | From  | To                  | vcg | siv | plg | prv | False | TO DO |
2754    ----------------------------------------------------------------------------
2755     1    | start | rtc check @ 220     |     | YES |     |     |       |       | 
2756     2    | start | rtc check @ 221     |     | YES |     |     |       |       | 
2757     3    | start | rtc check @ 221     |     | YES |     |     |       |       | 
2758     4    | start | rtc check @ 233     |     | YES |     |     |       |       | 
2759     5    | start | rtc check @ 237     |     | YES |     |     |       |       | 
2760     6    | start | rtc check @ 243     |     | YES |     |     |       |       | 
2761     7    | start | rtc check @ 243     |     | YES |     |     |       |       | 
2762     8    | start |    assert @ finish  | YES |     |     |     |       |       | 
2763     9    | start |    assert @ finish  | YES |     |     |     |       |       | 
2764     10   | start |    assert @ finish  | YES |     |     |     |       |       | 
2765     11   | start |    assert @ finish  | YES |     |     |     |       |       | 
2766    ----------------------------------------------------------------------------

Note that the columns False and TO DO are empty, which means that the SPARK tools found no errors in the verification of procedure ReadDuration.

Step 2: Inject Erroneous Input Validation Code

Now, replace lines 234 and 235 in the file configdata.adb with the erroneous code:

         233      if Success and then
         234         (RawDuration * 10 <= Integer(DurationT'Last) and
         235          RawDuration * 10 >= Integer(DurationT'First)) then
         236         Value := DurationT(RawDuration * 10);
         237      else

Essentially this code concerns the validation of an input – an integer value RawDuration – that is read from a file, and is expected to be in the range 0..200 seconds before it is converted into a number of tenths of seconds in the range 0..2000.

Step 3: Re-Analyse the Faulty Code

Re-analyse Tokeneer.

The results from the analysis of the procedure ReadDuration is shown below.

2750    VCs for procedure_readduration :
2751    ----------------------------------------------------------------------------
2752          |       |                     |  -----Proved In-----  |       |       |
2753     #    | From  | To                  | vcg | siv | plg | prv | False | TO DO |
2754    ----------------------------------------------------------------------------
2755     1    | start | rtc check @ 220     |     | YES |     |     |       |       | 
2756     2    | start | rtc check @ 221     |     | YES |     |     |       |       | 
2757     3    | start | rtc check @ 221     |     | YES |     |     |       |       | 
2758     4    | start | rtc check @ 233     |     |     |     |     |       |  YES  | 
2759     5    | start | rtc check @ 237     |     | YES |     |     |       |       | 
2760     6    | start | rtc check @ 243     |     | YES |     |     |       |       | 
2761     7    | start | rtc check @ 243     |     | YES |     |     |       |       | 
2762     8    | start |    assert @ finish  | YES |     |     |     |       |       | 
2763     9    | start |    assert @ finish  | YES |     |     |     |       |       | 
2764     10   | start |    assert @ finish  | YES |     |     |     |       |       | 
2765     11   | start |    assert @ finish  | YES |     |     |     |       |       | 
2766    ----------------------------------------------------------------------------

Notice that this time there is one YES in the TO DO column. The SPARK Toolset has detected a potential problem with the procedure ReadDuration.

Step 4: Investigate the Verification Output

Now, let us have look into what the error that the SPARK tools have found really means.

The file readduration.siv contains the Simplifier’s analysis of the procedure. Lines 38 to 55 (shown below) of the file show the potential problem the SPARK Toolset has identified. The Simplifier, on line 54, is trying to check no arithmetic overflow errors will occur when evaluating the expression RawDuration * 10 – that is, when Success is True then RawDuration * 10 >= -2147483648 (Integer’First) and RawDuration * 10 <= 2147483648 (Integer'Last).

38      procedure_readduration_4.
39      H1:    rawduration__1 >= - 2147483648 .
40      H2:    rawduration__1 <= 2147483647 .
    ... 
52             ->
53      C1:    success__1 -> rawduration__1 * 10 >= - 2147483648 and rawduration__1 * 
54                10 <= 2147483647 .
55      

Here the SPARK theorem prover is trying to prove that RawDuration times 10 is within the limits of Integer, assuming only that it was within the limits before it was multiplied by 10. This should not be possible to prove. Think about a scenario where Tokeneer was given an input floppy disk where RawDuration was set to 1000000000. Both the assumptions H1 and H2 would be true, but C1 – the conclusion – would be false!

This is a serious defect since a malicious user holding the “security officer” role can deliberately attack the system by supplying a file that contains a malformed configuration data file – one that contains a value for RawDuration that is greater than Integer’Last/10.

Summary

In SPARK, developers need to be explicit about the intended input and output of program components. This has the benefit of the SPARK tools being able to automatically find defects that are hard to prevent, hard to detect, and with important security consequences. In this Gem we have seen this exemplified with input validation. In the next Gem, we will look into SPARK contracts.

Every statement should have a purpose. An ineffective statement has no effect on any output variable and therefore has no effect on the behaviour of the code. The presence of ineffective statements reduces the quality and the maintainabiliy of the code. The SPARK Toolset identifies all ineffective statements.

In this Gem, we show how the SPARK Toolset finds ineffective statements to ensure that SPARK programs are free from them. We will inject an ineffective statement into the Tokeneer code and use the Examiner to locate it.

Step 1: Inject an ineffective statement

We will inject an ineffective statement into the implementation of the function NextListIndex in auditlog.adb.

       189  function NextListIndex(Value : LogFileIndexT) return  LogFileIndexT
       190  is
       191     Result : LogFileIndexT;
       192  begin
       193     if Value = LogFileIndexT'Last then
       194        Result := LogFileIndexT'First;
       195     else
       196        Result := Value + 1;
       197     end if;
       198     return Result;
       199  end NextListIndex;

Let’s modify the above code by adding the new variable Result_Tmp of type LogFileTypeT (line 191) and change line 196 so that Value + 1 is assigned to the variable Result_Tmp instead of Result (see code below).

       189  function NextListIndex(Value : LogFileIndexT) return  LogFileIndexT
       190  is
       191     Result, Result_Tmp: LogFileIndexT;
       192  begin
       193     if Value = LogFileIndexT'Last then
       194        Result := LogFileIndexT'First;
       195     else
       196        Result_Tmp := Value + 1;
       197     end if;
       198     return Result;
       199  end NextListIndex;

The statement on line 196 is ineffective because Result_Tmp is never used. Furthermore, the value for Result may be undefined when Value /= LogFileIndexT’Last.

Step 2: See how the Examiner finds the problem

Run the Examiner for auditlog.adb and, as expected, it finds the ineffective statement as well as the possible undefined value for Result.

auditlog.adb:196:10:
  Flow Error  10 - Ineffective statement.
auditlog.adb:198:14:
  Flow Error 501 - Expression contains reference(s) to variable Result, which may have an undefined value.
auditlog.adb:199:8:
  Flow Error  33 - The variable Result_Tmp is neither referenced nor exported.
auditlog.adb:199:8:
  Flow Error 602 - The undefined initial value of Result may be used in the derivation of the function value.

Summary

In this Gem we have seen an example of the SPARK tools finding an ineffective statement. The SPARK tools will find all ineffective statements. In the next Gem we will study Input Validation.